Wikipedia:Wikipedia Signpost/2012-05-28/Technology report

Technology report

Developer divide wrangles; plus Wikimedia Zero, MediaWiki 1.20wmf4, and IPv6

English Wikipedians discuss editor–developer divide

A minor change—tweaks to the default heading used at the top of diff pages—provoked a long debate on the English Wikipedia when it went live this week. The discussion focussed on an issue that has bubbled to the surface intermittently for the past few years: as the MediaWiki developer base professionalises, are developers becoming less responsive to English Wikipedian demands?

Most developers would agree that editors of the English Wikipedia are given less priority than they used to be. There are overtly more projects than there used to be and more languages to support on each of them. Staff development projects are far more likely to target "newbie" editors than existing stalwart editors (a decision that seems to have significant support given this week's poll results, below); design choices are increasingly being made in the name of helping the former, potentially at the expense of upsetting the latter. Needless to say, decisions that fit such a paradigm (including the recent diff colours switchover) have not proved universally popular.

“	The horrific technological inertia that is developing within the community is only going to lead to two possible outcomes ... Either the developers abandon any hope of satisfying the community and stop bothering to even try to engage with it, or they stop trying to develop beneficial features at all.	”
—Happy-melon

Ultimately, a number of viewpoints emerged from the resulting discussion. They centre on two questions: firstly, whether developers are targetting the "wrong" things, and secondly whether they should be expected to communicate the changes they have made better. Both have proved to be contentious issues. Equazcion, in proposing the former, talked of developers implementing their "own whims regarding what is best for the community"; but such a critique relies on a certain view of the community as being a superior judge of what is best for itself and its future members, rather than as an insider group keen to resist any kind of novelty. Moreover, volunteer developers, much like Wikimedians who work in an idiosyncratically narrow area, are likely to resist any attempt to tell them what new features they can and cannot work on, especially since virtually all will have been proposed by some community or other at some point.

The issue on which a consensus is more likely to form revolves around the need for better communication between developers (who frequent the wikitech-l mailing list, MediaWiki.org, and Bugzilla, which, in unrelated news, was down for considerable periods this week) and editors who frequent their home wikis. When pushed for comment on the thread, WMF developer Ryan Kaldari was the first to admit that despite the amount of time WMF developers were putting in to communicating with communities, more could still be done. "Right now", he wrote "there are so many different venues for discussion it's rather unmanagable [and] we have a very hard time getting people to beta test things for us. ... It seems no matter where we advertise it, we generally only get significant community feedback after the features are deployed".

The issue is not restricted to the English Wikipedia, although it is certainly the place that the issue is invoked most frequently. By contrast, members of smaller wikis are more likely to complain not that too many changes are being forced on them, but that rather too few are made—that their many feature requests are simply never acted on because they are neither WMF strategic priorities nor aligned with the personal interests of volunteer developers. The difficulty for WMF development coordinators undoubtedly lies in addressing all of these multifarious complaints simultaneously and without trade-off.

In brief

Signpost poll

Long-term threats

You can now give your opinion now on next week's poll: What's your take on developer–user misunderstandings?

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for many weeks.

MediaWiki 1.20wmf4 begins deployment: The fourth Wikimedia deployment from the MediaWiki 1.20 branch began today, with an additional two weeks' worth of bugfixes and other incremental improvements going live to MediaWiki.org and two test wikis. Among the 230 changes included in the deployment, two are of the small, visible change type that has caused so much controversy in the past couple of weeks: the first excludes immovable namespaces from Special:MovePage's namespace selector; the second broadens Special:Shortpages to include pages from all content (non-talk) namespaces, rather than just article space. All non-Wikipedia sites will be next to receive the changes on or around May 30; the English Wikipedia will receive them on June 4 and other Wikipedias on June 6, major problems notwithstanding.
UploadWizard updated: A series of updates went live to Wikimedia Commons' Upload Wizard this week, including "disabling multiple-file selection in browsers where it didn't work", tweaking the configuration such that the "'skip tutorial' step ... won't come back every time you switch computers any more", and ensuring that "files now start uploading immediately when selected" (wikitech-l mailing list). The changes, which include background support for so-called "upload campaigns" such as Wiki Loves Monuments, have largely been welcomed by the Commons community. It was noted that very large uploads (those greater than the previous limit of 100 MB) continue to fail on a regular basis, a major disappointment for those seeking to take advantage of the new, much greater upload limit of 500 MB.
Character support interview: Localisation team member Gerard Meijssen has used a post on his personal blog to publish an interview with Unicode specialist Michael Everson. In it, Meijssen and Everson discuss the possibilities with regard to improved web-font support (particularly the eradication of the ☐☐☐☐☐☐ ☐☐☐☐☐-phenomenon) for Wikimedia wikis. Everson noted, however, that it could be necessary to establish, either unilaterally or multilaterally, a body concerned with the creation of freely licensed web-friendly typefaces. "Right now", Everson wrote, "anyone viewing any Wikipedia in any language may encounter text in Ol Chiki, or in Runic, or in the simple International Phonetic Alphabet, and pages have to apologize to the reader because their computer may not display the material correctly".
Wikipedia Zero launches in Malaysia: Malaysia has become the first Asian country, and the third country in the world, to have a national mobile provider offer Wikipedia free of charge. According to a post on the Wikimedia blog, "Digi’s 10 million customers can read as many Wikipedia articles as they like (provided they have an internet-capable phone), in any language, through the Opera Mini browser data; WebKitFormBoufree access applies to the lightweight, text-only mobile version of Wikipedia, which Digi customers can now access by going to zero.wiki.x.io". Support for Tunisia and Uganda was brought in earlier this year; further networks will be added later in the year, should negotiations go well; at the very least, Digi is part of the already signed-up Telenor group, suggesting that Telenor's other 115 million customers will soon be enjoying the same service.
Wikimedia wikis to take part in World IPv6 Day: The foundation will attempt to take part in this year's World IPv6 Day, it was revealed this week. Deputy Director Erik Möller, answering a question from the community about the event (scheduled for June 6), outlined how the WMF was "planning to do some limited production testing at the Berlin Hackathon (June 1–3) and on IPv6 Day, but we'll only keep it enabled if the issues are manageable". Last year the foundation was forced to back out at the last minute due to problems with the database schemas of several extensions (see previous Signpost coverage). These have now been updated in readiness for the day, which last year was supported by a long list of major websites including Facebook and Google. Realistically, it has been pointed out that any proper addition of IPv6 support to Wikimedia wikis would require large scale "reeducation" of admins and the rewriting of a considerable number of user scripts, though such a move is regarded as inevitable in the longer term.
One bot approved: 1 BRFA was recently approved for use on the English Wikipedia:

Joe's Null Bot, once-daily application of WP:NULLEDITs special purges with the "forcelinkupdate" option set to each of the articles within Category:BLP articles proposed for deletion by days left.
At the time of writing, 17 BRFAs are active. As usual, community input is encouraged.

← Previous "Technology report"

Next "Technology report" →

In this issue

28 May 2012 (all comments)

News and notes

Recent research

WikiProject report

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Just to note that across the next 2–3 weeks, it may not be possible for me to put out a full Tech Report. If others want to chip in though, that would be cool :) - Jarry1250 ^{[Deliberation needed]} 14:21, 28 May 2012 (UTC)[reply]

User–developer divide is the result of a confused relationship

There's a major question-mark about how the tech side is managed and what relationship it should have with the community. Software development is normally subject to a contract between producer and client; from that central fact emerges the decision-making, the planning, and the timeline for building and refining. Since the whole WMF enterprise rests on the backs of gigantic amounts of skilled volunteer labour, it's a wonder the community isn't regarded as a client in a more standard model, closer to best-practice tech development in the corporate and government sectors.

But rather than software producer for the community-as-client, perhaps the foundation sees itself as providing for a user base without their input during the process, like the Facebook/Yahoo commercial model? This might explain why the tech side seems to be left to its own devices, relatively dissociated from editorial needs.

The current fluffy model seems to emphasise little things. Sue Gardner told the Signpost in January that "The Foundation will lead on technical initiatives such as the visual editor, ... There are other initiatives where we’re partnering with the editing community [such as] the feedback dashboard [and] new page triage." But aside from the upload wizard on Commons, there's been a glaring failure to resolve the big issues; several significant projects almost got there, I'm told, only to be left by the roadside for the next set of goals. For example, the new editing interface looks sound in principle, but I'm not holding my breath this time, and it would be nice to see a bit more of what's going on behind closed doors.

Should the community have a formal say in the tech programs, particularly through an open and inclusive system for prioritising needs on the basis of professional estimates of costs and feasibility? If we're going to talk little things, when has revamping the table and maths-notation syntaxes ever been put out to discussion and even a vote? I'm sure there are lots of other software-related matters affecting daily editing that the community of editors is well-placed to work through given the chance. But that would need a more systemic, client-focused approach to the relationship, in which options, projected budgets, and timelines are explicitly set out, and in which there's regular reporting on progress and hurdles (in layperson's language as well as tech-speak—possibly via this Signpost page).

A close, trusting producer–client interaction is always a bit messy and difficult, but it's the only way to get things done in industry and government. $Tony (talk)$ 03:33, 29 May 2012 (UTC)[reply]

You are not discussing a "user-developer" divide. You are discussing an "editor-developer" divide. There are 500 million users and 90,000 editors. Users don't care about watchlist bolding or anything along those lines. Editors do. The changes to the headline are misleading; I'd change that back to the original wording.--Jorm (WMF) (talk) 05:31, 29 May 2012 (UTC)[reply]

The definition of "user" needs a little work, says User:Tony1. You might have two sets of users on the table.

Tony (talk)

05:42, 29 May 2012 (UTC)[reply]

So why confuse the issue? "User" is a set of people who contain all people who use the encyclopedia. That's 500 million people. Of them, we then classify 499,910 of them as "readers" and 90,000 as "editors". This discussion is about "developers and editors", not "developers and readers". Please fix the headline to be more accurate and NPOV.--Jorm (WMF) (talk) 05:45, 29 May 2012 (UTC)[reply]

I don't see confusion, and "user" is the term that Jarry, the regular journalist for this report, chose.

Tony (talk)

05:50, 29 May 2012 (UTC)[reply]

Well, it appears to have been your edit that made the change. If you were directed to make this change, I apologize for implying that it was your decision. Either way: my point stands. I'll take this up with Jarry.--Jorm (WMF) (talk) 05:54, 29 May 2012 (UTC)[reply]

I wasn't "directed" to do anything. Jarry had already used "user" twice in the text, including in the caption of the poll results. The original "developer divide" raised the question of "divide between what or whom?" It needed to be fixed.

Tony (talk)

Then perhaps Jarry needs to correct his captions, as they are misleading. --Jorm (WMF) (talk) 06:05, 29 May 2012 (UTC)[reply]

I believe this attempt to categorise only the readers as users, in disregard of the 90K editors, is part of the problem. Editors are users in any normal sense of the software development/technical process. I've just visited your talk page to find this statement to one of our editors: "... the primary thing that can be done is to understand that the "editors" are not the bosses of the "developers". That seems to be a common misconception: that the editor community says "jump" and the developer community asks "how high?" The developer community – both paid staff and volunteer – is not a collection of serfs." I'd be hoping for something less adversarial than this us-and-them frame. The key to the foundation's technical process is that users fall into at least two categories. No one's pretending to be "boss", by the way. $Tony (talk)$ 06:15, 29 May 2012 (UTC)[reply]

I think you're taking a different discussion out of context to try to make some point here, which I'm not sure is correct, and my comments there don't support your argument. You seem to be willfully conflating 499.9 million readers with 90k editors, calling them "users", and implying that both have (or should have) equal voices - which is what I am stating is wrong. 499.9 million users do not have a voice, and should not be included in the calculation of any "developer vs. anyone else" divide. Say what you will about the developers (both volunteer and Foundation staff) and tensions with the editor community, but please leave the reader community out of it. Since "users" is a superset, please reduce to simply "editors". --Jorm (WMF) (talk) 06:22, 29 May 2012 (UTC)[reply]

Jorm, perhaps you're willing to respond to the substantive points in the article and in my comments here, rather than arguing about the semantics of "user".

Tony (talk)

06:24, 29 May 2012 (UTC)[reply]

I'd be willing to respond to substantive points when the headline is not biased, I think. It's difficult to want to engage in conversation when it appears, via headline, that the outcome is already decided.--Jorm (WMF) (talk) 06:26, 29 May 2012 (UTC)[reply]

But just a quick point: the definition of the word "user" is very much at issue here. One must understand that the development teams serves the whole "500 million", and not exclusively the 90k.--Jorm (WMF) (talk) 06:28, 29 May 2012 (UTC)[reply]

(responding to no-one in particular) I actually used the phrase "user-developer divide" (or indeed referred to "users" at all) once in the whole article, in the poll caption, in order to avoid excluding any potential readers who don't edit from answering the poll. The focus of the main story is clearly open to questions of insider-outsider mentalities between editors and users (potential new editors), which I made reference to during it. I've updated the subtitle on the main story. - Jarry1250 ^{[Deliberation needed]} 09:51, 29 May 2012 (UTC)[reply]

The substantive issue that Tony1 raises is important, and has needed addressing for some time. As always, WP has entrusted its editors to make decisions on behalf of its readers, and accordingly WP's editors should be the driving force for all software changes that affect readers. (As background: I've been in the IT industry for over two decades and have written a large amount of code for both large and small organisations; and I've also created a number of functional requirements documents that have led to successful software development.)

At the highest level, software development is initiated by a requirements specification which then leads to the (very important) functional specification (which clearly defines what the programmers must build, and provides a basis for building test cases). While venues such as the Village Pump allow high-level requirements to be suggested and debated, I've always felt that the project's editors have been marginalised from the important task of creating functional specifications. Of course, the programmers must be involved at all levels as they are after all editors, and must assist to keep the specifications technically realistic. The construction of detailed specifications prior to development will also assist those who are in a position to prioritise development work.

WP has a wide variety of editors: some of whom will suggest changes to the software, and many of whom will get involved with a consensus-driven approach to figuring out a good set of functional requirements. I know that figuring out the process (by which good consensus-driven specifications can be achieved) will be hard to define, however I'm convinced that a better result will eventuate for WP when it has been.

Please note that I admire and respect the work of WP's programmers and don't intend any of the above as an attack on their commitment and hard work. More than ever, WP needs their dedication—only hopefully (in our more mature WP), that dedication will be driven by the needs of the users.

GFHandel ♬ 22:06, 29 May 2012 (UTC)[reply]

"the project's editors have been marginalised from the important task of creating functional specifications" = "the project's editors have marginalised themselves from the important task of creating functional specifications"? (Just playing devil's advocate here) - Jarry1250 ^{[Deliberation needed]} 11:12, 30 May 2012 (UTC)[reply]

If it's to be a blame-game, can you say how they've marginalised themselves? I'm not aware of marginalising myself, for example. Tony (talk) 13:29, 30 May 2012 (UTC)[reply]

I'm just going to mention two extensions, and we can discuss why the community writing specs is almost always a terrible, terrible idea: FlaggedRevs and LiquidThreads. FlaggedRevs is the poster child for how design by committee does not work in software. Go back and read the background on that. The idea of LiquidThreads is awesome, but the spec was incredibly bloated. Also, the software design approach you are describing is waterfall, is it not? Waterfall techniques have high failure rates and lead to bloated software. --Ryan lane (talk) 14:07, 30 May 2012 (UTC)[reply]

I don't accept your examples as demonstrating the point you are trying to make (however if you want to provide specific links that give more detailed examples, I'll be happy to read further). On the contrary, flagged revisions has been adopted on at least 16 wikis around the world, and the concept's evolution (Pending Changes) is making a resurgence (with at least 63% support in a current RfC).

Understanding the requirements and documenting the functionals is the only realistic way of producing software that can effect 500 million users (not to mention 90,000 editors).

I also don't believe that an appropriate methodology for the community contributing towards proper "business" requirements has ever been implemented. One really good example of this is date formatting—which was something that seemed okay at the (programmer's) time, but was overwhelmingly rejected when put to the community (after years of debate). Programmers are not necessarily experts in any of the requirements they are implementing, so it is not appropriate that business-related decisions could be left up to them (often at the time of coding) that should previously have been decided by community consensus. I don't believe it would be the most difficult problem in the world for Wikipedia to borrow/develop/adapt a methodology that documents the functionals of a proposed development (e.g. some form of Use case documentation).

Anyhow, nothing is going to be solved here; but I'm very willing to throw my hat in the ring if/when the time comes for Wikipedia to move to a more professional and mature way of handling (at least the beginning of) the software engineering life-cycle.

GFHandel ♬ 02:37, 31 May 2012 (UTC)[reply]

Comment: What surprises me is the dismissiveness, negativity, and barely concealed hostility expressed by technicians towards editors, and by proxy towards all users. At the very least, it's blame the editors, not us:

Happy-Melon, for example, talks of "the horrific technological inertia that is developing within the community". Excuse me? He continues in a frame that is almost a threat (and note the irritable "bothering"): "[this] is only going to lead to two possible outcomes ... Either the developers abandon any hope of satisfying the community and stop bothering to even try to engage with it, or they stop trying to develop beneficial features at all." He is speaking as a volunteer, I suppose, so he's on less wobbly moral ground when venting; but the foundation employs a whole technical department.

Among those technicians, Jorm, above, has chosen to employ a bossy demeanour, point-blank refusing to discuss the substantive issues until the journalist accedes to his demand that "user" be changed to "editor" in the article. He's got his way on the semantics, but we've heard nothing more from him.

In this environment, Jarry seems to have taken to doing the technicians' bidding for them: "the project's editors have marginalised themselves", he says.

The blame-the-editors line continued with Ryan lane: "writing specs is almost always a terrible, terrible idea ... design by committee doesn't work ... incredibly bloated". No self-examination of how our salaried technicians might engage more productively with the community? Why bother: it's always editors' fault, not the technicians'. Tony (talk) 03:53, 31 May 2012 (UTC)[reply]

We all just flew into Berlin for the hackathon. We're not ignoring the discussion, we're just recovering from our flights.

I'm fine with being called dismissive of the idea that we should be developing specifically to the community's specs. It's been proven that it doesn't work. FlaggedRevs took years, went through numerous total redesigns, and as such is a very difficult piece of software to use, configure, and develop. It's a great example of design by committee. Please read that article; here's a choice quote: "One maxim is that a camel is a horse designed by committee." Also, we don't do waterfall style development. We do agile.

With Agile, the community can easily be part of the process by writing user stories. Maybe we need better ways to communicate, but it's really difficult to do so with every single community we need to support (we'd need to track hundreds of talk pages on hundreds of projects - we can't do this). Ideally the communication would occur in one place, where the projects are being managed (currently mediawiki.org). --Ryan lane (talk) 10:27, 31 May 2012 (UTC)[reply]

What I was quoted on (and while I don't mind the quote and stand by it, it would have been nice to be informed that I had been quoted) is not a threat, as there is nothing there that I personally am going to 'inflict' upon the enwiki community if they don't change their ways; but it is a fairly unappealing portent.

I don't think anyone in the developer community is dismissive or hostile towards editors, either in general or as a community. There is certainly a substantial problem with the way the two communities communicate. FlaggedRevs is the archetypal example not just for its design process but also its social implementation. Designed by the communities, for the communities; yet four years later this community (which was amongst the loudest voices calling for the implementation) can't make up its mind on deploying it (can't, it seems, even make up its mind about how to make up its mind), and seems on the verge of discarding thousands of hours and tens of thousands of dollars worth of work which it demanded be done. Fine, no problem; a quarter or so of the rest of WMF's editor community is able to communicate efficiently with the developers and sysadmins and are perfectly happy with the system. Why should the sysadmins spend so much time and energy dealing with grumpy, cantankerous enwiki, when the other wikis are so much easier to work with?

Of course you'd (rightly) say that FlaggedRevs never developed an adequate consensus on enwiki, and the problems that highlights with our consensus model are independent of the editor-developer relationship; and that's partly true. But remember that the developer community, and particularly the sysadmins, operate on a very different decision-making model and have relatively little experience of how the consensus model works. Decision-making amongst developers is much more hierarchical; there are a whole tier of senior developers whose opinion simply matters more than that of more junior devs; and of course amongst the paid staff and sysadmins the hierarchy is even clearer. When communicating with developers, they're not looking for a consensus they can judge, they're looking for a clear statement of what to do from someone who has the authority to speak on behalf of the editor community when doing so. Of course in judging that they can and do note the structure of the wiki involved; but it is not the developers' or sysadmins' role to either judge consensus for a change, or to judge whether an admin has correctly judged the consensus.

Rather, it is up to the community to establish a process by which they can 'decide' to make a change (in our case, discussion-based consensus, for dewiki a much more vote-based poll, and for some wikis an absolute straight vote; this is in place), establish a procedure for agreeing when a decision has been made, and establish a procedure for communicating that decision to the developers. Enwiki is substantially lacking in both of the latter two areas. Currently there is no policy or procedure explaining to the sysadmins what should be considered a change "wanted by enwiki"; if enwiki saw fit to create one, whether that be "all changes must be endorsed by 20 editors in an RfC closed by an impartial admin and reported by anyone to bugzilla", "there is a BAG-esque panel responsible for judging the consensus for shell requests and reporting them, and no other changes should be considered" or "all changes must be proposed and requested by ArbCom", the sysadmins would be delighted to have that clarity. But when 600 editors can't demonstrate a consensus for a FlaggedRevs configuration, and someone proudly presents a "consensus" of 20 editors which is then thrown back in the sysadmins' face, you can't blame the sysadmins for concluding that enwiki Simply Doesn't Know What It Wants. And in that case, why should they waste volunteered or paid-for-by-donations time trying to work it out? Happy‑melon 11:52, 31 May 2012 (UTC)[reply]

If anyone is interested in seeing the requirements building and functional specs that the WMF is working on, they are all on public wikis and are editable -- though I wouldn't follow BRD, rather, discuss first if it's not minor. If you want to comment on and help build requirements, your help is most welcome on mediawiki.org for most software and Meta for research and analysis. (Specifications are not on English Wikipedia because there are hundreds of wikis to serve with MediaWiki development.) In particular, editors are probably interested in the visual editor and editor engagement projects, all of which have public specifications. Steven Walling (WMF) • talk 18:02, 31 May 2012 (UTC)[reply]

Since you mentioned the visual editor project; that project has a Software design page (that was started 60 weeks ago) and even a Sandbox page where a prototype can be tested, however the section under "User requirements:" is empty. Now I know that no one is going to suggest that there exists a development methodology that shouldn't have requirements (Agile included), so could someone please add something (including links) under "User requirements:". Thank you. GFHandel ♬ 00:15, 1 June 2012 (UTC)[reply]

Arbitrary section break

What worries me is less the management of new feature and tool creation, which seems to dominate almost all discussion and mindshare; but rather the apparent almost total lack of resources and management given to existing features which fall short, even when they impact directly on user -- and reader -- experience. For example, if you look at the archives of Commons:Graphics village pump or WP:SVG Help, it is clear that the brokenness of our SVG renderer causes immense trouble to editors, and sub-standard output to users, with problems known about for at least seven years that could be cured by switching to a different back-end renderer, yet central interest seems always to be exclusively focussed on whatever pretty new bauble somebody's dreamed up, rather than nuts-and-bolts technical issues like this that directly affect the quality of the output that WP can serve up.

Or to take another example, a particular personal sore point, consider the bug that means every UK geographical grid co-ordinate served by GeoHack is 200 metres out. So, as recently pointed out try to use the WP's link to Streetmap.co.uk, or the 19th century old-maps.co.uk site for e.g. White_Tower_(Tower_of_London) and you get dumped in the middle of the moat. This has been known about for at least four years, including precisely why it occurs and what needs to be done to fix it, even including links to drop-in replacement code that would fix it, all filed four years ago, and repeatedly raised since. Yet in all that time... nothing. And meanwhile every single UK National Grid co-ordinate reference we output is wrong; every map service we link to that uses them points to the wrong place; and every user that tries to add coords using those services may give up because WP won't point to the right place. I suppose it's all considered less important than what new colour we can turn the most recent changes on my watchlist.

Similarly just read the immense frustrations at the Maths project about the delays in implementing MathJAX.

Despite WP's multimillion dollar income, there simply appears to be no management or resourcing to fix issues of basic functionality, that affect directly the quality and even basic accuracy of what we serve our end readers. Jheald (talk) 18:36, 31 May 2012 (UTC)[reply]

I am not surprised that it is hard to get volunteer developers to work on fixing problems with existing features rather than on new features. SW engineers ALWAYS prefer new feature development to maintenance coding. It should be no surprise that volunteer developers are as catlike as volunteer editors when it comes to herding them. I would expect it to be easier to focus the paid developers with the help of the Wikimedia foundation, but I suspect that to many of the board members Wikipedia (especially the English language Wikipedia) is no longer a shiny new toy, and thus not of much interest. Of course it could also be that they find the community here to be incredibly difficult to work with, and I sympathize. Rusty Cashman (talk) 19:10, 31 May 2012 (UTC)[reply]

Let's get to the meat of the matter. I'm afraid I don't sympathise: the technicians and the foundation have made little attempt to develop effective processes for engaging with the community on the technical needs of editing. Instead, the professional salaried technicians we pay resort to defeatist and self-serving mantras about how the community gives a billion different opinions and doesn't understand.

Yes, unfocused bloat is what you'll get from the community if you don't go about it in a smart, structured way. There are procedures for honing diverse opinions into priorities for action, by successively pruning a billion ideas into a manageable number based on iterative back-and-forths by a team of technicians and, possibly, a few elected editorial reps. It makes sense for this to happen in batches, to deadlines.

Nothing can be done without connecting the dotted lines between (a) the resources the foundation is prepared to allocate to a set of community-prioritised projects over each set time-interval, (b) professional initial estimates of the timeframe and labour-hours required for the candidate projects that float some way up to higher priority. So we need a noticeboard or some other forum on en.WP where editors can suggest improvements, the technicians can use their expertise to dampen or encourage expectations where indicated, and the community can discuss and !vote on how suggestions can be prioritised in the light of perceived importance, technical feasibility, relative cost, and other technical opinion.

Without structured community liaison by a technical department whose employees seem to be out of touch with the demands of regular editing, technicians will gravitate towards strategies that produce only patchy improvements to the basics, plus blue-sky toys of marginal utility. We have ample evidence of this misallocation just above, and there's much much more I'm sure editors will document here if asked: basic functionalities have been allowed to rot for years.

A few major points in the tech department's 2012–13 goals are relevant:

"We will connect with the user" [apparently referring to the editorial community; btw, Jorm, it uses the term "user"—your contributions to this talk page have comprised much text solely objecting to the use of that word in the article draft.]
"We can't afford review bottlenecks. Whether it's feedback on features or contributions of code, the process of triaging, responding to, or acting upon any kind of community contribution should be an open one – and we need to actively work to grow communities of volunteer responders, liaisons, code reviewers, and so on. If we become the bottleneck, we will always fall behind, no matter how responsive we aim to be." [That's a good one. Let's talk about how.]
"We must reduce our technical debt. ... improvements to team culture and team collaboration, are equally important." [Yup.]

While the tech department is in Europe at our expense attending the Hackathon, they might think carefully about the need to get more serious about the workaday stuff like maintenance coding—like clearing the backlog of thousands of items and working on the improvement of existing functionalities on the basis of effective negotiations with the community. A better-oiled process for liaising with the other language WPs might be possible after we get it right on en.WP; and let's not forget the 30,000 or so registered users of the wiki software elsewhere. Tony (talk) 03:10, 1 June 2012 (UTC)[reply]

You speak a lot of sense here, Tony; I disagree on a few principles, but agree on all the substantive conclusions.

The situation is massively complicated by the fact that there are numerous different groups at play here which are often confused but are in fact very diverse. The "developers" are the people who write MediaWiki core code and extensions; the majority are volunteers but there is a large cohort of people employed by the WMF. The "sysadmins" are responsible for 'operations' (maintenance of the cluster hardware and software) and for deploying configuration changes; with I think only one volunteer exception, these are all staff. The issue Jheald raises above with GeoHack has nothing to do with either of these groups, but is with a Toolserver user; who are a separate (volunteer) group again who are very much a law unto themselves.

While I fervently support the right of all of those volunteers to work on whatever they choose to donate their time to, I've felt for years that the Foundation should be devoting more of its paid programmer resources towards bugfixes and less towards "blue-sky" projects. But that's a somewhat tangential issue, because blue-sky projects can succeed or fail with the community just like little changes: Collection (PDF book renderer) is an example of a project that utterly bombed with enwiki, while AbuseFilter and UploadWizard are examples of big projects that have gone much more 'right'. And regardless, the editor community needs to build communications with the developers as a whole community, if they want to have any influence in shaping the overall structure of MediaWiki rather than just the contributions of individual developers.

Pretty much everything else I agree with. I think enwiki needs a separate noticeboard purely for discussing software changes, and a clear procedure for managing, triaging and evaluating discussions there. The developer community needs to engage with that noticeboard; I'm pretty confident that they will be effective at doing so if it has a good signal-to-noise ratio. And the developers, led primarily by the WMF staff, need to provide a realistic evaluation of how viable and what priority an approved request is, and what progress is made towards realising it. The emphasis, from both sides, is on "realistic"; as I've said before elsewhere enwiki needs to have a realistic view of how much time it is 'entitled' to (none at all from the volunteer developers, and not as much from the staff developers as it generally tends to expect). But as you say, having a clearer picture of how the relationship works will allow both sides to better prioritise their contributions, and hopefully help the interaction produce less heat and more light. Happy‑melon 09:44, 1 June 2012 (UTC)[reply]

A lot of these issues are bottlenecked on operations. Until very recently we've had basically no employees. We're making great strides in catching up with the technical debt that's been collecting over the years. Also, we're putting a lot of effort into scaling our operations by opening it up to the community, like MediaWiki is. If you'd like to help with these problems, please join us in Labs. In Labs you can fix the issue, demo it, and add the necessary support into our configuration management repository, where we can then code review and deploy it to the production systems. We'd love to have your help! --Ryan lane (talk) 09:21, 1 June 2012 (UTC)[reply]

I rather lost faith in the developers having any meaningful engagement with the community when the community came to a consensus to restrict new article creation to autoconfirmed users, duly filed a bug citing the discussion (which, unusually for such a major proposal, came to a very clear result), and got a middle finger and a "Nah, would rather not" from the devs. If that's the type of "support" we can expect, I think any attempt at engaging the developers is relatively pointless, since they're pretty clearly going to do as they will regardless of what anyone thinks. Seraphimblade ^{Talk to me} 04:51, 5 June 2012 (UTC)[reply]

Yes, a lot of us were pretty shocked at the technicians' "fuck you" attitude to our clear consensus. The issue remains unresolved. Tony (talk) 05:31, 5 June 2012 (UTC)[reply]

The final decision was made by Erik, the Deputy Director, not the developers. On community engagement - I've spent the last six months doing community engagement. I've spent the last four of those months doing community engagement on precisely this issue with the New Pages Feed project. I appreciate that it isn't what some people want (by which I mean "it isn't ACTRIAL") but it is, speaking as a patroller, much better than what we currently have. We can sit here and debate a decision made close to a year ago, a decision that isn't going to change, or we can go "okay, so X isn't happening, but Y is. Lets try and make Y as non-sucky as possible". In my mind, that's far more productive, and I encourage you all to try out the prototype and give us any feedback you can so we can make this as good a tool as it can be. Okeyes (WMF) (talk) 17:22, 5 June 2012 (UTC)[reply]

@Happy-melon: I simply don't understand why we don't have at least something like de:Wikipedia:Projektneuheiten where always new stuff is posted, nearly all day! mabdul 18:07, 5 June 2012 (UTC)[reply]

de.wp has that because Raymond makes it happen; no-one has yet volunteered to recreate the same thing here (although I do follow that page myself). - Jarry1250 ^{[Deliberation needed]} 19:04, 5 June 2012 (UTC)[reply]

The reason we don't have such a system on enwiki is very simple: because no one has made it yet. As Jarry says, Raymond is a German-speaking editor and occasional volunteer developer who devotes time and effort to compiling that noticeboard, and the rest of the developer community and tech staff are happy to engage with that project as much as possible. A couple of people have tried to invest a similar sort of energy and time on enwiki, but have usually had it thrown back in their face by editors taking a shotgun to the messenger. Happy‑melon 09:13, 6 June 2012 (UTC)[reply]

This should be a top priority, H-M; thanks for this information. If I weren't a tech-dummy I'd think of running (or co-running) such a noticeboard. But I'm quite willing to ping those who tried, to encourage them to try again. Tony (talk) 10:16, 6 June 2012 (UTC)[reply]

@Tony: I would help and try to translate Raymond's German msgs ;) mabdul 10:54, 6 June 2012 (UTC)[reply]

@Okeyes: I understand the decision isn't under your control, and that the earth's core will be ice rather than iron before the decision changes. But why would you expect me or anyone to waste my time giving input again, when the community's clear input on this matter has already been ignored? The interface tweaks are not going to solve the problem, but they're what WMF decided to do, and so regardless of community input, that's apparently what we get. We see a report saying that there's a disconnect between editors and WMF, and well, for a lot of us (certainly for me), that issue and others like it were the reason why. This is ultimately WMF's project, and they certainly have the ability to ignore any consensus of the community they like, but they shouldn't disenfranchise the community and then wander around wondering why the community feels disenfranchised and why more editors leave all the time. If you'd rather not hear that, no one's making you listen to it. No one has thus far. Seraphimblade ^{Talk to me} 01:22, 6 June 2012 (UTC)[reply]

I listen to things all the time :). It's not "interface tweaks"; it's an entirely new setup with much more metadata, many more filtering options and a built-in twinkle-esque system that includes a tutorial for new patrollers. I appreciate it might not look too impressive now; it's still a half-finished prototype, but I wanted to push it out to enwiki as soon as possible because (quite frankly) I'm tired of seeing projects where the devs fling it on a temporary prototype server nobody is familiar with and are suprised when they don't get any feedback.

And if I had to give a reason why you should trust us to listen on this, I'd say, well, because we're listening. Talk to Utah or Bensin or Tom Morris or any of the other editors who have been knee-deep on giving us feedback on the Article Feedback Tool or New Pages Feed. We, or at least I, am listening. One of the reasons I was hired was to try and help rebuild the bridge between the community and the foundation, and that's why we've been doing all this project-centric feedback - it's why you have the opportunity to play with the software and criticise it here before it's deployed, and why you have someone to directly tell you're disappointed. But without feedback on what would make it better and a way forward, the sad fact is it isn't going to make much difference, because we can't infer improvements from "I don't like it". I understand if you don't want to give us another chance on this front, but I hope you will, because if you talk to the aforementioned editors you'll find out that we have been trying our hardest. Okeyes (WMF) (talk) 20:52, 6 June 2012 (UTC)[reply]

You, in particular, I'll give credit for trying awfully hard. Here's hoping! Seraphimblade ^{Talk to me} 21:14, 6 June 2012 (UTC)[reply]

Thanks! :). I do my best. Hopefully we can rebuild; I'm under no illusions that it'll take time. Okeyes (WMF) (talk) 22:28, 6 June 2012 (UTC)[reply]

Could we return to exactly why Erik Möller thought it was his call to quash clear en.WP community consensus on the need for autoconfirmation before one can create articles? It required a minor technical change. So why does that morph into that kind of policy power in defiance of the community? By contrast, when I gained overwhelming en.WP community consensus to boost the default thumbnail size by almost 50%, the technicians properly advised and held off for logistical and technical reasons (for about six months). And if there had been technical reasons not to do it at all, one would have respected their opinion. Tony (talk) 03:00, 7 June 2012 (UTC)[reply]
You'd have to ask Erik :). Okeyes (WMF) (talk) 10:09, 7 June 2012 (UTC)[reply]
Note that I'm going to sort of disengage from this thread of the conversation, simply because I'm not sure if it's going anywhere that hasn't already been discussed ad infinitum. Okeyes (WMF) (talk) 12:35, 7 June 2012 (UTC)[reply]