Wikipedia:Wikipedia Signpost/2012-04-09/Technology report
Next Wikimedia deployment already in the pipeline and details of recent performance improvements
1.20wmf1 in the pipeline
| “ | To reiterate, I think most of the problem [volunteer resentment at the apparent preferential treatment of staff-written code] will disappear when we have regular code deployment again. At this point, it's best to focus solely on that and forget about all other complaints. If problems linger for long after everyone's code is getting deployed on a regular basis, we can talk about that then, and I think everyone will be talking on much more amicable basis. | ” |
| — Aryeh Gregor, October 2010 | ||
A provisional timetable was released for the first mini-deployment on Wikimedia sites from new version control system Git. Beyond the technical challenge of releasing against a substantially different code management background, the release should also herald the start of a new era of far quicker deployments to Wikimedia wikis—itself a development that has been discussed off-and-on for at least eighteen months (Signpost coverage).
With only a couple of months' worth of changes included (most of them simple bug fixes), it would be easy to overlook the release. Such a mistake is unlikely to be made by any developer aware of its historical significance, however, it being the quickest deployment of a block of changes (as opposed to individual "emergency" merges) in nearly two and a half years. The release thus marks the retirement of the previous paradigm – merging a small subset of revisions but retaining the majority for irregular watershed deployments – in favour of a new model focused on regular "mini"-deployments. Only time will tell whether or not Aryeh Gregor (quoted above in October 2010) will be proven right, and whether the problem of the volunteer-staff average review time divide (a problem which flared up once again on the wikitech-l mailing list only this week) will be settled once and for all.
According to the plan published on wikitech-l this week, non-Wikipedia sites (that is to say, Wikimedia Commons, Wiktionary, Wikisource, Wikinews, Wikibooks, Wikiquote and Wikiversity) should receive the update on April 16, with the English Wikipedia following on April 23. Should the deployment go well, the remaining wikis will be updated on April 25, just two months after they enjoyed the benefits of 1.19.0, which will only complete its own release cycle when it is made available to external sites later in the month.
March Engineering Report published
| “ | In March 2012:
|
” |
—Engineering metrics, Wikimedia blog | ||
The Wikimedia Foundation's engineering report for March 2012 was published this week on the Wikimedia Techblog and on the MediaWiki wiki, giving an overview of all Foundation-sponsored technical operations in that month. March was dominated at first by the deployment of MediaWiki 1.19 to all Wikipedias and more latterly by the move to Git and its associated code review system Gerrit. Other headlines from the month will also be familiar to regular Signpost readers, including the completion of the move of all Wikimedia domain names away from registrar GoDaddy in protest at their political stance on SOPA; the publication of a report on the first phase of Article Feedback version 5; and design improvements to the mobile front-end.
As is often the case, many of the changes that came in under the radar relate to incremental performance improvements aimed at allowing Wikimedia to support a rapidly increasing audience. For example, significant work was done with regard to preparing the newer Ashburn data centre to share responsibility with its Tampa counterpart for internal search functionality. Attempts to improve image caching were stymied by lingering concerns about "overloading the NIC cards and the risk of concentrating too much cache on each server", yielding only a trial improvement thus far. Network peering was also added to the Ashburn site, allowing it to pool resources with a dozen or so websites and ISPs—a move expected to reduce latency for users in Europe, Japan and Hong Kong. Similar motivations also led the Foundation to begin investigating the possibility of establishing a caching centre on the West Coast of the United States, the report said. Meanwhile, the switch in default thumbnail handling system to Swift finally settled down during the month after numerous problematic attempts at deployment during February; the same system is now expected to start handling non-thumbnailed images sometime in late May.
Elsewhere, it was announced that Wikimedia Labs' main per-project storage space (71,000 GB, currently distributed in 300 GB chunks) came online during March, though there were also two labs outages during the month. In addition, the Visual Editor team have now finalised a decision to base the new WYSIWYG editor around the contentEditable HTML5 property, having previously worked on a separate "editsurface" system in parallel, paving the way ahead towards a summer release. Finally, the first release of a complete copy of the English Wikipedia in the specialised ZIM file format (containing about 4 million articles, 11 million redirects, and 300,000 mathematical images) was also completed during March; the hope is to use regularly generated ZIM files – viewable with the WMF-supported Kiwix reader – to provide a complete offline browsing experience in the so-called "global south".
In brief
.png)
Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for many weeks.
- Updated apps for iOS and Android: This week saw new releases of the official Wikipedia app on both target platforms – version 1.1 for Android (a significant incremental release) and version 3.1 for iOS (which jumps forward to parity with the Android version, which before this week was significantly more advanced). It was ultimately a stressful week for the Wikipedia Mobile team, who were forced to work around the clock to fix a number of bugs with the initial releases of both versions despite significant pre-release testing. Thus the Android line of releases saw not just 1.1.0 this week but also 1.1.1, a minor bugfix release, while the iOS line received 3.1.0, 3.1.1 (a major bugfix release) and will shortly receive 3.1.2 to fix another high priority bug. Both version releases add improved tablet support as well as full text search and "Did you Mean?"’ correction support (Wikimedia blog). It was the replacement of Google Maps with OpenStreetMap as the base map provider for both versions that really caught the attention of the media, however, being highlighted on technology news websites such as CNET and PCMag, as well as the website of Austrian daily Der Standard.
- Developer starts work on Gerrit replacement: After discussions last week about the apparent shortcomings of new code review system Gerrit (Signpost coverage), one developer announced this week that he would be starting work on a new, custom-built system (wikitech-l mailing list). In announcing the "long term" project, experienced coder Daniel Friesen stressed that it was a response to "issues stemming fundamentally from how Gerrit is designed" rather than the minor UI issues that have also been discussed at length. The initial reception was largely negative, with Wikimedia regulars stressing that Friesen was taking a risk putting time into something that would only be considered for Foundation support and eventually implementation when it became a "real alternative" to Gerrit.
- Google Summer of Code applications closed: At midnight on April 6, applications closed for summer placements at the Wikimedia Foundation (provided via the Google Summer of Code programme). Volunteer Development Coordinator Sumana Harihareswara reported that the Foundation had received 63 such applications, of which 41 were of sufficient quality to go forward to the assessment phase. There they will compete for a number of places decided by Google; last year that number was eight, but it could well inch up slightly this week. Foundation staff and other top developers will now begin discussing and ranking proposals; successful applicants with be notified on April 23. In unrelated news, applications are now open to present at the Strange Loop conference, which will be held in Missouri in September this year.
- One-ish bots approved: 4 BRfAs were recently approved:
- Thehelpfulbot's 13th BRfA, general tagging (and untagging) of categories as requested, typically after CfD closures
- Hazard-Bot's 7th BRfA, a re-implementation of its 3rd BRfA
- Two extensions to H3llBot's 2nd BRfA (2b and 2c), switching to existing archived copies once a link dies; and tagging identified, dead domains/links with {{dead link}}
- Additionally, Rcsprinter123 has nominated himself for BAG membership; the community is encouraged to join that discussion, as well as those relating to the 16 currently active bot approvals.
- WMF organisation chart published: As reported by Erik Möller on the wikitech-l mailing list, Mark Holmquist (a contractor working for the Foundation) has generated a series of organisation charts for the organisation. The charts, which show staff names, locations and job titles hierarchically, currently cover all engineering departments; the hope is to expand them to also cover the remaining Wikimedia Foundation departments.
Discuss this story