Wikipedia:Village pump (technical)/Archive 218
This page contains discussions that have been archived from Village pump (technical). Please do not edit the contents of this page. If you wish to revive any of these discussions, either start a new thread or use the talk page associated with that topic.
< Older discussions · Archives: A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, AA, AB, AC, AD, AE, AF, AG, AH, AI, AJ, AK, AL, AM, AN, AO, AP, AQ, AR, AS, AT, AU, AV, AW, AX · 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218
Invisibly populated category redirects
Can anyone work out why Category:1951 events in Europe by month, Category:2007 events in Asia by month and Category:2008 events in Asia by month are appearing in Category:Wikipedia non-empty soft redirected categories? No contents are displayed, not even delayed caches, and yet they declare themselves non-empty. Timrollpickering (talk) 12:01, 27 January 2025 (UTC)
- Probably the job queue being slow to update the categorylinks, or (less likely) it having dropped some jobs. When null-edited one of the cats, it disappeared from Category:Wikipedia non-empty soft redirected categories. Anomie⚔ 12:10, 27 January 2025 (UTC)
- Is there supposed to be a job for this? Category:1951 events in Europe by month has {{Category redirect}} which tests whether the category is non-empty and should be added to Category:Wikipedia non-empty soft redirected categories. If the category is emptied without editing the category page or any template it transcludes then I wouldn't expect the wikitext of the category page to be reparsed automatically but I don't know whether it happens. PrimeHunter (talk) 13:25, 27 January 2025 (UTC)
- Yes, the MediaWiki servers should be re-parsing every page periodically, but they do not do so. See T132467, a long-standing feature request from 2016. (And the related T157670.) As far as I know, a cron job needs to be set up, but it has never been followed through on. I think Wbm1058 is still running a bot on the English Wikipedia to refresh stale pages, and that this query shows the current staleness of pages by date (the maximum appears to be 88 days right now). It is not great to be dependent on a bot for this critical maintenance, and 88 days of staleness is too much. It would be great to know that pages would never be more than X hours or days stale, with X being a small number. – Jonesey95 (talk) 15:07, 27 January 2025 (UTC)
- I briefly discussed this matter with a Foundation employee at Wikiconference North America in Indianapolis last October. As the English wiki continues to grow, closing in on 7 million articles, it becomes technically more and more difficult to frequently work though the entire database and refresh each and every page, whether they need refreshed or not (the vast majority don't). At my bot's peak performance, I had the refresh lag down to about 30 days for mainspace and 80 days for all other namespaces. After the database was restructured last year, my bots struggled to keep up and the lag times increased substantially. Only recently, they've come back down to 41 and 87 days, and the "new normal" may be 40 and 90 days, rather than 30 and 80. My bots should be considered as equivalent to that "cron job" – basically, I think, if such an internal job were set up, I doubt it would be much more efficient or timely at refreshing links than my bots are. My bots should be viewed as a stopgap; the last line of defense insuring that a link possibly still needing to be refreshed is refreshed after 90 days, and not nine years. The path forward is to identify the links refreshed by my bot that actually needed to be refreshed, determine why they failed to get refreshed before my stopgap bot refreshed them, and then fix that issue in order to refresh them a lot more quickly than my bot refreshes them. To that end, Phabs like T132467 are helpful, and I suggest that a higher priority be placed on T132467 than T157670. I'll look closer at what needs to happen with T132467 – maybe I can develop yet another bot to address that specific issue. – wbm1058 (talk) 16:57, 27 January 2025 (UTC)
- Probably worth mentioning this issue to the WMF annual plan and the community wishlist since both are open. Snævar (talk) 19:09, 29 January 2025 (UTC)
- I briefly discussed this matter with a Foundation employee at Wikiconference North America in Indianapolis last October. As the English wiki continues to grow, closing in on 7 million articles, it becomes technically more and more difficult to frequently work though the entire database and refresh each and every page, whether they need refreshed or not (the vast majority don't). At my bot's peak performance, I had the refresh lag down to about 30 days for mainspace and 80 days for all other namespaces. After the database was restructured last year, my bots struggled to keep up and the lag times increased substantially. Only recently, they've come back down to 41 and 87 days, and the "new normal" may be 40 and 90 days, rather than 30 and 80. My bots should be considered as equivalent to that "cron job" – basically, I think, if such an internal job were set up, I doubt it would be much more efficient or timely at refreshing links than my bots are. My bots should be viewed as a stopgap; the last line of defense insuring that a link possibly still needing to be refreshed is refreshed after 90 days, and not nine years. The path forward is to identify the links refreshed by my bot that actually needed to be refreshed, determine why they failed to get refreshed before my stopgap bot refreshed them, and then fix that issue in order to refresh them a lot more quickly than my bot refreshes them. To that end, Phabs like T132467 are helpful, and I suggest that a higher priority be placed on T132467 than T157670. I'll look closer at what needs to happen with T132467 – maybe I can develop yet another bot to address that specific issue. – wbm1058 (talk) 16:57, 27 January 2025 (UTC)
- Yes, the MediaWiki servers should be re-parsing every page periodically, but they do not do so. See T132467, a long-standing feature request from 2016. (And the related T157670.) As far as I know, a cron job needs to be set up, but it has never been followed through on. I think Wbm1058 is still running a bot on the English Wikipedia to refresh stale pages, and that this query shows the current staleness of pages by date (the maximum appears to be 88 days right now). It is not great to be dependent on a bot for this critical maintenance, and 88 days of staleness is too much. It would be great to know that pages would never be more than X hours or days stale, with X being a small number. – Jonesey95 (talk) 15:07, 27 January 2025 (UTC)
- Is there supposed to be a job for this? Category:1951 events in Europe by month has {{Category redirect}} which tests whether the category is non-empty and should be added to Category:Wikipedia non-empty soft redirected categories. If the category is emptied without editing the category page or any template it transcludes then I wouldn't expect the wikitext of the category page to be reparsed automatically but I don't know whether it happens. PrimeHunter (talk) 13:25, 27 January 2025 (UTC)
- This particular category is an easy case to manage. I just ran a script to purge the cache of each member of the category, which quickly reduced the category membership from 90 to 30. Then I noticed that there were still newly-empty categories in this category, so I ran the script again, which reduced membership to 25. There were still newly-empty members, so I ran the script a third time and that kept the membership at 25 as just as many new members arrived as my script had just purged out. Is this category always so active, or is something special happening now to make it more active than usual? I can add this operation to my bot that runs twice hourly, or maybe run it even more frequently than twice an hour; that would keep the membership better, with a minimum number of short-term empty members. – wbm1058 (talk) 01:10, 28 January 2025 (UTC)
- Looks like User:JJMC89 bot III is moving a bunch of categories for Wikipedia:Categories for discussion/Speedy#Current requests, which are apparently showing up in Category:Wikipedia non-empty soft redirected categories momentarily. Anomie⚔ 01:19, 28 January 2025 (UTC)
- Yes. Basically, there's an ongoing WP:CFD/S process to rename categories of the form "Date events in Foo" to "Date in Foo", that is, to remove the word "events" and one adjacent space. So for example Category:March 1979 events in North America has been moved to Category:March 1979 in North America. I think that it should have been a full CFD and not a speedy, but there you go. --Redrose64 🌹 (talk) 10:54, 28 January 2025 (UTC)
- Addendum: as I typed the above, Category:March 1979 events in North America was in Category:Wikipedia non-empty soft redirected categories, and its cat page was listing March 1979 in Canada as a subcat, whereas a visit to Category:March 1979 in Canada showed the cat box containing March 1979 in North America. Visiting Category:March 1979 in North America did not list March 1979 in Canada as a subcat. I tried a WP:PURGE of all three categories, which had no effect (as I suspected it wouldn't), and then performed a WP:NULLEDIT of Category:March 1979 in Canada, which did not itself change, but it did cause both Category:March 1979 events in North America and Category:March 1979 in North America to be corrected, and the former to drop out of Category:Wikipedia non-empty soft redirected categories. --Redrose64 🌹 (talk) 11:04, 28 January 2025 (UTC)
- Right, looking at Special:Log/move/JJMC89 bot III, that's the culprit. My understanding is that my "null edit" cache-purging bot enters tasks into the "job queue", or, rather usually executes its tasks nearly instantaneously, and its tasks only spend time waiting in the job queue at times when the system is particularly busy and overwhelmed by too many task requests being pushed at it simultaneously. The fact that my bot's purges are happening right away indicates to me that the page-moving software, which should be purging categories right after it moves them, isn't doing that. Search Phabricator for something like "Special:MovePage needs to purge the cache of Category: namespace pages immediately after moving them". I'm adding this to-do item to my MediaWiki core developers thread. Foundation management hasn't assigned the page-moving code to any employee's responsibilities as I guess they're waiting for volunteer me to push myself into the role. – wbm1058 (talk) 11:23, 28 January 2025 (UTC)
- In the meantime, while waiting for Special:MovePage code fixes, maybe User:JJMC89 could enhance his bot to make it purge each category page right after it moves the category. Updating bot code is magnitudes easier than updating MediaWiki code. – wbm1058 (talk) 11:43, 28 January 2025 (UTC)
- Looking at the timestamps of Redrose64's example, the category really was non-empty for a few seconds.
- Category:March 1979 events in North America was moved at 2025-01-27 23:41:02. At this point it becomes a non-empty soft redirected category, and
recentchangesindicates it was added to Category:Wikipedia non-empty soft redirected categories at this time. - Category:March 1979 events in North America had its template updated at 2025-01-27 23:41:04.
- Havana Jam was removed from Category:March 1979 events in North America at 2025-01-27 23:41:08. At this point Category:March 1979 events in North America presumably becomes empty, but the category page isn't reparsed.
- Category:March 1979 events in North America was moved at 2025-01-27 23:41:02. At this point it becomes a non-empty soft redirected category, and
- So for about 6 seconds from 23:41:02 to 23:41:08, Category:March 1979 events in North America really was a non-empty soft redirected category. Based on the
mw.categorizeentries inrecentchanges, it looks like all three of the above edits did immediately update the category links. What didn't happen immediately is the re-parsing of Category:March 1979 events in North America to determine that it was now empty. If User:JJMC89 bot III was going to purge to have an effect here, it would have to have been after the Havana Jam edit emptied the category, not after the category was moved. Anomie⚔ 13:02, 28 January 2025 (UTC)- Oh, I see. This bot is editing at an incredibly high speed. 42 edits at 23:59, 27 January 2025, that's like an edit every 1.4 seconds, a majority of them being page moves. – wbm1058 (talk) 14:14, 28 January 2025 (UTC)
- Here is the bot's edit log for the relevant time span. March 1979 events in North America-related activity seems to be co-mingled with Novels with lesbian themes-related activity. What's the algorithm here? Are two separate instances of the bot running in parallel? wbm1058 (talk) 14:14, 28 January 2025 (UTC)
- There's some misunderstanding here. A purge doesn't work, it must be a WP:NULLEDIT; and doing that on the moved category isn't any good either, it needs to be performed on the category's member pages. --Redrose64 🌹 (talk) 22:12, 28 January 2025 (UTC)
- @Redrose64: Indeed. I use User:RMCD bot/botclasses.php
function purgeCache($page), which in turn uses mw:API:Purge with|forcerecursivelinkupdate=1, which is more or less functionally equivalent to what you call a null edit. The category's member pages are indeed categories themselves. – wbm1058 (talk) 23:06, 28 January 2025 (UTC)
- @Redrose64: Indeed. I use User:RMCD bot/botclasses.php
- There can be up to two instances running at the same time, one for WP:CFD/W and one WP:CFD/W/L. This is so the large batches on CFD/W/L do not delay processing of the ones on CFD/W. Usually there is only one running since CFD/W/L is not used most of the time. — JJMC89 (T·C) 08:05, 29 January 2025 (UTC)
- There's some misunderstanding here. A purge doesn't work, it must be a WP:NULLEDIT; and doing that on the moved category isn't any good either, it needs to be performed on the category's member pages. --Redrose64 🌹 (talk) 22:12, 28 January 2025 (UTC)
- The bot makes a follow-up edit to the category after the move. I've reordered that step to after it recategorizes the contents instead of immediately after the move. That should remove the need to purge. — JJMC89 (T·C) 07:58, 29 January 2025 (UTC)
- Thanks. An editor User:Gray eyes is creating category soft redirects (e.g., Category:Sports in Gdańsk, Category:Organizations based in Łódź, Category:Sports in Lublin) which are populating Category:Wikipedia non-empty soft redirected categories. I don't know why these empty soft redirects are populating the non-empty category, nor why they are being created in the first place, given that the template produces a message "Administrators: If this category name is unlikely to be entered on new pages, and all incoming links have been cleaned up, click here to delete." implying that these newly-created categories should be deleted. – wbm1058 (talk) 17:02, 29 January 2025 (UTC)
- I had to use this template (Template:Sports clubs and teams in Fooland category header) to create a Category:Sports clubs and teams in Gdańsk. These categories will be automatically emptied. Gray eyes (talk) 06:22, 30 January 2025 (UTC)
- Thanks. An editor User:Gray eyes is creating category soft redirects (e.g., Category:Sports in Gdańsk, Category:Organizations based in Łódź, Category:Sports in Lublin) which are populating Category:Wikipedia non-empty soft redirected categories. I don't know why these empty soft redirects are populating the non-empty category, nor why they are being created in the first place, given that the template produces a message "Administrators: If this category name is unlikely to be entered on new pages, and all incoming links have been cleaned up, click here to delete." implying that these newly-created categories should be deleted. – wbm1058 (talk) 17:02, 29 January 2025 (UTC)
- Looking at the timestamps of Redrose64's example, the category really was non-empty for a few seconds.
- Looks like User:JJMC89 bot III is moving a bunch of categories for Wikipedia:Categories for discussion/Speedy#Current requests, which are apparently showing up in Category:Wikipedia non-empty soft redirected categories momentarily. Anomie⚔ 01:19, 28 January 2025 (UTC)
- OK, now there are hundreds of empty categories in Category:Wikipedia non-empty soft redirected categories. I'll add a twice-hourly purge/null-edit to my bot, to manage this issue as a stopgap, until the issue with the MediaWiki software is identified and fixed. Any time a category is removed from a page, I think a forcerecursivelinkupdate purge should be done. – wbm1058 (talk) 12:55, 30 January 2025 (UTC)
No deletion log for long-ago-deleted article
When I went to http://en.wiki.x.io/wiki/Resource_discovery, I was surprised to see a MediaWiki:thisisdeleted notice (View or undelete 2 deleted edits? (view logs for this page | view filter log)) but no deletion log entry, nothing like what you'll see if you visit the recently deleted http://en.wiki.x.io/wiki/Snape_kills_Dumbledore. (Sorry for external-style links, but the message there is different from the message you see on the edit screen.) Turns out that the article was deleted in 2004, when its entire content was:
{{delete}} I LOVE ALEXANDER DESPATIE
Is this normal behaviour for a page that was deleted so, so long ago and never recreated? Nyttend (talk) 20:28, 29 January 2025 (UTC)
- Yes. The current log system dates back to December 2004. Older logs can be found at WP:Historical archive/Logs/Deletion log. * Pppery * it has begun... 20:34, 29 January 2025 (UTC)
- Yes, indeed. The days when the deletion log was a list that people with deletion privileges maintained by hand. And there was no undeletion mechanism. And the deleted revisions table got cleared by the people who ran the servers every so often. Uncle G (talk) 14:48, 31 January 2025 (UTC)
- @Nyttend: It's easy to check; go to Special:Log/delete, and on the line "(newest | oldest) View (newer 50 | older 50) (20 | 50 | 100 | 250 | 500)", click the "oldest" link. --Redrose64 🌹 (talk) 21:42, 29 January 2025 (UTC)
Help with selective transclusion
ResolvedAs pointed out here, this template needs to be edited so that the floating link appears on Wikipedia:Requests for permissions, but not Template:Admin dashboard. Any takers? ⇌ Jake Wartenberg 16:00, 31 January 2025 (UTC)
Next steps towards OWID visualization within MediaWiki
We at Wiki Project Med have built a method to visualize Our World in Data with all material coming from Commons. You can see it functional at mdwiki:WikiProjectMed:OWID#Way_3_(current_effort).
Wondering if we can get this and this copied to EN WP so we can begin testing here.
On MDWiki you should be able to:
- scroll through the years of data,
- if you put your cursor over a country it should highlight and give you the name,
- if you put your cursor over the ranges bar, it should highlight all the countries in that range,
- if you click on a country it should pull up a graph of how data has changed in that country over time
- if you select a region of the world it will zoom into that region
It is built from about 500 seperate images. Doc James (talk · contribs · email) 05:06, 1 February 2025 (UTC)
- We are working on improving functionality on mobile as currently this is poor. Just wanting to begin testing here, it is not ready for us in mainspace. Doc James (talk · contribs · email) 05:15, 1 February 2025 (UTC)
Global watchlist (for wikis in different languages)
Hi everyone,
I was wondering what the status of using the GlobalWatchlist extension on Wikipedia to have a unified watchlist across different wikis (all Wikipedia, but different languages)?
It seems like there is on-going development work on the extension itself, but I am not finding anything recent on its use for Wikipedia. Is there a trail of this somewhere?
Thanks a lot in advance!
Best, Julius Schwarz (talk) 08:39, 1 February 2025 (UTC)
- It's on Meta-Wiki: m:Special:GlobalWatchlist. Nardog (talk) 09:29, 1 February 2025 (UTC)
- Oh that's neat, thanks a lot! Julius Schwarz (talk) 15:21, 1 February 2025 (UTC)