Page MenuHomePhabricator

Magnus (Magnus Manske)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 23 2014, 3:02 PM (332 w, 3 d)
Availability
Available
LDAP User
Magnus Manske
MediaWiki User
Unknown

Recent Activity

Fri, Mar 5

Magnus closed T276253: flickr2commons.toolforge.org stuck at status "Running" due to "Content Security Policy error" as Resolved.

Turns out flickr retired the secure.flickr.com domain without forwarding. Removed the "secure" but still using https, should work again.

Fri, Mar 5, 11:33 AM · Tools

Thu, Feb 25

Magnus added a comment to T179879: Provide a 5-minute timeout in WDQS for trusted users using OAuth.

Also, doesn't seem to work for "has a birth date"?
https://www.wikidata.org/w/index.php?search=haswbstatement%3AP21%3DQ6581072++haswbstatement%3AP569&title=Special:Search&profile=advanced&fulltext=1&advancedSearch-current=%7B%7D&ns0=1&ns120=1

Thu, Feb 25, 9:17 AM · Discovery, Wikidata, Wikidata-Query-Service
Magnus added a comment to T179879: Provide a 5-minute timeout in WDQS for trusted users using OAuth.

Both of these can be done using CirrusSearch API and can't they?

Thu, Feb 25, 9:13 AM · Discovery, Wikidata, Wikidata-Query-Service

Fri, Feb 19

Magnus added a comment to T274811: "duplicity" on wikidata-todo.toolforge.org returns "503 Service Not Available" error message.

Restarted wikidata-todo.

Fri, Feb 19, 1:14 PM · Tools

Jan 28 2021

Magnus added a comment to T197871: Add articlequality model in PetScan.

It does work, however, since there is currently not a single page on euwiki where the latest revision has a wp10 score, the results will be empty.

Jan 28 2021, 10:36 AM · artificial-intelligence, Machine-Learning-Team, articlequality-modeling

Jan 21 2021

Magnus updated the task description for T269609: Ensure all ToolsDB databases comply with current naming conventions.
Jan 21 2021, 4:52 PM · Developer-Advocacy (Jan-Mar 2021), cloud-services-team (Kanban), Data-Services

Jan 20 2021

Bstorm awarded T272430: mixnmatch microsync process has a large error file a Like token.
Jan 20 2021, 2:59 PM · Tools
Magnus closed T272430: mixnmatch microsync process has a large error file as Resolved.

Thanks, I think I fixed it now.

Jan 20 2021, 9:07 AM · Tools
Magnus closed T272430: mixnmatch microsync process has a large error file, a subtask of T272247: 2021-01-17: tools NFS share cleanup, as Resolved.
Jan 20 2021, 9:07 AM · Toolforge, cloud-services-team (Kanban)

Dec 18 2020

Magnus closed T270467: Quickstatements background run is stuck as Resolved.
Dec 18 2020, 9:53 AM · Wikidata, Tools
Magnus added a comment to T270467: Quickstatements background run is stuck.

Should restart now

Dec 18 2020, 9:12 AM · Wikidata, Tools
Magnus added a comment to T270467: Quickstatements background run is stuck.

On it

Dec 18 2020, 9:06 AM · Wikidata, Tools

Nov 25 2020

Magnus added a comment to T205014: Data input: Forms for Item Entry.

I would also like to point out https://tabernacle.toolforge.org/ which ATM is for editing existing items, but I could easily create a new blank item to edit.

Nov 25 2020, 4:31 PM · Wikibase-Containers, Wikidata, Federated-Wikibase-Workshops@NewYork-2018

Nov 23 2020

Magnus closed T205549: book2scroll is not working (due to jquery 404?) as Resolved.
Nov 23 2020, 2:09 PM · Tools, Wikisource

Oct 14 2020

Magnus added a comment to T250621: Items created through QuickStatements don't get added to Watchlist on wikidata.org.

Just saw this one. I think fixing wikibase to add to watchlist is the way to go. I could update the OAuth permissions but that would break everyone's server-based edits as they would use an old OAuth, correct?

Oct 14 2020, 9:29 AM · Wikidata-Campsite, MediaWiki-extensions-WikibaseRepository, Wikidata

Oct 9 2020

Magnus closed T264758: PetScan returns "502 Bad Gateway" for some days as Resolved.

Should be fixed now

Oct 9 2020, 9:02 AM · Tools

Oct 7 2020

Magnus added a comment to T264758: PetScan returns "502 Bad Gateway" for some days.

I'm on it.

Oct 7 2020, 10:04 AM · Tools

Jul 13 2020

Magnus closed T257555: Lost connection to MySQL server while trying to import 39M rows of csv data (toolsdb) as Resolved.

Worked with 10K batches. I still think this rapid disconnect in the middle of a query is a bug.

Jul 13 2020, 7:23 PM · Data-Services, Toolforge
Magnus added a comment to T257555: Lost connection to MySQL server while trying to import 39M rows of csv data (toolsdb).

Trying 1M chunks, first two worked, but then:

mysqlimport: Error: 2013, Lost connection to MySQL server during query, when using table: osopenuprn_202006

I'll try even smaller ones, but it's getting a bit ridiculous. Why disconnect the server in the middle of a query?

Jul 13 2020, 8:51 AM · Data-Services, Toolforge

Jul 9 2020

Magnus created T257555: Lost connection to MySQL server while trying to import 39M rows of csv data (toolsdb).
Jul 9 2020, 10:13 AM · Data-Services, Toolforge

Jun 30 2020

Magnus added a comment to T255730: Request increased quota for petscan Toolforge tool for database access.

I have switched the configuration to use the petscan db connections only.

Jun 30 2020, 8:36 AM · Data-Services (Quota-requests)

Jun 26 2020

Magnus added a comment to T255730: Request increased quota for petscan Toolforge tool for database access.

Just to add, if some queries take too long I can likely just change a parameter to fix that. Someone with access to the stats, let me know.

Jun 26 2020, 7:46 AM · Data-Services (Quota-requests)
Magnus added a comment to T255730: Request increased quota for petscan Toolforge tool for database access.

The reason I use that many connections is precisely to avoid long-running ones, as I had in a previous version; they would time out or lose database connection, so I re-wrote the code to use more but shorter queries.

Jun 26 2020, 7:45 AM · Data-Services (Quota-requests)

Jun 24 2020

Magnus added a comment to T243848: Can't push to diffusion.

@Magnus: Does that mean that https://phabricator.wikimedia.org/source/tool-quickstatements/browse/master/ will get out of sync and should be archived?

Yes.

Jun 24 2020, 8:45 AM · Diffusion
Magnus changed the status of T243848: Can't push to diffusion from Declined to Resolved.

I have "forked" it into github. Using diffusion was a mistake in the first place. Now at https://github.com/magnusmanske/quickstatements

Jun 24 2020, 8:03 AM · Diffusion
Magnus added a comment to T243848: Can't push to diffusion.

Tried again with ssh://vcs@git-ssh.wikimedia.org/source/tool-quickstatements.git, no joy:

Permission denied (publickey,keyboard-interactive).
fatal: Could not read from remote repository.
Jun 24 2020, 7:34 AM · Diffusion
Magnus added a comment to T243848: Can't push to diffusion.

tools.quickstatements@tools-sgebastion-07:~$ git remote remove origin
tools.quickstatements@tools-sgebastion-07:~$ git remote add origin ssh://phabricator.wikimedia.org/diffusion/2010/tool-quickstatements.git/
tools.quickstatements@tools-sgebastion-07:~$ git push fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use

Jun 24 2020, 7:32 AM · Diffusion

Jun 22 2020

Magnus added a comment to T234617: Toolforge. introduce new domain toolforge.org.

FWIW, I believe all my (active) tools now use the new canonical URL.

Jun 22 2020, 3:08 PM · Patch-For-Review, Goal, Toolforge, cloud-services-team (Kanban), Kubernetes

Jun 19 2020

Magnus closed T100973: Flickr2Commons tool: Throttle needed as Resolved.

As I wrote five years ago, I added a limit to upload only two files concurrently.
No further comment has been made since, so I consider this issue to be solved.

Jun 19 2020, 9:30 AM · Tools, Commons

Jun 18 2020

Magnus added a comment to T255730: Request increased quota for petscan Toolforge tool for database access.

Thanks, that seems perfect!

Jun 18 2020, 5:18 PM · Data-Services (Quota-requests)

Jun 8 2020

Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

This config is part of the default configuration generated by webservice. That means we need to fix it upstream of the usage by this particular tool.

OK thanks

As a horrible short term fix for reasonator, I did:

$ ln -s $HOME/public_html/reasonator_types.js $HOME/public_html/_types.js
Jun 8 2020, 6:57 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

I don’t understand why this is an UBN when as far as I understand we know a workaround (restart without --canonical)?

Jun 8 2020, 6:55 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T194953: Support hosting Rust tools on Toolforge.

Is there a movement towards Rust in any significant segment of the Wikimedia movement? Are there clear benefits for Toolforge tools to be written and maintained in Rust?

Jun 8 2020, 7:16 AM · cloud-services-team (Kanban), Toolforge

Jun 7 2020

Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

I think this is a blocker to moving all tools to the canonical URL. T234617 ?

Jun 7 2020, 4:04 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

I cleared the error log on reasonator and switched to the previous URL schema, then back to the canonical one. No server errors in the log (some unrelated PhP ones, now cleared again)

Jun 7 2020, 3:34 PM · cloud-services-team (Kanban), Tools, Toolforge

Jun 6 2020

Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

@LucasWerkmeister thanks that worked, for now

Jun 6 2020, 3:10 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus raised the priority of T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical` from High to Unbreak Now!.

Same with https://geohack.toolforge.org/geohack.php . Geohack is kinda central for Wikipedia so please help!

Jun 6 2020, 12:43 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T254361: Tool `most-wanted` is 502.

Thanks @bd808 !

Jun 6 2020, 9:23 AM · cloud-services-team (Kanban), Toolforge
Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

Other files load fine. Haven't checked all though

Jun 6 2020, 8:57 AM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.

URL is https://reasonator.toolforge.org/reasonator_types.js

Jun 6 2020, 8:56 AM · cloud-services-team (Kanban), Tools, Toolforge
Magnus triaged T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical` as High priority.
Jun 6 2020, 8:54 AM · cloud-services-team (Kanban), Tools, Toolforge
Magnus created T254640: Default lighttpd config created by `webservice` breaks serving files starting with the same string as the tool's name under `--canonical`.
Jun 6 2020, 8:54 AM · cloud-services-team (Kanban), Tools, Toolforge

Jun 3 2020

Magnus created T254361: Tool `most-wanted` is 502.
Jun 3 2020, 3:07 PM · cloud-services-team (Kanban), Toolforge

May 28 2020

Magnus added a comment to T251567: PetScan problem: 504 Gateway Time-out.

Restarted it now.
I am also working on a watcher script to restart it automatically when it goes down.

May 28 2020, 8:27 AM · Cloud-VPS
Magnus added a comment to T215035: 502 /504 Bad Gateway issue on Petscan.

It's my dev server, should work for any query size. Feel free to use when the main site is down.

May 28 2020, 8:26 AM · VPS-Projects

May 22 2020

Magnus created T253373: Commons API search/imageinfo generator not working.
May 22 2020, 8:30 AM · MediaWiki-API, Commons

May 21 2020

Magnus placed T181062: Adapt QuickStatements2 to be able to work with structured data on Commons as well up for grabs.
May 21 2020, 7:24 AM · WMSE-Tools-for-Partnerships-2020, GLAM-Tech, SDC General, Wikidata

Apr 29 2020

Magnus added a comment to T215035: 502 /504 Bad Gateway issue on Petscan.

@Aklapper actually the repo is https://github.com/magnusmanske/petscan_rs (the bitbucket one is the old C++ version).

Apr 29 2020, 1:37 PM · VPS-Projects

Apr 22 2020

Addshore awarded T249587: Wikidata API wbgetentities: Allowed memory size exceeded a Hungry Hippo token.
Apr 22 2020, 11:08 PM · Wikidata-Campsite, Wikidata

Apr 20 2020

Magnus added a comment to T215035: 502 /504 Bad Gateway issue on Petscan.

There are some PetScan outages, reason unknown so far, but outside those, the above queries all work fine.

Apr 20 2020, 7:43 AM · VPS-Projects

Apr 14 2020

Magnus added a comment to T249587: Wikidata API wbgetentities: Allowed memory size exceeded.

Still happening.

Apr 14 2020, 9:43 AM · Wikidata-Campsite, Wikidata

Apr 7 2020

Magnus added a comment to T249587: Wikidata API wbgetentities: Allowed memory size exceeded.

Maybe this helps:

Apr 7 2020, 2:26 PM · Wikidata-Campsite, Wikidata
Magnus added a comment to T249587: Wikidata API wbgetentities: Allowed memory size exceeded.

Yes, still seeing it. Runs for ~20sec so probably not a cache. Changed the JSONP callback(s) and still same error, so definitely no content-level cache.

Apr 7 2020, 2:25 PM · Wikidata-Campsite, Wikidata
Magnus created T249587: Wikidata API wbgetentities: Allowed memory size exceeded.
Apr 7 2020, 7:20 AM · Wikidata-Campsite, Wikidata

Mar 18 2020

bd808 awarded T246096: Tools cdnjs returns blank page a Yellow Medal token.
Mar 18 2020, 7:57 PM · Tools, cloud-services-team (Kanban)

Mar 17 2020

Magnus updated subscribers of T246096: Tools cdnjs returns blank page.
Mar 17 2020, 10:56 AM · Tools, cloud-services-team (Kanban)

Mar 13 2020

Magnus closed T247596: Strange wikibase API error as Resolved.

It appears that the third-party bot framework quietly swallows this and just throws an exception with the "info" field :-(

Mar 13 2020, 2:18 PM · Wikidata
Magnus added a comment to T247596: Strange wikibase API error.

In the error I mean

Mar 13 2020, 1:24 PM · Wikidata
Magnus added a comment to T247596: Strange wikibase API error.

Hmm, I think what tripped my code is that there was no code:'maxlag' in the response?

Mar 13 2020, 1:23 PM · Wikidata
Magnus added a comment to T247596: Strange wikibase API error.

Seems to clear up now

Mar 13 2020, 1:22 PM · Wikidata
Magnus added a comment to T247596: Strange wikibase API error.

But it's all kinds of wikis that show up. Something with the dispatcher?

Mar 13 2020, 1:17 PM · Wikidata
Magnus created T247596: Strange wikibase API error.
Mar 13 2020, 11:46 AM · Wikidata

Mar 4 2020

Magnus added a comment to T246746: Bot edits on Wikidata should be hidden in other project's recent changes.

QuickStatements edits as the user who submitted the job, using OAuth, as is the WMF-preferred method for tools.

Mar 4 2020, 1:27 PM · Upstream, MediaWiki-extensions-WikibaseClient, Wikidata, MediaWiki-Recent-changes, Growth-Team, Edit-Review-Improvements-RC-Page

Feb 25 2020

Magnus created T246096: Tools cdnjs returns blank page.
Feb 25 2020, 12:32 PM · Tools, cloud-services-team (Kanban)

Feb 19 2020

Magnus closed T232917: XSS in listeria as Resolved.

Should now be fixed with https://bitbucket.org/magnusmanske/listeria/commits/5b53c94ffb10

Feb 19 2020, 9:35 AM · Security, Vuln-XSS, Tools

Feb 4 2020

Magnus added a comment to T156626: k8s webservice restart failure with `ValueError: get() more than one object; use filter`.

Getting this error for mix-n-match tool.

Feb 4 2020, 3:52 PM · Patch-For-Review, Kubernetes, Tools-Kubernetes, Toolforge

Jan 28 2020

Magnus added a project to T243848: Can't push to diffusion: Diffusion.
Jan 28 2020, 3:55 PM · Diffusion
Magnus created T243848: Can't push to diffusion.
Jan 28 2020, 3:55 PM · Diffusion

Jan 26 2020

Magnus added a comment to T214278: Quickstatements, "backend is overloaded".

@bd808 Thanks, now running quickstatements as two repilcas

Jan 26 2020, 9:28 AM · TestMe, Tools, Toolforge

Jan 24 2020

Magnus added a comment to T243580: "413 Request Entity Too Large" / Kubernetes 2020?.

Update: Setting server.max-request-size = 1000000 in $HOME/.lighttpd.conf and restarting the webservice did not help.

Jan 24 2020, 9:29 AM · cloud-services-team (Kanban), Tools, Kubernetes, Toolforge
Magnus created T243580: "413 Request Entity Too Large" / Kubernetes 2020?.
Jan 24 2020, 9:22 AM · cloud-services-team (Kanban), Tools, Kubernetes, Toolforge

Jan 14 2020

Magnus created T242720: Regression in URL path handling on 2020 (?) kubernetes.
Jan 14 2020, 9:39 AM · Cloud-Services

Dec 10 2019

Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

@Bugreporter That seems unlikely. QuickStatementsBot is blocked since October 28th, and daily users of the tool only started reporting this issue yesterday.

Dec 10 2019, 3:06 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

For debugging:

Dec 10 2019, 12:32 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

It's not a stale cache on my side. If I get different properties from https://www.wikidata.org/w/api.php?action=help&modules=query%2Buserinfo I get the requested properties but still the bogus block.

Dec 10 2019, 11:30 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

If I check the API directly in the browser, it doesn't show. Maybe because OAuth login?

Dec 10 2019, 11:26 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

I seem to be getting the block for [[User:Doqume]] on Wikidata, but for every user

Dec 10 2019, 11:25 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

I think I found the reason. Using MW API to get user info, I get:

{
  "id": 4420,
  "name": "Magnus Manske",
  "blockid": 15320,
  "blockedby": "Mahir256",
  "blockedbyid": 203574,
  "blockreason": "আপনাকে স্বয়ংক্রিয়ভাবে বাধা দেওয়া হয়েছে, কারণ আপনার আইপি ঠিকানাটি সম্প্রতি \"[[User:Doqume|Doqume]]\" ব্যবহার করেছেন। Doqume-কে বাধাদানের কারণ \"Freebald-ish behavior\"",
  "blockedtimestamp": "2019-12-09T16:59:04Z",
  "blockexpiry": "2019-12-10T16:59:04Z",
  "groups": [
    "rollbacker",
    "*",
    "user",
    "autoconfirmed"
  ],
  "rights": [
    "autopatrol",
    "editsemiprotected",
    "move",
    "autoconfirmed",
    "skipcaptcha",
    "abusefilter-log-detail",
    "suppressredirect",
    "read",
    "edit",
    "createpage",
    "createtalk",
    "writeapi",
    "translate",
    "item-term",
    "property-term",
    "item-merge",
    "item-redirect",
    "abusefilter-view",
    "abusefilter-log",
    "flow-hide",
    "reupload-own",
    "move-rootuserpages",
    "move-categorypages",
    "minoredit",
    "purge",
    "applychangetags",
    "changetags",
    "reupload",
    "upload",
    "flow-edit-post"
  ]
}

But I am not blocked; editing works fine.

Dec 10 2019, 11:14 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

I can't create batches either, so good for testing!

Dec 10 2019, 11:02 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata
Magnus added a comment to T240316: Issue with QuickStatements "you are blocked on Wikidata".

I did not touch QS for weeks. No idea why this is happening.

Dec 10 2019, 11:01 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata

Dec 6 2019

Magnus closed T239817: Cross Site Scripting on tools.wmflabs.org/multidesc as Resolved.
Dec 6 2019, 12:37 PM · Security, Security-Team, Tools, Vuln-XSS
Magnus added a comment to T239817: Cross Site Scripting on tools.wmflabs.org/multidesc.

Fixed now.

Dec 6 2019, 8:39 AM · Security, Security-Team, Tools, Vuln-XSS

Dec 5 2019

Magnus added a comment to T239036: OBB-1022554 XSS in not-in-the-other-language.

Hello Team,

I am still able to have a XSS vulnerability in the same website. Should I paste the details over here or make a new report. Waiting for your response

@sbassett @Reedy

Dec 5 2019, 1:46 PM · Security, Tools, Vuln-XSS, Toolforge-standards-committee, cloud-services-team, Security-Team

Dec 4 2019

Magnus added a comment to T239036: OBB-1022554 XSS in not-in-the-other-language.

Done.

Dec 4 2019, 11:34 AM · Security, Tools, Vuln-XSS, Toolforge-standards-committee, cloud-services-team, Security-Team

Dec 3 2019

Magnus added a comment to T239036: OBB-1022554 XSS in not-in-the-other-language.

This should get them all:

Dec 3 2019, 2:42 PM · Security, Tools, Vuln-XSS, Toolforge-standards-committee, cloud-services-team, Security-Team
Magnus added a comment to T239036: OBB-1022554 XSS in not-in-the-other-language.

on it

Dec 3 2019, 2:33 PM · Security, Tools, Vuln-XSS, Toolforge-standards-committee, cloud-services-team, Security-Team

Nov 12 2019

Magnus added a comment to T236974: Establish a process for increasing a toolforge tool's connections to the wiki replicas.

To add another use case (and to ping the issue):

Nov 12 2019, 9:58 AM · cloud-services-team (Kanban), Data-Services

Sep 25 2019

Magnus closed T233730: Reducing geohack/access.log size (58GB), a subtask of T233120: 2019-09-17: tools share cleanup (high usage), as Resolved.
Sep 25 2019, 8:30 AM · Data-Services, cloud-services-team (Kanban)
Magnus closed T233730: Reducing geohack/access.log size (58GB) as Resolved.

deleted access and error log files

Sep 25 2019, 8:30 AM · Data-Services, cloud-services-team (Kanban)

Sep 24 2019

Magnus added a comment to T181062: Adapt QuickStatements2 to be able to work with structured data on Commons as well.

As of today, QuickStatements supports MediaInfo items (Mxxx).
For now, you'll have to supply the IDs manually, which is a pain.
I am working on a QS syntax parser in Rust, which will support

  • ranks
  • page/filename => ID conversion on-the-fly

This will require some more testing

Sep 24 2019, 10:04 AM · WMSE-Tools-for-Partnerships-2020, GLAM-Tech, SDC General, Wikidata

Sep 17 2019

Magnus closed T232960: QuickStatements batch running in background not stopping with the stop button as Resolved.
Sep 17 2019, 9:25 AM · Tools, Wikidata

Sep 16 2019

Magnus added a comment to T232960: QuickStatements batch running in background not stopping with the stop button.

Bot code patched, deployed, someone please test

Sep 16 2019, 1:11 PM · Tools, Wikidata
Magnus added a comment to T232986: Enable all possible language pairs in cxserver, apply wiki specific configuration in the wiki.

+1

Sep 16 2019, 10:46 AM · Language-Team (Language-2019-October-December), MW-1.34-notes (1.34.0-wmf.25; 2019-10-01), ContentTranslation
Magnus added a comment to T232960: QuickStatements batch running in background not stopping with the stop button.

That's true, but a reload of the batch page should return the STOP button, as its state is only read from the database. The bot, in turn, only checks the database (or should, I suspect it doesn't).

Sep 16 2019, 8:09 AM · Tools, Wikidata

Sep 8 2019

Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

Another idea came to me:
What is it's not just "page lists", but any (general, of one of pre-defined types) tables?
One table type would be "page title/page namespace", giving us the above lists.
Others could be, say, Mix'n'match catalogs ("external ID/url/name/description/instance of").

Sep 8 2019, 9:07 AM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request

Sep 6 2019

Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

Started some design notes of such a product: https://meta.wikimedia.org/wiki/Gulp

Ooooh, thank you Magnus, that's a really great first pass at thinking about that. For the reuse by something like Listeria or tabernacle would it make sense to store the associated Wikidata item (or also commons MediaInfo id?) with the page ? (so that you wouldn't have to query those pages for them to do things like add properties?) One thing that kindof "fails" for me in the user experience of the current Petscan->Pagepile->Tabernacle workflows (and I am thinking this might be true in other workflows as well) is that the endtool expects _only_ Wikidata items, so if I don't generate a Wikidata list first, the tool either needs to have code to retrieve that or you have to generate a new list. If there was a second column with the optional Wikidata id, it would probably make lists made with one wiki in mind more portable.

Sep 6 2019, 6:24 AM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request
Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

Started some design notes of such a product: https://meta.wikimedia.org/wiki/Gulp

For the List data structure, in addition too or as part of the Description: would it make sense to require a field for "Source" of the data (i.e. Petscan id, shorturl for query, etc) so that anyone "seeing" the pile could go to it, recreate the query/input, and modify it? (kindof like how folks use the Listeriabot lists).

Sep 6 2019, 6:23 AM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request
Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

Minimum viable product

  • Import from various sources
    • All sources offered in PagePile
  • Export to various places
    • All consumers offered in PagePile

How is this supposed to work? As far as I can tell, these imports and exports would have to go through the PagePile tool in some form, so to me these read like requirements that can only be fulfilled by one person: the PagePile maintainer.

Sep 6 2019, 6:22 AM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request

Sep 5 2019

Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

Started some design notes of such a product: https://meta.wikimedia.org/wiki/Gulp

Sep 5 2019, 1:33 PM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request
Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

OK, some initial thoughts and remarks on this:

  • I have actually rewritten Listeria in Rust, to use the Commons Data: namespace (aka .tab files) to store the lists, and use Lua to display them.
  • I think the Commons Data: namespace would technically work for a generalized "list storage", thugh it seems to be a bit of abandonware (will this feature be long-term supported by the WMF?)
  • Commons Data: namespace, if supported, would also have the proper scaling, caching etc. that PagePile is lacking
  • It should, in principle, be possible to change PagePile to write new piles to the Commons Data: namespace, and return queries from there. That would give the new list storage a running start. We can replace PagePile later.
  • Drawbacks of Commons Data: namespace are (a) cell size limit (400 characters, so should work for simple page lists), and (b) total page size (thus limiting the max list length)
  • If Labs were to offer a scalable, backed-up object store for tools, that might be better suited for general list management
  • Much of the "average Wikimedian" integration will have to come from (user-supplied) JavaScript, such as "snapshot this category tree" or something. I doubt waiting for WMF would be a timely solution.
  • Short term, we (I?) could write a slim web API on Labs that abstracts the implementation away, offering a to-be-discussed set of functions (create/amend/remove list etc). Initially, this could run on PagePile in the background, or Commons Data: namespace, or even both (large lists go to pagepile, short ones into a MySQL database or Commons Data: namespace, etc.)
Sep 5 2019, 1:09 PM · WMSE-Tools-for-Partnerships-2020, Technical-Tool-Request