Page MenuHomePhabricator

Magnus (Magnus Manske)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Oct 23 2014, 3:02 PM (439 w, 2 d)
Availability
Available
LDAP User
Magnus Manske
MediaWiki User
Unknown

Recent Activity

Thu, Mar 23

Magnus added a comment to T332762: New tool not allowed to connect to toolsdb.

Apologies, my mistake, needs two underscores! All is well.

Thu, Mar 23, 8:30 AM · cloud-services-team, Toolforge
Magnus added a comment to T332762: New tool not allowed to connect to toolsdb.

Thanks, better, but not quite there yet:

tools.wdqsbe@tools-sgebastion-10:~$ sql local
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 73249977
Server version: 10.1.44-MariaDB MariaDB Server
Thu, Mar 23, 8:29 AM · cloud-services-team, Toolforge

Wed, Mar 22

Magnus created T332762: New tool not allowed to connect to toolsdb.
Wed, Mar 22, 8:59 AM · cloud-services-team, Toolforge

Tue, Mar 7

Magnus closed T330880: OAuth authentication changed? as Resolved.

Found the problem, on my side. Apologies.

Tue, Mar 7, 11:08 AM · MediaWiki-extensions-OAuth

Mon, Mar 6

Magnus added a comment to T330880: OAuth authentication changed?.

Consumer key is b5dc46b99399f49d03757216abd14e66 (QuickStatements). I didn't change the URL, and I think the toolforge clock is working fine.

Mon, Mar 6, 8:50 AM · MediaWiki-extensions-OAuth

Wed, Mar 1

Magnus added a comment to T330880: OAuth authentication changed?.

Note: Oauth 1, not 2!

Wed, Mar 1, 3:34 PM · MediaWiki-extensions-OAuth
Magnus added a comment to T330846: Help community projects on Bitbucket™ to evaluate the adoption of something less external and libre (for example Wikimedia GitLab).

The reason I used bitbucket in the past was a (long removed) policy on github that limited the number of free repos. I also have some repos on github, which is owned by M$ and probably not very libre either.

Wed, Mar 1, 2:31 PM · Epic
Magnus added a comment to T330880: OAuth authentication changed?.

Relevant code: https://github.com/magnusmanske/mediawiki_rust/blob/master/src/api.rs#L644

Wed, Mar 1, 2:25 PM · MediaWiki-extensions-OAuth
Magnus created T330880: OAuth authentication changed?.
Wed, Mar 1, 2:23 PM · MediaWiki-extensions-OAuth

Fri, Feb 24

Magnus added a comment to T231891: Create a Generic List-building tool that can meet and exceed the applications of Pagepile.

FWIW I started developing an new tool called GULP, now under early development.

Fri, Feb 24, 2:57 PM · Campaign-Registration, WMSE-Content-partnerships-support-2021-Software-development, Campaign-Tools, Technical-Tool-Request

Feb 17 2023

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

After T329853 can we please add something so databases don't stay vanished until I complain at a "a proper support venue" (which seems to be only IRC, with mattermost link broken)?

Feb 17 2023, 10:01 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Feb 16 2023

Magnus created T329853: Trove database vanished.
Feb 16 2023, 3:39 PM · Toolforge

Feb 7 2023

Magnus closed T329066: Oauth2 with Rust as Resolved.

Thanks, I forgot to check the "URL needs to start with" checkbox...

Feb 7 2023, 4:10 PM · MediaWiki-extensions-OAuth
Magnus created T329066: Oauth2 with Rust.
Feb 7 2023, 3:45 PM · MediaWiki-extensions-OAuth

Jan 24 2023

Magnus added a comment to T282796: Design a file format to represent Wikibase edits.

Just for posterity, I'd like to mention my own wikibase diff engine in Rust: https://gitlab.com/tobias47n9e/wikibase_rs/-/blob/master/src/entity_diff.rs

Jan 24 2023, 1:54 PM · OpenRefine
Magnus added a comment to T327748: Remove 1 CPU limit for toolforge-jobs.

3 CPUs would be plenty, I think that's actually the per-tool limit, including the webservice?

Jan 24 2023, 1:52 PM · Toolforge
Magnus added a comment to T327775: toolforge-jobs script stopped working.

This stopped working about an hour ago

Jan 24 2023, 11:57 AM · Toolforge Jobs framework
Magnus created T327775: toolforge-jobs script stopped working.
Jan 24 2023, 11:57 AM · Toolforge Jobs framework
Magnus created T327748: Remove 1 CPU limit for toolforge-jobs.
Jan 24 2023, 9:31 AM · Toolforge

Jan 9 2023

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

I have successfully moved all data over to trove, and took a snapshot of the toolsdb version.
The web interface and the background tools have been switched over to the trove version and are reading/writing successfully.
As far as I am concerned, the toolsdb s51203__baglama2_p can be deleted.
Should I do that, or do you want to do the honors?

Jan 9 2023, 11:04 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 23 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

It took a few days but the database has been successfully copied over to trove. I am taking a final mysqldump from toolsdb now, then s51203__baglama2_p can be removed. I will post here when it's done.

Dec 23 2022, 9:50 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 20 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Yes that's fine. Let me know when I can re-import it

Dec 20 2022, 5:36 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Seems to have happened again just now. I was importing a rather large table (views), that has been running for hours(?). Not sure if that's related.

Dec 20 2022, 3:58 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

In other words, I would be hesitant to switch to a system where I have to manually restart the MySQL server every other week. I don't have time to work on all my tools as I would like, I can't run around kicking infrastructure as well.

Dec 20 2022, 2:37 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

OK I restarted the DB instance via horizon, when I saw that even horizon couldn't connect to it any more.

Dec 20 2022, 2:36 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

More concise, from toolforge:

tools.glamtools@tools-sgebastion-10:~$ mysql --defaults-file=~/replica.trove.my.cnf -h pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud baglama2
ERROR 2002 (HY000): Can't connect to MySQL server on 'pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud' (115)
Dec 20 2022, 1:11 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Everything worked fine but (after a few days) I now can't connect to the instance any more:

Used command:  /usr/bin/ssh -v -N -S none -o ControlMaster=no -o ExitOnForwardFailure=yes -o ConnectTimeout=10 -o NumberOfPasswordPrompts=3 -i /Users/mm6/SpiderOak Hive/Configurations/ssh/id_rsa -o TCPKeepAlive=no -o ServerAliveInterval=60 -o ServerAliveCountMax=1 magnus@tools-login.wmflabs.org -L 61284:pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud:3306
Dec 20 2022, 1:10 PM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 14 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

@Andrew I have created a new baglama2 DB there, and am currently importing the tooldb database. For that, I made a new replica file (~/replica.trove.my.cnf), and run in a screen:

mysqldump --defaults-file=~/replica.my.cnf --host=tools-db s51203__baglama2_p | mysql --defaults-file=~/replica.trove.my.cnf -h pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud baglama2

This seems to be mostly done (after ~14h). I will then point everything to the trove DB. I might also dump the tooldb, and set up a regular trove dump. What's the best place to store compressed dumps, just the tool directory?

Dec 14 2022, 9:56 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 9 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Is there a case for these large ones to move to their own instance, to take pressure of the toolforge DB system?

The answer to that question is YES! If you're willing to pick a database (not necessarily mix-n-match) and be a test subject I'd love to work on you with that.

Dec 9 2022, 9:21 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 8 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Maybe this is a more general issue as well. I checked and it looks like I "own" 4 of the 10 largest tool databases. Is there a case for these large ones to move to their own instance, to take pressure of the toolforge DB system?

Dec 8 2022, 9:14 AM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

@nskaggs Thanks, the Cloud VPS DB option looks very interesting, but I think it would be overkill to move the 120GB DB over. I'll stick with the 10 connections for now, unless you recommend that this is hosted more efficiently (for both you and me) on Cloud VPS.

Dec 8 2022, 9:09 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 6 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Thank you @nskaggs!

Dec 6 2022, 12:25 PM · Tools, cloud-services-team (FY2022/2023-Q3)

Dec 1 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Also, the per-container limit still seems to be 1 CPU?

Dec 1 2022, 1:38 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

Accepted the CPU quota change. The DB request is pending discussion.

Dec 1 2022, 1:34 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

And while I'm at it, I would like to request that the maximum number of database connections be increased to 20. This is mostly for the tool user database, not so much for the DB replicas, in case that makes a difference.

Could you explain a bit why this is needed?

Dec 1 2022, 1:33 PM · Tools, cloud-services-team (FY2022/2023-Q3)

Nov 23 2022

Magnus added a comment to T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).

And while I'm at it, I would like to request that the maximum number of database connections be increased to 20. This is mostly for the tool user database, not so much for the DB replicas, in case that makes a difference.

Nov 23 2022, 11:01 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Nov 22 2022

Magnus updated the task description for T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).
Nov 22 2022, 7:39 AM · Tools, cloud-services-team (FY2022/2023-Q3)

Nov 21 2022

Magnus renamed T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool) from Request increased quota for <Replace Me> Toolforge tool to Request increased quota for mix-n-match Toolforge tool.
Nov 21 2022, 2:42 PM · Tools, cloud-services-team (FY2022/2023-Q3)
Magnus created T323502: Move some of magnus's tools to Trove databases (was: Request increased quota for mix-n-match Toolforge tool).
Nov 21 2022, 2:41 PM · Tools, cloud-services-team (FY2022/2023-Q3)

Oct 27 2022

Magnus added a comment to T321791: `toolforge-jobs run` fails with 403 error.

--wait instead of --continuous seems to work?

Oct 27 2022, 3:13 PM · Toolforge Jobs framework, Grid-Engine-to-K8s-Migration
Magnus added a comment to T321791: `toolforge-jobs run` fails with 403 error.

I have other k8s jobs scheduled/running for this tool, they all work fine

Oct 27 2022, 12:02 PM · Toolforge Jobs framework, Grid-Engine-to-K8s-Migration
Magnus added a comment to T321791: `toolforge-jobs run` fails with 403 error.

The command is a thin bash wrapper around a Rust binary, compiled on toolforge. Binary starts just fine when run manually in shell.

Oct 27 2022, 12:02 PM · Toolforge Jobs framework, Grid-Engine-to-K8s-Migration
Magnus created T321791: `toolforge-jobs run` fails with 403 error.
Oct 27 2022, 12:00 PM · Toolforge Jobs framework, Grid-Engine-to-K8s-Migration

Oct 20 2022

Magnus closed T319859: Migrate listeria from Toolforge GridEngine to Toolforge Kubernetes as Resolved.
Oct 20 2022, 2:46 PM · Grid-Engine-to-K8s-Migration

Oct 17 2022

Magnus closed T319896: Migrate mix-n-match from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Never mind, I just copied the binary to the tools directory and run it from there.

Oct 17 2022, 4:05 PM · Grid-Engine-to-K8s-Migration
Magnus added a comment to T319896: Migrate mix-n-match from Toolforge GridEngine to Toolforge Kubernetes.

Moved all the jobs now except one, which depends on mysqldump, but that command is not in tf-bullseye-std. Is there any way to access that command from k8s?

Oct 17 2022, 10:32 AM · Grid-Engine-to-K8s-Migration

Oct 14 2022

Magnus closed T319640: Migrate commons-delinquent from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done (already using k8s but someone left the cronjobs in place for just-in-case restarting, this is removed now).

Oct 14 2022, 12:10 PM · Grid-Engine-to-K8s-Migration
Magnus closed T320093: Migrate topicmatcher from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 14 2022, 10:53 AM · Grid-Engine-to-K8s-Migration
Magnus closed T319729: Migrate erwin85 from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 14 2022, 7:45 AM · Grid-Engine-to-K8s-Migration
Magnus added a comment to T319729: Migrate erwin85 from Toolforge GridEngine to Toolforge Kubernetes.

Done.

Oct 14 2022, 7:45 AM · Grid-Engine-to-K8s-Migration

Oct 13 2022

Magnus closed T320155: Migrate wikidata-todo from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 13 2022, 10:58 AM · Grid-Engine-to-K8s-Migration
Magnus closed T319773: Migrate glamtools from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 13 2022, 10:29 AM · Grid-Engine-to-K8s-Migration

Oct 12 2022

Magnus closed T320050: Migrate sourcemd from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 2:35 PM · Grid-Engine-to-K8s-Migration
Magnus closed T319872: Migrate magnustools from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 2:33 PM · Grid-Engine-to-K8s-Migration
Magnus closed T320151: Migrate wikidata-game from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 2:28 PM · Grid-Engine-to-K8s-Migration
Magnus closed T319988: Migrate quickstatements from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 2:12 PM · Grid-Engine-to-K8s-Migration
Magnus added a comment to T319859: Migrate listeria from Toolforge GridEngine to Toolforge Kubernetes.

Done.

Oct 12 2022, 1:57 PM · Grid-Engine-to-K8s-Migration
Magnus closed T319951: Migrate pagepile from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 1:02 PM · Grid-Engine-to-K8s-Migration
Magnus closed T320153: Migrate wikidata-terminator from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 12:57 PM · Grid-Engine-to-K8s-Migration
Magnus closed T319907: Migrate most-wanted from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 12:51 PM · Grid-Engine-to-K8s-Migration
Magnus closed T320143: Migrate wdrc from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 11:59 AM · Grid-Engine-to-K8s-Migration
Magnus closed T319987: Migrate quest from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 11:54 AM · Grid-Engine-to-K8s-Migration
Magnus added a comment to T319896: Migrate mix-n-match from Toolforge GridEngine to Toolforge Kubernetes.

Update: I migrated most of the cronjobs (except two) to kubernetes. Best I can do for now, I'll have a look at the remaining ones again once I migrate all my other tools.

Oct 12 2022, 10:44 AM · Grid-Engine-to-K8s-Migration
Magnus closed T319744: Migrate fist from Toolforge GridEngine to Toolforge Kubernetes as Resolved.

Done.

Oct 12 2022, 10:42 AM · Grid-Engine-to-K8s-Migration

Oct 7 2022

Magnus added a comment to T319896: Migrate mix-n-match from Toolforge GridEngine to Toolforge Kubernetes.

Hi, as you may know, I am a massive user of the toolforge environment. I am also an eager early adopter of new technologies. I will use this ticket (and not the other dozen or so I got for other tools) for some feedback, until it's specific to other tools.

Oct 7 2022, 10:16 AM · Grid-Engine-to-K8s-Migration

May 31 2022

Magnus added a comment to T112730: Failure to OAuth after login on mobile.

I think I fixed the URL for Special:OAuth, someone please try it in the wild

May 31 2022, 12:48 PM · MediaWiki-extensions-CentralAuth, Mobile, MediaWiki-User-login-and-signup

May 19 2022

Magnus closed T305764: PetScan XSS vulnerability as Resolved.

Fixed

May 19 2022, 12:30 PM · SecTeam-Processed, Vuln-XSS, Tools, Security

May 18 2022

Magnus closed T305774: wikidata-todo: dupe_finder XSS vulnerability as Resolved.

Fixed

May 18 2022, 3:53 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305765: commonshelper XSS vulnerability as Resolved.

Fixed

May 18 2022, 3:52 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305775: usualsuspects XSS vulnerability as Resolved.

Fixed

May 18 2022, 3:48 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305767: file-siblings XSS vulnerability as Resolved.

Fixed

May 18 2022, 3:41 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305771: dnbtools: map2wp XSS vulnerability as Resolved.

Fixed

May 18 2022, 3:37 PM · SecTeam-Processed, Vuln-XSS, Tools, Security

May 17 2022

Magnus closed T305769: magnustools: unused_files XSS vulnerability as Resolved.

Fixed

May 17 2022, 2:13 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305770: magnustools: randomarticle XSS vulnerability as Resolved.

Fixed

May 17 2022, 2:11 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305768: magnustools: wiki2playlist XSS vulnerability as Resolved.

Fixed

May 17 2022, 2:10 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305773: catnap XSS vulnerability as Resolved.

Fixed

May 17 2022, 2:05 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305772: catfood XSS vulnerability as Resolved.

Fixed

May 17 2022, 2:01 PM · SecTeam-Processed, Vuln-XSS, Tools, Security
Magnus closed T305776: GLAMorous XSS vulnerability as Resolved.

Fixed

May 17 2022, 1:58 PM · SecTeam-Processed, Vuln-XSS, Tools, Security

Mar 20 2022

Magnus closed T284372: wikidata-todo throws uncaught exception ToolforgeCommon::getSQL as Resolved.
Mar 20 2022, 3:41 PM · Wikidata, Tools
Magnus added a comment to T284372: wikidata-todo throws uncaught exception ToolforgeCommon::getSQL.

Different bug though. Fixed now.

Mar 20 2022, 3:40 PM · Wikidata, Tools

Oct 28 2021

Magnus added a comment to T294125: Duplicate URLs do not provide user visible reason when rejected.

Thanks, I added descriptions and toolhub now picked it up.

Oct 28 2021, 12:25 PM · Toolhub

Oct 27 2021

Magnus added a comment to T294125: Duplicate URLs do not provide user visible reason when rejected.

So it does not look like the crawler used the JSON I provided. For example, "drawshield" is clearly in https://magnustools.toolforge.org/toolinfo_buggregator.json but it does not show up in Toolhub search.

Oct 27 2021, 12:05 PM · Toolhub

Oct 22 2021

Magnus added a comment to T294125: Duplicate URLs do not provide user visible reason when rejected.

Ah it looks like the crawler runs once every hour: https://toolhub.wikimedia.org/crawler-history

Oct 22 2021, 2:12 PM · Toolhub
Magnus created T294125: Duplicate URLs do not provide user visible reason when rejected.
Oct 22 2021, 2:11 PM · Toolhub

Jul 28 2021

Magnus added a comment to T115650: Create an authoritative and well promoted catalog of Wikimedia tools.

Adding my own list of my tools, matched to the toolhub name where I could. Note that some of these are deactivated/broken (see notes), some are JavaScript on-wiki so not sure if that counts. The "open/closed' tickets are biased towards "open" while I go through them over time, but could become a quality score indicator down the line.

Jul 28 2021, 12:12 PM · Developer-Advocacy (Oct-Dec 2021), User-bd808, Goal, Toolhub, Documentation, Developer-Wishlist (2017)

Jul 21 2021

Magnus closed T282474: tools.topicmatcher update_items_from_sparql.php frequently running Toolforge nodes out of resources as Resolved.

I think I had a misconfiguration in the cronfile, this is fixed now. Please re-open/ping me if this still happens.

Jul 21 2021, 2:07 PM · cloud-services-team (Kanban), Tools, Toolforge
Magnus added a comment to T282474: tools.topicmatcher update_items_from_sparql.php frequently running Toolforge nodes out of resources.

First time I see this issue here, will investigate.

Jul 21 2021, 1:27 PM · cloud-services-team (Kanban), Tools, Toolforge

Jul 12 2021

Magnus closed T286333: Wikidata Game: autodesc endpoint returning 404 as Resolved.
Jul 12 2021, 3:48 PM · Wikidata
Magnus added a comment to T286333: Wikidata Game: autodesc endpoint returning 404.

This was broken by the "toolforge subdomain migration".

Jul 12 2021, 3:47 PM · Wikidata

Jun 30 2021

Magnus added a comment to T285017: glamtools using increasing amounts of space on NFS.

Turns out that single catalog takes ~21GB:

Jun 30 2021, 8:20 AM · Tools
Magnus added a comment to T285017: glamtools using increasing amounts of space on NFS.

The collected data (from the BaGLAMa2 tool) is quite valuable for GLAM, including the historic ones, which is why I try to keep all of it.

Jun 30 2021, 8:08 AM · Tools

Jun 7 2021

Magnus closed T284372: wikidata-todo throws uncaught exception ToolforgeCommon::getSQL as Resolved.

Fixed now.

Jun 7 2021, 4:08 PM · Wikidata, Tools

May 21 2021

Magnus added a comment to T282158: Roundtable on Listeria enhancements.

I'll just put this bug here, because I'm not 100% sure I can attend the session: https://github.com/magnusmanske/listeria_rs/issues/67

This is a rather urgent bug we noticed a couple of months ago, and might at this point be related to the 90/10 issue Andrew was discussing in a previous comment.

May 21 2021, 4:25 PM · Wikimedia-Hackathon-2021

May 13 2021

Magnus added a comment to T282158: Roundtable on Listeria enhancements.

A few notes:

  • happy to help with any new development, as time allows; prioritized list preferred
  • issue tracker at https://github.com/magnusmanske/listeria_rs but it's a bit messy, partially because no one reads the existing issues...
  • the "killed" errors come from Toolforge killing the processes even though there is enough memory, but they allocate memory in large blocks which is suspicious, apparently. I have an idea how to get around that, but will require a bit of time
  • the "central location" is the Commons Data: namespace. This works ~80-90%, and requires a final push, both in Rust and Lua. See http://magnusmanske.de/wordpress/?p=650
May 13 2021, 7:46 PM · Wikimedia-Hackathon-2021

Mar 5 2021

Magnus closed T276253: flickr2commons.toolforge.org stuck at status "Running" due to "Content Security Policy error" as Resolved.

Turns out flickr retired the secure.flickr.com domain without forwarding. Removed the "secure" but still using https, should work again.

Mar 5 2021, 11:33 AM · Tools

Feb 25 2021

Magnus added a comment to T179879: Provide a 5-minute timeout in WDQS for trusted users using OAuth.

Also, doesn't seem to work for "has a birth date"?
https://www.wikidata.org/w/index.php?search=haswbstatement%3AP21%3DQ6581072++haswbstatement%3AP569&title=Special:Search&profile=advanced&fulltext=1&advancedSearch-current=%7B%7D&ns0=1&ns120=1

Feb 25 2021, 9:17 AM · affects-Kiwix-and-openZIM, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service
Magnus added a comment to T179879: Provide a 5-minute timeout in WDQS for trusted users using OAuth.

Both of these can be done using CirrusSearch API and can't they?

Feb 25 2021, 9:13 AM · affects-Kiwix-and-openZIM, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service

Feb 19 2021

Magnus added a comment to T274811: "duplicity" on wikidata-todo.toolforge.org returns "503 Service Not Available" error message.

Restarted wikidata-todo.

Feb 19 2021, 1:14 PM · Tools

Jan 28 2021

Magnus added a comment to T197871: Add articlequality model in PetScan.

It does work, however, since there is currently not a single page on euwiki where the latest revision has a wp10 score, the results will be empty.

Jan 28 2021, 10:36 AM · artificial-intelligence, Machine-Learning-Team, articlequality-modeling

Jan 21 2021

Magnus updated the task description for T269609: Ensure all ToolsDB databases comply with current naming conventions.
Jan 21 2021, 4:52 PM · cloud-services-team, Data-Services