Apologies, my mistake, needs two underscores! All is well.

Thanks, better, but not quite there yet:

tools.wdqsbe@tools-sgebastion-10:~$ sql local
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 73249977
Server version: 10.1.44-MariaDB MariaDB Server

Found the problem, on my side. Apologies.

Consumer key is b5dc46b99399f49d03757216abd14e66 (QuickStatements). I didn't change the URL, and I think the toolforge clock is working fine.

Note: Oauth 1, not 2!

The reason I used bitbucket in the past was a (long removed) policy on github that limited the number of free repos. I also have some repos on github, which is owned by M$ and probably not very libre either.

Relevant code: https://github.com/magnusmanske/mediawiki_rust/blob/master/src/api.rs#L644

FWIW I started developing an new tool called GULP, now under early development.

After T329853 can we please add something so databases don't stay vanished until I complain at a "a proper support venue" (which seems to be only IRC, with mattermost link broken)?

Thanks, I forgot to check the "URL needs to start with" checkbox...

Just for posterity, I'd like to mention my own wikibase diff engine in Rust: https://gitlab.com/tobias47n9e/wikibase_rs/-/blob/master/src/entity_diff.rs

3 CPUs would be plenty, I think that's actually the per-tool limit, including the webservice?

This stopped working about an hour ago

I have successfully moved all data over to trove, and took a snapshot of the toolsdb version.
The web interface and the background tools have been switched over to the trove version and are reading/writing successfully.
As far as I am concerned, the toolsdb s51203__baglama2_p can be deleted.
Should I do that, or do you want to do the honors?

It took a few days but the database has been successfully copied over to trove. I am taking a final mysqldump from toolsdb now, then s51203__baglama2_p can be removed. I will post here when it's done.

Yes that's fine. Let me know when I can re-import it

Seems to have happened again just now. I was importing a rather large table (views), that has been running for hours(?). Not sure if that's related.

In other words, I would be hesitant to switch to a system where I have to manually restart the MySQL server every other week. I don't have time to work on all my tools as I would like, I can't run around kicking infrastructure as well.

OK I restarted the DB instance via horizon, when I saw that even horizon couldn't connect to it any more.

More concise, from toolforge:

tools.glamtools@tools-sgebastion-10:~$ mysql --defaults-file=~/replica.trove.my.cnf -h pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud baglama2
ERROR 2002 (HY000): Can't connect to MySQL server on 'pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud' (115)

Everything worked fine but (after a few days) I now can't connect to the instance any more:

Used command:  /usr/bin/ssh -v -N -S none -o ControlMaster=no -o ExitOnForwardFailure=yes -o ConnectTimeout=10 -o NumberOfPasswordPrompts=3 -i /Users/mm6/SpiderOak Hive/Configurations/ssh/id_rsa -o TCPKeepAlive=no -o ServerAliveInterval=60 -o ServerAliveCountMax=1 magnus@tools-login.wmflabs.org -L 61284:pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud:3306

@Andrew I have created a new baglama2 DB there, and am currently importing the tooldb database. For that, I made a new replica file (~/replica.trove.my.cnf), and run in a screen:

mysqldump --defaults-file=~/replica.my.cnf --host=tools-db s51203__baglama2_p | mysql --defaults-file=~/replica.trove.my.cnf -h pwupvyu6i6k.svc.trove.eqiad1.wikimedia.cloud baglama2

This seems to be mostly done (after ~14h). I will then point everything to the trove DB. I might also dump the tooldb, and set up a regular trove dump. What's the best place to store compressed dumps, just the tool directory?

In T323502#8454473, @Andrew wrote:

In T323502#8453521, @Magnus wrote:

Is there a case for these large ones to move to their own instance, to take pressure of the toolforge DB system?

The answer to that question is YES! If you're willing to pick a database (not necessarily mix-n-match) and be a test subject I'd love to work on you with that.

Maybe this is a more general issue as well. I checked and it looks like I "own" 4 of the 10 largest tool databases. Is there a case for these large ones to move to their own instance, to take pressure of the toolforge DB system?

@nskaggs Thanks, the Cloud VPS DB option looks very interesting, but I think it would be overkill to move the 120GB DB over. I'll stick with the 10 connections for now, unless you recommend that this is hosted more efficiently (for both you and me) on Cloud VPS.

Thank you @nskaggs!

Also, the per-container limit still seems to be 1 CPU?

In T323502#8419755, @aborrero wrote:

Accepted the CPU quota change. The DB request is pending discussion.

In T323502#8419749, @taavi wrote:

In T323502#8416721, @Magnus wrote:

And while I'm at it, I would like to request that the maximum number of database connections be increased to 20. This is mostly for the tool user database, not so much for the DB replicas, in case that makes a difference.

Could you explain a bit why this is needed?

And while I'm at it, I would like to request that the maximum number of database connections be increased to 20. This is mostly for the tool user database, not so much for the DB replicas, in case that makes a difference.

--wait instead of --continuous seems to work?

I have other k8s jobs scheduled/running for this tool, they all work fine

The command is a thin bash wrapper around a Rust binary, compiled on toolforge. Binary starts just fine when run manually in shell.

Never mind, I just copied the binary to the tools directory and run it from there.

Moved all the jobs now except one, which depends on mysqldump, but that command is not in tf-bullseye-std. Is there any way to access that command from k8s?

Done (already using k8s but someone left the cronjobs in place for just-in-case restarting, this is removed now).

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Done.

Update: I migrated most of the cronjobs (except two) to kubernetes. Best I can do for now, I'll have a look at the remaining ones again once I migrate all my other tools.

Done.

Hi, as you may know, I am a massive user of the toolforge environment. I am also an eager early adopter of new technologies. I will use this ticket (and not the other dozen or so I got for other tools) for some feedback, until it's specific to other tools.

I think I fixed the URL for Special:OAuth, someone please try it in the wild

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Fixed

Different bug though. Fixed now.

Thanks, I added descriptions and toolhub now picked it up.

So it does not look like the crawler used the JSON I provided. For example, "drawshield" is clearly in https://magnustools.toolforge.org/toolinfo_buggregator.json but it does not show up in Toolhub search.

Ah it looks like the crawler runs once every hour: https://toolhub.wikimedia.org/crawler-history

Adding my own list of my tools, matched to the toolhub name where I could. Note that some of these are deactivated/broken (see notes), some are JavaScript on-wiki so not sure if that counts. The "open/closed' tickets are biased towards "open" while I go through them over time, but could become a quality score indicator down the line.

I think I had a misconfiguration in the cronfile, this is fixed now. Please re-open/ping me if this still happens.

First time I see this issue here, will investigate.

This was broken by the "toolforge subdomain migration".

Turns out that single catalog takes ~21GB:

The collected data (from the BaGLAMa2 tool) is quite valuable for GLAM, including the historic ones, which is why I try to keep all of it.

Fixed now.

In T282158#7100475, @Sannita wrote:

I'll just put this bug here, because I'm not 100% sure I can attend the session: https://github.com/magnusmanske/listeria_rs/issues/67

This is a rather urgent bug we noticed a couple of months ago, and might at this point be related to the 90/10 issue Andrew was discussing in a previous comment.

A few notes:

• happy to help with any new development, as time allows; prioritized list preferred
• issue tracker at https://github.com/magnusmanske/listeria_rs but it's a bit messy, partially because no one reads the existing issues...
• the "killed" errors come from Toolforge killing the processes even though there is enough memory, but they allocate memory in large blocks which is suspicious, apparently. I have an idea how to get around that, but will require a bit of time
• the "central location" is the Commons Data: namespace. This works ~80-90%, and requires a final push, both in Rust and Lua. See http://magnusmanske.de/wordpress/?p=650

Turns out flickr retired the secure.flickr.com domain without forwarding. Removed the "secure" but still using https, should work again.

Also, doesn't seem to work for "has a birth date"?
https://www.wikidata.org/w/index.php?search=haswbstatement%3AP21%3DQ6581072++haswbstatement%3AP569&title=Special:Search&profile=advanced&fulltext=1&advancedSearch-current=%7B%7D&ns0=1&ns120=1

In T179879#6859731, @So9q wrote:

Both of these can be done using CirrusSearch API and can't they?

Restarted wikidata-todo.

It does work, however, since there is currently not a single page on euwiki where the latest revision has a wp10 score, the results will be empty.