My pleasure. Did I save mwdiffs and mwpersistence? That's the goal. If so should one of us update the announcement to [Cloud-announce]?

I successfully built the singleuser image with jupyterlab=3.6.3 and this PR: https://github.com/toolforge/paws/pull/309

Maybe not quite that. Looks like mwpersistence requires deltas -> yamlconf -> PyYAML 5.4.1:

My guess is that the implicated libraries (mwdiffs and mwpersistence) are written in Python 2 and can't upgrade to the latest version of PyYAML, but that's just a guess.

I think removing those libraries simply causes pip to resolve the dependency to a later version of PyYAML which doesn't have the issue, as seen here: https://github.com/flyteorg/flytekit/pull/1752/files

I tried bumping jupyterlab to 3.6.3 as seen in this commit: https://github.com/toolforge/paws/pull/308/commits/425a53a6449198beca9e3466c32f3604bdfbe31e

Thank you!

2fa request file (following instructions here) is at:

Yes we'd like to continue to use toolsdb.

Okay I've stopped the gridengine job using tools.enwp10@tools-sgebastion-10:~$ webservice --backend=gridengine stop and applied the redirect using the commands you linked @bd808. It looks like it's working. Do we still need to delete our toolsdb database as well?

Our tool has been running at wp1.openzim.org for multiple years now. I don't think we actually require the redirect, but if it's easy enough to setup we could look into it. Thanks for the pointers.

@nskaggs So it sounds like the answer is "No, we cannot keep our toolsdb database" and we would have to migrate to the linked VPS solution?

That's right. We're still using the Toolforge database, but we don't have any jobs running in GridEngine.

Copied from the duplicate bug:

Okay I've migrated s51114_enwp10 to s51114__enwp10. There are a few tables left in the old database, but they are leftovers from the tool migration process and can be deleted.

Shouldn't step 5 be CREATE DATABASE not CREATE TABLE?

s51114_enwp10 is definitely still in use. I don't know how to rename it safely, so would need help with that.

Yes it's all resolved now, you can publish it if you like, I don't have an opinion and am not familiar with the policy.

It worked! Thanks so much both of you for your help!

In T269513#6683826, @DannyS712 wrote:

Should T269898: Bot needs account unlocked be merged here? Or is there anything that can be done publically there?

In T269513#6672167, @Reedy wrote:

That seems to be @OlegAlexandrov who is still active on enwiki. CC'd them to this task...

You are correct, this database is quite old. Thank you for your help with this, I have confirmed that I can read from the database again.

How do you not have the login password for it? Does someone else?

Regarding https://phabricator.wikimedia.org/T269513#6671553 it looks like the credentials in replica.my.cnf allow login access to the databases, but the user doesn't have permissions to s51114_enwp10 which is the name of the database that the tool uses on tools.db.svc.eqiad.wmflabs. Would it be possible to grant permissions?

Thank you for this information, but I don't have access to the login password for WP 1.0 bot, I only have the bot password.

I've got a new replica.my.cnf, but I'm getting the following:

Thank you so much Andrew! How do I get new credentials?

I also leaked the "bot password" for the WP 1.0 Bot. I don't know where or how to reset or rescind this password. When I log into mediawiki.org I see "Audiodude@WP_1.0_bot" but I believe that makes edits as my user account. The account I need to reset is "WP 1.0 bot@WP_1.0_bot"

Thank you, sorry about this. I looked around a bunch for my recovery codes but I can't remember at all where I put them!

Okay I've deployed the fix to WP 1.0 bot as of tonight's update. It should be good the next time you run the numbers. Please let me know.

Wanted to come here and acknowledge this issue. I've opened a bug against the WP 1.0 project to fix this. It should be taken care of this weekend. Thank you @DeltaQuad for the link to mwclient workaround!

That seems to have resolved the issue!

I'm going to try replacing DictCursor with SSDictCursor and see if that helps.

Hi @Kelson , I think you gave extra parameters to your restart command. I tried:

Okay, I had previously added a setenv command for adding the PERL5LIB to the environment, but I didn't add it for list2.fcgi and log.fcgi, which now have their own sections in their configs for the environment variable. Things seem to be working now

This isn't entirely accurate. The tool was in fact migrated to stretch last week. However, certain pages may still be erroring out, and that needs to be addressed. I will look at it later today when I have my laptop.

And yes, the ORM I was speaking of was SQLAlchemy. It was much too slow with the session management strategy I was using, and manually managing multiple sessions and object eviction in order to speed it up would make the code too complicated in my opinion and negate the benefits of using the ORM.

Thanks for the comments. I realized that the test db never needs to be live in production and that helped me figure out the solution.

That looks promising for the Travis CI environment yes. I guess my problem is that I'm not developing locally, I'm developing directly on the Toolforge instance and I guess I just need to provision a database for test there?

The web tool (https://tools.wmflabs.org/enwp10) looks up to me. What specifically is not working?

Restarted it again. Let me know if it works.

I just restarted it. The tool seems to die every 30 days or so, this last happened on Sept 28.

Actually, I believe plaintext would be more useful, but HTML would be a good starting point since it's reasonable to strip out.

In fact the XML is not the problem, it's the wiki markup. From what I understand, the wiki markup is not defined by any formal grammar, so parsing it is non-trivial. Tools like mwparserfromhell help, but can be difficult to configure and not always accurate. There's also of course the problem of recursive template expansion.