Replied there, cc @Prototyperspective

@Aklapper https://github.com/magnusmanske/quickstatements_rs/commit/3e50c3a9356e1ddc34a0de7cfd345a4a78b51be5

FWIW I added a check on every ~20s batch edit (to not overload the Wikidata API) if the user is blocked, which should then block all batches by this user. Please let me know if this is effective, or if it has unintended side effects.

FWIW I have added a "cross-cats" functionality to my "WD-infernal" API. Any tool can submit a query for a Wikidata item that is a category, a (Wikipedia) language, and a search depth. Output format is JSON. CORS open for all. Just need some nice interface...

Never mind, VPN was on

@Jdforrester-WMF Thanks!

I am trying to fix my tools to use the new tables, once available. I am looking at the DB replicas, which have them enables but not complete. I have some questions which I did not find answered above:

• How do I get the img_user_text via the filerevison table? Which field is the ID to that, and which table would it reference?
• The fr_timestamp field appears to be an integer. How do I convert that to the previous text format, preferably within the query itself?
• What table does the fr_description_id reference?

Thanks!

I have limited the RAM for PetScan via systemctl, which also should restart PetScan after VM reboot. Please let me know if that doesn't take care of the problem.

Working on it.

Restarted the VM, seems to work now

Never mind, got it to work, bastion didn't have my keys for some reason anymore

Yes I meant the OpenStack keys

My key pair was gone so I just made a new PetScan one, if that helps?

The instance is still inaccessible via ssh to me.

UPDATE: The instance has restarted but apparently has no longer the key pair associated. I tried to ssh in from login.toolforge.org and bastion.wmflabs.org, without success.

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

I have dropped the indices for the table, it works now.

Trying to repair it manually:

REPAIR TABLE group2view USE_FRM;
...
The storage engine for the table doesn't support repair

See https://stackoverflow.com/questions/2428738/how-do-you-fix-a-mysql-incorrect-key-file-error-when-you-cant-repair-the-tabl

@Andrew Works again, thanks!

Yes. No joy.

Sorry, but I don't think it's done yet.

@rook Thanks, the project is called baglama2. At the moment the volume size shows 60GB, which I suggest to increast to 100GB.

It does not, unfortunately:

Fixed now.

Or put the tool name in there so I can see it's mine...

tl;dr: I was on vacation with no ssh access. Webservice restarted (which could/should be done by Toolforge automatically but isn't), seems to work again.

This is still happening, reproducibly, so not a stale server or something

Thanks. It would be helpful if that were mentioned in https://www.mediawiki.org/wiki/Help:Extension:WikibaseCirrusSearch

Works again

The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

Hi @fnegri if this will take a while, could you throw me a few more DB connections for mix-n-match on ToolsDB in the meantime? I have a lot of background tasks running to keep everything useful and in sync with Wikidata.

Thnaks, that was it. Could have a better error message but if the old host is shut down soon anyway...

Update: API response is {"code":"unexpected-error","message":"Unexpected error"} for the second call.

At least it should fail with an "edit conflict" error or something, but since the statements are different, I would expect both calls to succeed

I am currently re-running all wikis to fix a trivial bug that led some articles to be marked as done when they were not. The issue is closed on a code level.

yowiki does not have any candidates. The wiki drop-down did show all pages in the database, not the available ones. This now fixed. I will investigate why yowiki has no more candidates.

Thanks, deployed, please test with your usecase.

I have

• deleted petscan4
• deleted petscan4a
• created petscan5 (as debian-12.0-bookworm)
• created a new web proxy to point to petscan5 (only seens to offer https://petscan.wmcloud.org/ instead of the previous https://petscan.wmflabs.org/ but redirect seems to work fine)

AFAICT this can be closed now.

Update: Building new VM now

Hello, didn't see this until now. Happy to move to to a new VM. Can you provide (and throw in a bit more disk space please?), or should I do this myself?

Silly question: Why don't we host the Docker images in the gitlab repo container service at https://gitlab.wikimedia.org?

See also: https://www.docker.com/increase-rate-limits/

Thanks @fnegri I changed the ticket title. Seems like an oversight not to add PHP?

Also no php as user:

magnus@tools-bastion-13:~$ php
-bash: php: command not found

Figured it out

FWIW this is the mix-n-match tool, in case it matters.

This tool would not scale to today's Wikidata size

It did happen three times in a row before I reported it, just worked now. Maybe a github problem or a server not seeing internet?

This appears to be reolved, and replaced by the too many open files bug (another Phab ticket is open).

@dcaro thanks, done

Got it working. For future reference:

I have some updates. With php.ini as

extension_dir="/layers/fagiani_apt/apt/usr/lib/php/20210902"
extension=mysqlnd
extension=mysqli

it loads the mysql library. Still fails with some password error but that seems like something code-wise to debug.

For my own testing, is there a way to get a shell into the docker container? I have seen the docs for the webservice one, but that doesn't seem to fly for non-webservice ones?

I created a custom php.ini and use it in the command:

php8.1 -c /data/project/mix-n-match/mixnmatch_rs/php.ini /data/project/mix-n-match/scripts/person_dates/update_person_dates.php 48922

The php.ini:

extension_dir="/layers/fagiani_apt/apt/usr/lib/php/20210902"

I also added a whole bunch of packages to the Aptfile, just in case.

@taavi I added php-mysql to the Aptfile and rebuild the image, same error.

Thanks @taavi Is there a list of packages required for a toolforge-equivalent php setup?

Looks like the php setup is incomplete. Running php8.1 path/to/my/php/script dies with Fatal error: Uncaught Error: Class "mysqli" not found.

Would it be possible to add a symlink in $PATH somewhere? php is stable, php8.1 maybe less so

Ah, I found /layers/fagiani_apt/apt/usr/bin/php8.1, I guess it's just not symlinked anywhere?

I added an Aptfile as described and rebuild the image, but:

tools.mix-n-match@tools-sgebastion-10:~/mixnmatch_rs$ toolforge jobs run --mem 2000Mi --cpu 1 --wait --mount=all --image tool-mix-n-match/tool-mix-n-match:latest --command "php --version" single
ERROR: job 'single' failed:
+-------------+------------------------------------------------------------------+
| Job name:   | single                                                           |
+-------------+------------------------------------------------------------------+
| Command:    | launcher php --version                                           |
+-------------+------------------------------------------------------------------+
| Job type:   | one-off                                                          |
+-------------+------------------------------------------------------------------+
| Image:      | tool-mix-n-match/tool-mix-n-match:latest                         |
+-------------+------------------------------------------------------------------+
| File log:   | no                                                               |
+-------------+------------------------------------------------------------------+
| Output log: |                                                                  |
+-------------+------------------------------------------------------------------+
| Error log:  |                                                                  |
+-------------+------------------------------------------------------------------+
| Emails:     | none                                                             |
+-------------+------------------------------------------------------------------+
| Resources:  | mem: 2000Mi, cpu: 1                                              |
+-------------+------------------------------------------------------------------+
| Mounts:     | all                                                              |
+-------------+------------------------------------------------------------------+
| Retry:      | no                                                               |
+-------------+------------------------------------------------------------------+
| Status:     | Failed                                                           |
+-------------+------------------------------------------------------------------+
| Hints:      | Last run at 2024-03-12T14:23:38Z. Pod in 'Failed' phase. State   |
|             | 'terminated'. Reason 'Error'. Started at '2024-03-12T14:23:39Z'. |
|             | Finished at '2024-03-12T14:23:39Z'. Exit code '127'.             |
+-------------+------------------------------------------------------------------+

I also tried commands like "sh -c 'php --version'" but no success.

Yes, the third attempt passed. However, 500 errors should be cause for concern, even if it eventually runs through.

Now happening repeatedly for the mix-n-match tool:

toolforge build start https://github.com/magnusmanske/mixnmatch_rs/

this has resolved itself

Yes, works for me, thanks