Retrying failed jobs is not always acceptable. There should be an option to try jobs only once.

I have now realized that I have the same problem with the bot GiftBot on dewiki. Several scripts are stuck in endless loops because they cannot get valid tokens.

The bots authenticate via OAuth owner-only consumers. The source code is in ~taxonbot on the instance taxonbot.dwl on WMCS.

I have flushed the jobs and rescheduled most of them on the 7th minute, like I did for sge. I hope that helps with the load (if it is that).

I recompiled vim on buster, this shouldn't be necessary anymore.

I'm looking forward to the custom containers, and in the meantime I compiled tdb/fdm.

Apparently k8s tries to run the command a second time in case of failure. Is that intentional?

Could this be because I already have 3 deployments (1 k8s webservice and 2 continuous jobs) and adding another continuous job would exceed my quota?

Can I get any progress on this?

I want to to process daily emails that contain data for a commons data table.

If you want, you can now lower CPU quota to 65 cores and RAM quota to 146.048 MB.

In T283293#7114669, @aborrero wrote:

Hi,

Let me see if I understand the request correctly. Are you talking about +36 CPU, +36 GB ram and +160 GB cinder storage?

Specifically:

• CPU from 45 to 81
• RAM from 182GB to 218GB
• Storage from 160GB to 320GB

Please, confirm this is your request.

I confirm. It also includes the new flavor.

Um, no. We can't move forward with transition yet. Afais, instance resizing isn't enabled yet, but that's a prerequisite. After that, we'll probably need a few days, depending on my and Doc Taxon's availability.

Should be fixed now.

I wonder why it worked with the new tclsh binary and tcl libraries on the old tcl container then.

Ok, so now my kubernetes webservice of type tcl doesn't work anymore, it is missing libmariadbclient.so.18.

I thought I needed the packages as runtime on the exec nodes. I don't know if anything has changed on them but I do no longer seem to need this as it just works fine now.

As it turns out, I found an old workaround that does away with the dependency of bc in the build process.

In T210519#4785439, @Bjs wrote:

The first sentence above of course applies only if the user has the corresponding rights, which I have. I also checked other cases: All my edits were auto-reviewed, only the moves from user namespace to article namespace not.

The repo apparently was archived in gerrit directly. This is now undone. There also was no +2 right for me, which is now added. A jenkins integration would be nice to have but in the meantime I can make do with submitting directly.

Despite https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL and IRC suggesting otherwise, the repo is still archived and gerrit still refuses patches.

Could you please un-archive labs/tools/giftbot?

You're right, it seems that we don't need so much RAM at the moment. At least it was necessary in the past or otherwise the instance would be really slow. I think we could first try with an xlarge instance.

I went through my table creation statements and added engine statements. I dropped four tables that I want to recreate and changed the engine of the remaining to InnoDB. Thank you for your prompt reaction!

< yuvipanda> !log tools set complex_values slots=300,release=trusty for tools-exec-gift-trusty-01.tools.eqiad.wmflabs
This seems to have done the trick, thank you!
@chasemp The old instance can be cleaned up now.

Killing the Precise instance (and defaulting to Trusty) shouldn't pose a problem. I switched my code to Trusty (I didn't even know anymore that I had to hardcode that) and I will test it with the next regular run starting on the 1st of March. (Also, I'm kinda swamped with school stuff but I'm glad to have found a minute for this.)

< annika> chasemp: […] I guess you mean: You will provide a trusty exec node with an appropriate queue? And I will just use that instead of the current one?
< chasemp> annika: basically, a second trusty node in the same queue for you to migrate to
< annika> that would be fine

So, we deleted the old instace before creating the new one anyway because we wanted to keep the name. You can now readjust the quota.

Considering my earlier comment (bigram instance), the needed numbers would be 1+8=9 cores and 2+36=38GB. But otherwise you're right. And I actually planned to do the transition without additional temporary resources. But it surely would give us more security.

I'm not aware of any script errors. Maybe there are db queries or other things that cause it to hang.

We need more RAM for data processing. We are not aware of other possibilities. Afais, "bigram" has even more RAM, so we'd like to have enough room for 1 small and 1 bigram instance.

Seems to work again.

I do.

@doctaxon apparently they do, but it seems the claim is unsubstantiated

(Ya.)

I had no illusions of resolving the ticket with my summary alone (which should be clear if you read my comment carefully). I just wanted to document what's happening. But thank you for mentioning the root mail (which I couldn't remember).