@Jdlrobson I'm sorry, I didn't realize this was waiting on me for a response. I'm not entirely sure what to tell you. My involvement here was mostly noticing somebody else describing the problem on Village Pump and doing some initial troubleshooting.

@Cryptic showed me how to do this on my own:

There's been three different threads on the enwiki Village Pump today talking about changes to search behavior.:

If you could generate the URLs for the other size images, I'd be happy to give them a try from here.

Could you please clarify what software (e.g. Firefox, Chrome, some other tool) you're using to access pages/images that is returning 429s?

My only concern here is that showing thumbnails in a larger size might be valuable for vision-impaired users. At the other end of the spectrum, showing them in a smaller size might be valuable to people on exceptionally slow links and/or small screen sizes such as wearable devices.

In T279141#6986316, @Tacsipacsi wrote:

Firing wikipage.content may however cause new issues as well. As far as I know, currently wikipage.content is always fired with the one and only content block on the page (although the documentation doesn’t state this). If it’s fired with the preview, this expectation is broken and this may cause undefined behavior regarding the rest of the page (as scripts may consider it to be removed from the DOM tree).

In T285103#7161421, @Ammarpad wrote:

Is this duplicate of T261087?

I'm not sure who's supposed to be prioritizing phab tickets, but I've taken the liberty of marking this as high priority. One look at https://www.mediawiki.org/wiki/Talk:Talk_pages_project/Replying shows the lack of ping functionality is the run-away top item of discussion.

Just got one:

Yup, thanks for your help.

No, I've been well-behaved lately. But, I'll be happy to nail a tmux session up on one of the bastions for as long as it takes to generate an example if you like :-)

I'm assuming the attached screenshot from https://en.wikipedia.org/wiki/Wikipedia:Deletion_review/Log/2021_May_10#Category:CS1_maint:_discouraged_parameter is another example of the same issue.

Done

Got another one today:

Thanks for the quick response. That looks like it covers most of the questions I had.

I agree with @DED in their May 24, 2020 comment. I stumbled upon this while exploring the dumps. Their estimate of half the data being incorrect seems accurate. As such, I would suggest that this dump is worthless and should be eliminated. I can't imagine anybody making use of this in its current form. Producing it is a waste of machine resources, but worse than that, it's a time sink for people like me who are trying to understand the dumps.

Oh, never mind. PEBKAC. I had a ssh config file on my end which auto-mapped my local username (roy) to roysmith on the bastion. Everything is working now, thanks.

$ ssh -i $HOME/.ssh/id_rsa_wikimedia dev-buster.toolforge.org
roy@dev-buster.toolforge.org: Permission denied (publickey,hostbased).

For my personal need, either option would be fine. I can't speak for other users, but my gut feeling is nobody's going to miss 24. It really is ancient history.

Is the apostrophe being used on iOS the one from the keyboard?

Cool, thanks. I see the file systems mounted now. I'm told today was a WMF holiday, so extra thanks for taking care of this on your day off. Very much appreciated.

Could I get an update on this? Thanks.

Hmmm, ok, I see what's going on. I started typing 'meta" in the url bar, and that's what Chrome auto-completed. I didn't really stop to think about it, just hit return.

As a current example of a query which needs a dedicated compute resource, see this thread on enwiki Village Pump:

From an editor point of view, it's very much not a bad idea. We have broken cite templates all over the place because the automated tool incorrectly generates "cite web". The only way to fix them now is to delete the reference and create a new one manually. An automated process, even if it's "fundamentally lossy" would be a big help, because deleting the existing one and starting over is also "fundamentally lossy".

Just noting that there's a thread about this on enwiki. I agree with the sentiment there; this should be triaged as high priority.

I would be happy to ask my questions in a better place. Can you point me to where that place would be?

There's not much more info to provide. A user observed that mw-reverted tags were missing. There's a thread on enwiki's village pump, but there's not really anything there beyond what I mentioned above.

Yeah, I can repro that here. On the command line:

I'm still getting it. I can reproduce on any of:

Thanks!

User:Davidwr reports on the VP:T thread that this diff hits the limit, and I can reproduce that myself:

+1 to implementing this functionality.

Oh, my bad. I thought that was something developed by MWF.

I'm not getting how this is supposed to work. I've reconfigured my django app to log to stderr. That output is ending up in $HOME/uwsgi.log. When I run "kubectl logs", all I get is the error message mentioned above, and no additional output, no matter how long I wait.

Ah, thanks for the explanation. I just assumed it was kubectl failing to start up properly.

As I pointed out above (and opened T256482 for), kubectl logs doesn't run.

Any update on this? Any kind of workaround to get real-time logging?

Yup, I can confirm that "watch tail" is significantly faster than "tail -f".

No joy.

Yeah, my immediate goal is just to get logging feedback during development. I've never used kubectl logs, but sounds straightforward. I'll give it a try. Thanks.

Thanks to everybody for diving in and working on this. In the meantime, is there some workaround I could use? Is there some other host I could log into which has better NFS connectivity?

@Bstorm , I'm not sure if your question was addressed to me, but I don't have root on the bastions :-)

OK, I've finally found a solution. You can configure tmux to map C-p to the up-arrow escape sequence, which works around docker eating the C-p. It's hacky and duct-tapey, but it works. Still, fie on docker for not providing a way to configure/disable this behavior.

As an external observer, I'm fearful of "log everyone out". This will cause a spike in traffic to the authentication infrastructure as everybody logs back in again. Which, of course, was the root cause of this problem to begin with. Is there some way to do a rolling logout, killing say, 1% of the sessions per hour?

Oddly enough, I was able to edit logged out in an incognito window.

This is happening on enwiki as well. I'm also unable to logout! I tried Special:Logout, but it doesn't seem to do anything.

I originally got here when I opened {T230700: Special:RandomInCategory does not return all pages with equal probability} a while ago. I notice that my original example from T230700 was working through AfC work queues, and so is the example given by Ahecht, above. Makes me think that the AfC use case may be different from most other things happening on the wiki. I'm wondering if this could be solved by occasionally re-populating page_random? I could see a couple of possible variations on this.