Sorry, I was (blindly) following the instructions on wikitech and didn't stop to think. I'll take care of this myself!

Thanks for the clarification. That sounds good, @MoritzMuehlenhoff. I have no objections to implementing this.

@MoritzMuehlenhoff I did some digging into what implementing this would entail. There are a couple of concerns, I think:

@ECohen_WMDE, how do you want to do the public key confirmation? One of the most common methods is to put your pubkey on your Mediawiki page, usually in the Contact section. You could also submit a Gerrit patchset. Feel free to reach out to me here (preferred) or on Slack (Chris Dobbins) or IRC (ChrisDobbins901_) if you have any questions.

From what I can tell by checking WMF-NDA group membership and the NDA spreadsheet, an NDA hasn't yet been signed. But @KFrancis would know better than I.

@Johannes_Richter_WMDE you should be set up now. If there's something I missed, please reply and I'll fix it ASAP!

@KFrancis: could you add @Johannes_Richter_WMDE to the NDA spreadsheet? I wasn't able to find that username, so if I'm mistaken, my apologies. Thank you!

@KFrancis: could you add @mahmoud.abdelsattar.wmde to the NDA spreadsheet? I wasn't able to find that username, so if I'm mistaken, my apologies. Thank you!

@mahmoud.abdelsattar.wmde you should be all set up. If not, please let me know and I'll fix it ASAP!

@ssingh: that's right. I thought about this a bit over the weekend, and I think the easiest approach is going to be doing a clean install in a VM, grabbing the yaml config, and diffing it against our existing settings. If this seems reasonable, I was thinking we could start with Wikimedia DNS to validate the approach. Any thoughts?

Edit: never mind

@Milimetric @Ahoelzl @Ottomata could any of you confirm that access should be granted access to the logstash group? Thanks!

@HCoplin-WMF would you mind pasting your pubkey on your metawiki page? If that isn't an option, feel free to reach out to me for other options to verify your SSH key.

@Gerges do you still need access to Logstash? If so, is there anything we can do to help with getting the NDA and other steps done?

We have a pipeline (currently a Jupyter Notebook) that ranks each DC for a given country/region by median latency. This is the result for China:

|continent|country|code|dc   |probenet_mean|probenet_median|probenet_std_dev|probenet_variance|probenet_sample_size|
+---------+-------+----+-----+-------------+---------------+----------------+-----------------+--------------------+
|Asia     |China  |CN  |ulsfo|228.86       |215.5          |45.76           |2093.98          |14                  |
|Asia     |China  |CN  |codfw|284.64       |228.0          |158.91          |25250.85         |11                  |
|Asia     |China  |CN  |esams|230.94       |249.0          |64.11           |4109.56          |17                  |
|Asia     |China  |CN  |eqiad|360.26       |266.0          |353.39          |124883.32        |19                  |
|Asia     |China  |CN  |drmrs|275.45       |272.0          |41.33           |1708.07          |11                  |
|Asia     |China  |CN  |eqsin|742.83       |334.0          |1067.6          |1139776.33       |12                  |
|Asia     |China  |CN  |magru|355.17       |350.5          |30.9            |954.74           |18                  |

Time-wise, this covers the maximum window (90 days). The notebook was run today.

Thanks for looking into this. I don't know what happened, but apparently it resolved itself. Sorry about the delay in responding.

On 4/2, we discussed the merits and pitfalls of the proposed implementation with @Volans and @Papaul (notes here). It was decided to run the check when the host starts up, as they suggested.

To provide an update, this is still happening on ldap-replica1003 https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&var-site=eqiad&var-cluster=misc&from=now-2d&to=now&viewPanel=4. Would it be helpful if I looked into it further?

Just wanted to check in, as it's been a while since the last comment, and I wanted to see if there’s been any progress or if there’s anything further I can provide to assist in resolving the issue.

Hi @elukey, thanks for responding. The impact isn't related to the metric, per se. The impact is that the server is refusing connections from real clients, not just locally initiated ones. The script behind this metric creates a local connection in order to get the MSS value; when it fails, it's because it cannot create a connection. When that happens, it's always preceded by a surge in slapd connections.

In T376267#10518842, @Reedy wrote:

In T376267#10518535, @CDobbins wrote:

Wikitech account/LDAP:	cdobbins
SUL account	CDobbins-WMF
Account linked on IDM	Y
I have visited MediaWiki:Loginprompt	Y
I have tried to reset my password using Special:PasswordReset	Y

Would you like your wikitech account renaming to CDobbins-WMF?

The above change has a cookbook that's used for both haproxy and pdns-recursor.

In T372473#10127905, @Yann wrote:

https://commons.wikimedia.org/w/index.php?title=File:The_Three_Musketeers_(1921).webm&action=revert

Request from 80.82.77.7 via cp3068 cp3068, Varnish XID 402366146
Error: 503, Backend fetch failed at Sat, 07 Sep 2024 16:26:00 GMT

Closing this now, since a fix was deployed last week and there haven't been any more reports of this happening. Please reopen if this occurs again.

Closing this now, since a fix was deployed last week and there haven't been any more reports of this happening. Please reopen if this occurs again.

A patch to fix this was merged 30 minutes ago, though it hasn't yet been deployed. It's now deployed everywhere.

I'm looking into this now. Sorry that this wasn't resolved by the earlier fix.

The fix for this was deployed this morning. If this error reappears, please reopen this task or create a new one.

This has been deployed as of 14:25 on 7/23/24, with CR #1041705.

• eqsin
• drmrs
• ulsfo
• eqiad
• codfw
• esams
• magru
• ulsfo

Description of changes:

I forgot to mention this, but the account that I want to be active is cdobbins