Mon, Jul 19
Tue, Jul 13
Mon, Jul 12
Jun 22 2021
Jun 18 2021
Hi all. I have brought this up and are discussing this internally in SRE. I will update the ticket when I have more information. Thank you.
@Jgiannelos: Thanks for the additional confirmation; the SSH key has been updated. Marking this as resolved, please feel free to reopen if there are any issues.
Jun 17 2021
Jun 15 2021
Hi @ChristineDeKock: Can you please read through and sign the L3 (Acknowledgement of Wikimedia Server Access Responsibilities) document? Thank you!
Marc has been added to VictorOps and the SRE team; resolving this task. Thanks, all.
Jun 14 2021
@mepps: You have been added to the restricted group. Please let us know if there are any questions, thanks!
Jun 10 2021
Marking this as resolved as we have completed all the intended tasks for now and the routers have been configured.
Additional confirmation, since I am enjoying the reduced latency of the new Toronto -> eqiad route instead of the old Toronto -> codfw :)
INFO:homer.transports.junos:Committing the configuration on cr2-eqiad.wikimedia.org
INFO:homer:Homer run completed successfully on 2 devices: ['cr1-eqiad.wikimedia.org', 'cr2-eqiad.wikimedia.org']
doh1001 and doh1002 have been created; closing this task. Thanks for the help!
INFO:homer.transports.junos:Committing the configuration on cr4-ulsfo.wikimedia.org
INFO:homer:Homer run completed successfully on 2 devices: ['cr3-ulsfo.wikimedia.org', 'cr4-ulsfo.wikimedia.org']
Jun 8 2021
Jun 7 2021
Jun 4 2021
doh5001 is also up; from Mumbai, we are reaching eqsin as desired:
Jun 3 2021
Thanks for all the help and sorry it took a while! I think 10G should be fine for now given the current usage on the other Wikidough hosts.
Jun 2 2021
Puppet roles/profiles/hiera configurations have not been removed as they will be needed for the cescout VM that will be provisioned later.
May 27 2021
May 26 2021
May 24 2021
May 21 2021
May 20 2021
May 19 2021
May 18 2021
May 17 2021
May 11 2021
May 10 2021
@ayounsi, @Aklapper: I forgot to update this ticket, apologies! Last I checked, all reports were in the green and there wasn't any debugging information. But since we have had no user updates and it's been a while since I looked, I will check again and update this ticket accordingly. Thanks!
Apr 30 2021
@MMandere: Added to "ops" LDAP group. https://wikitech.wikimedia.org/wiki/LDAP/Groups#ops_group
Apr 27 2021
Apr 26 2021
Mar 2 2021
Hi @JAllemandou: thanks, I think proceeding through Wikimedia Taiwan as @Shizhao mentioned is probably a good idea. In the meantime, I will ask for a contact in the community and also if the partnerships team has some idea (we have asked for their help in the recent past with a similar issue).
Feb 24 2021
Feb 22 2021
We decided to file https://github.com/citizenlab/test-lists/pull/730 so that we can get some test data.
Feb 21 2021
Feb 16 2021
I edited the task but I wanted to add here as well that this happens when I run the Hive query from Spark.
Thanks for opening this task, Marcel.
Feb 10 2021
Feb 9 2021
Thank you for the interest in this task! Like we shared yesterday, we have identified that the traffic is coming from a popular mobile app in India. We have initiated contact with the app developers, and are waiting to hear back from them. In the meantime, given the volume of requests, we have decided to ban those specific requests until the issue is resolved. While we will refrain from naming the app at this time, we can share that it is not on the list of apps mentioned in this task. Nevertheless, we thank you for your comments and suggestions on how to debug this!
Thank you everyone for the comments and suggestions. I just wanted to share that we have identified the app and will update this task tomorrow. (And yes, it is a mobile app.)