Page MenuHomePhabricator

RLazarus (Reuven Lazarus)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Oct 15 2019, 4:02 PM (53 w, 1 d)
Availability
Available
IRC Nick
rlazarus
LDAP User
RLazarus
MediaWiki User
Unknown

Recent Activity

Yesterday

RLazarus added a subtask for T264821: Hourly read spikes against s8 resulting in occasional user-visible latency & error spikes: T266040: Large text objects are randomized to cache backends.
Tue, Oct 20, 4:22 PM · Wikidata, serviceops, Operations
RLazarus added a parent task for T266040: Large text objects are randomized to cache backends: T264821: Hourly read spikes against s8 resulting in occasional user-visible latency & error spikes.
Tue, Oct 20, 4:22 PM · Patch-For-Review, Operations, Traffic

Mon, Oct 19

RLazarus added a comment to T265288: Enable L3 routing on cloudsw nodes.

Note October 27 is also scheduled for the MediaWiki datacenter switchback -- please let's not have both events going at the same time. :) The switchback is scheduled to wrap up by 15:00 UTC, but it's possible we'll still be doing cleanup work for a while after, depending on how things go.

Mon, Oct 19, 4:01 PM · Operations, netops, cloud-services-team (Kanban)

Wed, Oct 7

RLazarus added a comment to T257118: Beta cluster has reached its quota.

cache-text, cache-upload, etcd, ircd, and memc are most certainly also still in use and we need at least one of each.

For memc we can most likely size down, I don't think we added this many intentionally but rather added new ones during rebuilds/upgrades and maybe left old ones?

cc @RLazarus Do you know if any of these were used for tests, and whether we could e.g. get a way with just one or two of the deployment-memc* instances? If so, where do we need to check for configs and update things etc. is the latest one the "right" one to keep?

Wed, Oct 7, 9:01 PM · Beta-Cluster-Infrastructure

Tue, Oct 6

RLazarus closed T244395: Investigate whether we can automatically share incident status docs with WMDE as Resolved.

This should be done! Thanks one last time for your patience.

Tue, Oct 6, 6:03 PM · Sustainability (Incident Followup), SRE-OnFire, Operations

Thu, Oct 1

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

October 27 confirmed, and I just filed T264364. Meet you over there. :)

Thu, Oct 1, 8:54 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations
RLazarus created T264364: CommRel support for FY2020-2021 Q2 DC switchback.
Thu, Oct 1, 8:54 PM · User-notice, Operations, CommRel-Specialists-Support (Oct-Dec-2020)

Tue, Sep 29

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Apologies, I've had a hectic few days. :) It turns out the engprod offsite is the week of Oct 26, so there already won't be a MW release train that week -- releng is understandably reluctant to skip the release two weeks in a row, which is a good argument against Oct 21. We don't need anyone from engprod online for the switchover itself, though, so we could run it during their offsite.

Tue, Sep 29, 4:15 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Mon, Sep 28

RLazarus updated the task description for T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group.
Mon, Sep 28, 6:03 PM · LDAP-Access-Requests, Operations
RLazarus reassigned T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group from KFrancis to ArielGlenn.

Thanks @KFrancis! Passing this along to @ArielGlenn as the current SRE Clinic Duty person.

Mon, Sep 28, 6:03 PM · LDAP-Access-Requests, Operations

Tue, Sep 22

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

October 21 looks good, tentatively. Let me confirm with folks.

Tue, Sep 22, 4:21 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Sep 17 2020

RLazarus added a comment to T263191: Allow Nicholas Skaggs to issue icinga commands.

I see you already put this on the agenda for the next SRE meeting on Monday, thanks. :) I expect it'll be uncontroversial, but if you don't need it urgently, we'll bring it up at the meeting and get it merged that afternoon.

Sep 17 2020, 10:17 PM · Operations, SRE-Access-Requests
RLazarus triaged T263191: Allow Nicholas Skaggs to issue icinga commands as Medium priority.
Sep 17 2020, 10:17 PM · Operations, SRE-Access-Requests
RLazarus closed T263105: Disabling the Daily-article-hy mailing list as Resolved.
rzl@lists1001:~$ sudo disable_list daily-article-hy
/var/lib/mailman/data/heldmsg-daily-article-hy-314.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-312.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-325.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-309.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-318.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-321.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-320.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-323.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-315.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-319.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-324.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-310.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-316.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-317.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-313.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-311.pck
/var/lib/mailman/data/heldmsg-daily-article-hy-322.pck
daily-article-hy disabled. Archives should be available at current location, all mail should be moderated and the list should not be on the listinfo page.
Sep 17 2020, 8:14 PM · Operations, Wikimedia-Mailing-lists
RLazarus updated subscribers of T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Thank you @Trizek-WMF! Really appreciate all your work on this, and your team's.

Sep 17 2020, 4:46 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations
RLazarus lowered the priority of T263132: Could not enqueue jobs from stream mediawiki.job.cirrusSearchIncomingLinkCount from Unbreak Now! to High.

Yeah, this does look like it was related to the eventgate-main deployments at 13:20 and 13:33 -- traffic may not have shifted gracefully from the old instances to the new ones, resulting in a transient spike of errors.

Sep 17 2020, 3:42 PM · Analytics-Kanban, Operations, Analytics, Event-Platform, Wikimedia-production-error

Sep 15 2020

RLazarus updated subscribers of T262773: Stop archiving the wikidata-bugs mailinglist in pipermail.

Any list administrator can disable archiving new messages, at https://lists.wikimedia.org/mailman/admin/wikidata-bugs/archive -- just change "Archive messages?" to "no" and submit.

Sep 15 2020, 11:04 PM · Wikidata, Operations, Wikimedia-Mailing-lists
RLazarus closed T262788: localisation-team mailing list to be archived and made read-only as Resolved.
rzl@lists1001:~$ sudo disable_list localisation-team
/var/lib/mailman/data/heldmsg-localisation-team-375.pck
localisation-team disabled. Archives should be available at current location, all mail should be moderated and the list should not be on the listinfo page.
Sep 15 2020, 10:08 PM · Wikimedia-Mailing-lists, Operations, Language-Team
RLazarus added a comment to T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group.

@conny-kawohl_WMDE Thanks!

Sep 15 2020, 3:55 PM · LDAP-Access-Requests, Operations
RLazarus added a comment to T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group.

And @bete one request (at the same time, to speed things along): Could you please ask your manager to comment on this Phab ticket, giving approval?

Sep 15 2020, 2:46 PM · LDAP-Access-Requests, Operations
RLazarus triaged T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group as Medium priority.
Sep 15 2020, 2:40 PM · LDAP-Access-Requests, Operations
RLazarus assigned T262921: Add Bereket teshome to the ldap/wmde and ldap/nda group to KFrancis.

Hello Bereket! I can handle the LDAP changes for you.

Sep 15 2020, 2:39 PM · LDAP-Access-Requests, Operations

Sep 14 2020

RLazarus closed T262785: LDAP access to the wmf group for Sudhanshu Gautam as Resolved.
rzl@mwmaint2001:~$ ldapsearch -x cn=wmf | grep sudhanshugautam
member: uid=sudhanshugautam,ou=people,dc=wikimedia,dc=org
Sep 14 2020, 7:37 PM · Operations, LDAP-Access-Requests
RLazarus added a comment to T262468: wikitech-static access for Sam Reed.

Added to the agenda for the next SRE meeting, 2020-09-21.

Sep 14 2020, 6:27 PM · cloud-services-team (Kanban), Operations, SRE-Access-Requests
RLazarus triaged T262468: wikitech-static access for Sam Reed as Medium priority.
Sep 14 2020, 6:26 PM · cloud-services-team (Kanban), Operations, SRE-Access-Requests

Sep 9 2020

RLazarus closed T262437: [Bug] Page content service is deployed with localhost links to the CSS and JS, breaking all pages that have been edited recently as Resolved.

I purged /api/rest_v1/page/mobile-html/ and /api/rest_v1/page/mobile-html-offline-resources/ in ATS, then re-ran Joe's command from T262437#6448216 for both. This should now be fully expunged from cache, although it may persist in your browser cache for up to a day or until you refresh.

Sep 9 2020, 9:29 PM · Operations, Traffic, serviceops, Wikimedia-production-error, Product-Infrastructure-Team-Backlog

Sep 1 2020

RLazarus created T261763: Unexplained increase in save times, possibly associated with DC switchover.
Sep 1 2020, 3:52 PM · Performance-Team, serviceops, Operations, Datacenter-Switchover

Aug 26 2020

RLazarus updated subscribers of T244808: CommRel support for FY2020-2021 Q1 DC switchover.

@Trizek-WMF Question from @debt earlier, will you be posting to wikitech-l also, or only wikitech-ambassadors? No wrong answer AFAIK, just confirming the plan.

Aug 26 2020, 3:22 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Aug 25 2020

RLazarus renamed Datacenter-Switchover from Datacenter-Switchover-2018 to Datacenter-Switchover.
Aug 25 2020, 9:36 PM
RLazarus added a member for Datacenter-Switchover: RLazarus.
Aug 25 2020, 9:35 PM
RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Done: https://wikitech.wikimedia.org/wiki/Switch_Datacenter#Schedule_for_2020_switch

Aug 25 2020, 5:35 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations
RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Yeah, sorry that's later than I expected -- we're meeting today to confirm the timing details and I'll post the update immediately after, so a little over two hours from now.

Aug 25 2020, 1:52 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Aug 24 2020

RLazarus added a comment to T261168: decommission mc2028.codfw.wmnet.

@Papaul Over to you, thanks!

Aug 24 2020, 10:47 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus reassigned T261168: decommission mc2028.codfw.wmnet from RLazarus to Papaul.
Aug 24 2020, 10:47 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus updated the task description for T261168: decommission mc2028.codfw.wmnet.
Aug 24 2020, 10:47 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus updated the task description for T261168: decommission mc2028.codfw.wmnet.
Aug 24 2020, 10:23 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus added a comment to T260224: mc2028 regular and mgmt interface down.

@Papaul Sorry for the delay, we only just got mc2037 up and serving. Opened T261168 for the decom, starting the service owner steps now.

Aug 24 2020, 9:49 PM · Patch-For-Review, Operations, ops-codfw
RLazarus added projects to T261168: decommission mc2028.codfw.wmnet: serviceops, Operations.
Aug 24 2020, 9:47 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus added a subtask for T260224: mc2028 regular and mgmt interface down: T261168: decommission mc2028.codfw.wmnet.
Aug 24 2020, 9:47 PM · Patch-For-Review, Operations, ops-codfw
RLazarus added a parent task for T261168: decommission mc2028.codfw.wmnet: T260224: mc2028 regular and mgmt interface down.
Aug 24 2020, 9:47 PM · ops-codfw, Operations, serviceops, decommission-hardware
RLazarus created T261168: decommission mc2028.codfw.wmnet.
Aug 24 2020, 9:46 PM · ops-codfw, Operations, serviceops, decommission-hardware

Aug 19 2020

RLazarus added a comment to T260694: Replace mc2028 with wmf6413 and name it mc2037.

I know it's 2020, but Jessie for uniformity please. Upgrading those hosts is a work in progress. If we weren't about to do the switchover, I'd say install something newer and see what happens, but let's do one thing at a time.

Aug 19 2020, 1:44 PM · Operations, ops-codfw

Aug 17 2020

RLazarus updated subscribers of T260224: mc2028 regular and mgmt interface down.

Yes please -- let's use the spare machine, add the memory from mc2028, and name the new host mc2037 (that is, let's use @elukey's option #1, but not reuse the name "mc2028", at @Volans's request).

Aug 17 2020, 4:00 PM · Patch-For-Review, Operations, ops-codfw

Aug 12 2020

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

I bet we can do 14:00 UTC. I'm finishing up the timeline with my SRE colleagues this week, I'll confirm and get back to you. After that we'll post on Wikitech, it'll be on the same page.

Aug 12 2020, 6:12 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations
RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

No, it'll be roughly a month -- there's a variety of maintenance we'd like to do in eqiad while we're serving from codfw.

Aug 12 2020, 3:03 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Aug 5 2020

RLazarus closed T259665: httpbb: Mapping between tests and hosts as Resolved.

The simple version of this is done. We might eventually want to do something more elaborate -- the advantage would be that httpbb could be run without explicitly passing a test file, and by default it could deduce (via Puppetry) the appropriate set of tests to run. But this is good enough for now.

Aug 5 2020, 8:20 PM · serviceops, Operations

Aug 4 2020

RLazarus triaged T259665: httpbb: Mapping between tests and hosts as Medium priority.
Aug 4 2020, 9:59 PM · serviceops, Operations

Jul 31 2020

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

The only user-impacting section of the process will be a read-only period for all wikis while we move MediaWiki itself -- that should last about 3-5 minutes, somewhere between 13:30 and 15:30 UTC on 2020-09-01.

Jul 31 2020, 3:28 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Jul 30 2020

RLazarus added a comment to T256983: Request increased quota for packaging Cloud VPS project.

Yep, we're all set. Please revert the RAM quota at your convenience, and if there are any Ceph resources to reclaim, feel free to do that too -- we won't be needing it for the foreseeable. Thanks!

Jul 30 2020, 2:50 PM · Cloud-VPS (Quota-requests)

Jul 29 2020

RLazarus updated subscribers of T256983: Request increased quota for packaging Cloud VPS project.

Thanks for the update; I was going to check back in here soon. We've actually rearranged this build a bit, and in the end we won't need the expanded disk after all, so we'll be settling back into a regular m1.xlarge instance.

Jul 29 2020, 11:20 PM · Cloud-VPS (Quota-requests)

Jul 28 2020

RLazarus closed T256291: Clean up the /*/mw/ mcrouter routing prefix as Invalid.
Jul 28 2020, 1:46 PM · serviceops, Operations

Jul 23 2020

Elitre awarded T244808: CommRel support for FY2020-2021 Q1 DC switchover a Like token.
Jul 23 2020, 9:08 AM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Jul 21 2020

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Thanks @Trizek-WMF! It took a moment to get everything else lined up, but we're moving forward with September 1.

Jul 21 2020, 8:36 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Jul 16 2020

RLazarus added a comment to T244395: Investigate whether we can automatically share incident status docs with WMDE.

Sorry for the delay, but this is still in progress -- I've checked in with Legal and they're still working on it. Thanks for your patience, still.

Jul 16 2020, 11:18 PM · Sustainability (Incident Followup), SRE-OnFire, Operations

Jul 2 2020

RLazarus added a parent task for T256983: Request increased quota for packaging Cloud VPS project: Unknown Object (Task).
Jul 2 2020, 3:18 PM · Cloud-VPS (Quota-requests)
RLazarus created T256983: Request increased quota for packaging Cloud VPS project.
Jul 2 2020, 3:16 PM · Cloud-VPS (Quota-requests)

Jun 24 2020

RLazarus triaged T256291: Clean up the /*/mw/ mcrouter routing prefix as Low priority.
Jun 24 2020, 5:49 PM · serviceops, Operations
RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

It looks like we'll try to do this: ideally we'll aim to do the switchover from eqiad to codfw in either mid-to-late August or early September, and the switch back to eqiad about a month later. (I can promise we won't do anything in July; we wouldn't ask you to work on that kind of short notice, and we won't have our act together yet anyway.)

Jun 24 2020, 3:59 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Jun 23 2020

RLazarus added a comment to T252391: Reimage one memcached shard to Buster.

Side note: This question is also interesting from a DC switchover perspective (T243316) since that will also effectively be a Redis flush. In previous switchovers we only explicitly handled replication for sessions data, and now that's out of Redis. If there's anything else in there that we can't afford to drop and recreate, now would be a great time to know that.

Jun 23 2020, 4:33 PM · User-jijiki, Growth-Team (Current Sprint), User-Elukey, Patch-For-Review, Operations, serviceops

Jun 22 2020

RLazarus added a comment to T255511: mcrouter memcached flapping in gutter pool.

Summarizing here a conversation @elukey and I had in #wikimedia-serviceops:

Jun 22 2020, 3:57 PM · serviceops

Jun 5 2020

RLazarus added a comment to T244808: CommRel support for FY2020-2021 Q1 DC switchover.

Thanks for checking -- not sure yet, but as we're planning out Q1 on our side too, I'm starting to take everyone's temperature about it. I'll let you know as soon as I have some idea whether it will happen, and I'll make sure to clear any potential dates with you.

Jun 5 2020, 2:19 PM · User-notice, CommRel-Specialists-Support (Jul-Sep-2020), Operations

Jun 2 2020

RLazarus created P11368 (An Untitled Masterwork).
Jun 2 2020, 11:31 PM

May 26 2020

RLazarus added a comment to T251349: Request for srv/phab/phabricator/bin/bulk make-silent --id * command via SSH for moving tasks quarterly.

Should I instead point to the private key I generated (per https://wikitech.wikimedia.org/wiki/Production_access#Generating_your_SSH_key)?

May 26 2020, 8:27 PM · Operations, SRE-Access-Requests
RLazarus reassigned T252705: Requesting access to sites from Google Search Console for soworu from RLazarus to soworu.

@soworu Ping on my question above. :)

May 26 2020, 2:17 PM · Operations, SRE-Access-Requests

May 22 2020

RLazarus added a project to T252826: Whitelist x-wikimedia-debug header field (currently not allowed by Access-Control-Allow-Headers in preflight response): Platform Engineering.

Hi CPT, adding you back -- is it possible this is due to this section of security_response_header_filter.js in Restbase?

May 22 2020, 6:23 PM · Wikidata, RESTBase-API, Wikidata-Query-Service, Platform Team Workboards (Clinic Duty Team), Developer Productivity
RLazarus triaged T253292: check_http and SNI support as Medium priority.
May 22 2020, 6:01 PM · Operations, Traffic

May 21 2020

RLazarus closed T253340: Create SDAW-Internal mailing list as Resolved.

All set!

May 21 2020, 9:42 PM · Wikimedia-Mailing-lists, Operations
RLazarus triaged T253197: Implement a prometheus exporter for rdkafka in golang as Medium priority.
May 21 2020, 6:09 PM · Operations, Traffic
RLazarus added a project to T252227: Mobile redirects drop provenance parameters: Traffic.
May 21 2020, 5:52 PM · Analytics-Radar, Readers-Web-Backlog (Tracking), Traffic, Operations
RLazarus assigned T183990: Publishing project anomaly data for censorship researchers. Evaluate privacy threats to ssingh.

Trying to route this -- @ssingh, should this be assigned to you?

May 21 2020, 5:51 PM · Analytics-Radar, Privacy Engineering, Operations, Traffic
RLazarus closed T249352: Onboarding Wolfgang Kandek as Resolved.
May 21 2020, 5:47 PM · LDAP-Access-Requests, SRE-Access-Requests, Operations
RLazarus added a comment to T251349: Request for srv/phab/phabricator/bin/bulk make-silent --id * command via SSH for moving tasks quarterly.

@MBinder_WMF On the offchance you'd already tried logging into phab1001, and it didn't work, but you hadn't gotten around to saying anything yet -- that was because of my mistake, just fixed it, so try again. :)

May 21 2020, 5:04 PM · Operations, SRE-Access-Requests

May 20 2020

RLazarus closed T250430: LDAP/NDA Access Request for mshaver as Resolved.
May 20 2020, 10:00 PM · Operations, LDAP-Access-Requests
RLazarus claimed T250430: LDAP/NDA Access Request for mshaver.

Thanks! I've removed mnoor but not mshaver from the wmf LDAP group, let me know if anything suddenly stops working.

May 20 2020, 8:46 PM · Operations, LDAP-Access-Requests
RLazarus added a comment to T250430: LDAP/NDA Access Request for mshaver.

Hi @MNoorWMF, just checking on this -- can you please confirm that the mshaver account is working for you? If it is, I'll remove access from mnoor so that we don't leave both lying around.

May 20 2020, 7:55 PM · Operations, LDAP-Access-Requests
RLazarus added a comment to T251349: Request for srv/phab/phabricator/bin/bulk make-silent --id * command via SSH for moving tasks quarterly.

Hi @MBinder_WMF! I've created your shell account and added it to the group that @Dzahn created for you. I just ran puppet on bast4002 (your nearest bastion) and both Phab hosts, so you should be able to test your access now by sshing to phab1001.eqiad.wmnet, after following the Setting_up_your_SSH_config steps in the page that Daniel linked.

May 20 2020, 7:48 PM · Operations, SRE-Access-Requests
RLazarus updated subscribers of T253155: Give access to the JupyterHub (SWAP) notebooks to (@Rvvalentim).

After chatting with @colewhite (thanks!) I've moved Rodolfo from the wmf group over to the nda group, which should still provide all the access necessary here. Let me know if you have any trouble.

May 20 2020, 6:08 PM · LDAP-Access-Requests, Operations, Jupyter-Hub
RLazarus added a comment to T253155: Give access to the JupyterHub (SWAP) notebooks to (@Rvvalentim).

This uses your LDAP password -- the same one you use to log into Wikitech. The username should be either your Wikitech username (Rodolfo Valentim) or your uid/"shell username" (rodolfovalentim) -- the login prompt should tell you which.

May 20 2020, 5:52 PM · LDAP-Access-Requests, Operations, Jupyter-Hub
RLazarus added a comment to T253086: Add Daniel Cipoletti to analytics-privatedata-users.

@dcipoletti Thanks, looks good! All I need you to do now is:

May 20 2020, 5:38 PM · Operations, LDAP-Access-Requests
RLazarus closed T253155: Give access to the JupyterHub (SWAP) notebooks to (@Rvvalentim), a subtask of T252476: Give access to the Analytics Cluster to Research Inter (Rodolfo), as Resolved.
May 20 2020, 5:18 PM · SRE-Access-Requests, Operations
RLazarus closed T253155: Give access to the JupyterHub (SWAP) notebooks to (@Rvvalentim) as Resolved.

Thanks Katie!

May 20 2020, 5:18 PM · LDAP-Access-Requests, Operations, Jupyter-Hub

May 19 2020

RLazarus triaged T253131: docker-pkg update cli renders unclear guidance as Medium priority.

Looks like this comes from docker_pkg/cli.py line 76, which is the last line here:

May 19 2020, 10:27 PM · serviceops, docker-pkg, Operations
RLazarus assigned T253155: Give access to the JupyterHub (SWAP) notebooks to (@Rvvalentim) to KFrancis.

I see @Nuria's authorization on the parent task, so going ahead with this.

May 19 2020, 8:56 PM · LDAP-Access-Requests, Operations, Jupyter-Hub
RLazarus triaged T253086: Add Daniel Cipoletti to analytics-privatedata-users as Medium priority.
May 19 2020, 8:47 PM · Operations, LDAP-Access-Requests
RLazarus assigned T253086: Add Daniel Cipoletti to analytics-privatedata-users to dcipoletti.

@dcipoletti Can you please ask your manager to comment on this task, approving your access?

May 19 2020, 8:46 PM · Operations, LDAP-Access-Requests
RLazarus triaged T252749: Reverse proxy supporting XFF-based per-IP concurrency limit and request queueing as Medium priority.

Agree we ought to do this, and I think it's something Envoy can do nearly out of the box. If there's an element Envoy doesn't support, I agree that it's the over-limit FIFO queue. (Out of all the reverse proxies we could use, I'm focusing on Envoy because we're actively deploying it in other parts of the infrastructure, and if it's feasible I think we'd like to use it here too for consistency.)

May 19 2020, 7:08 PM · serviceops, Sustainability (Incident Followup), Operations, Thumbor
RLazarus assigned T252773: Move kafkamon hosts to Debian Buster to herron.
May 19 2020, 5:38 PM · Patch-For-Review, Analytics-Clusters, Analytics-Radar, observability, Operations
RLazarus triaged T252890: scrape ripe atlas data for a few anchors at other large networks as Medium priority.
May 19 2020, 5:34 PM · netops, Operations
RLazarus triaged T252913: Switch ELK7 to use the distro Java as Medium priority.
May 19 2020, 5:31 PM · User-MoritzMuehlenhoff, observability, Operations
RLazarus triaged T253025: dbctl gives user-hostile diffs as Medium priority.
May 19 2020, 5:27 PM · conftool, Operations
RLazarus triaged T252956: Add Git LFS support for research/wikiworkshop as Medium priority.
May 19 2020, 5:26 PM · Operations, Research

May 18 2020

RLazarus closed T252674: Add `dcipoletti` to `wmf` Access Group as Resolved.

Hi Daniel, welcome to the Foundation! I've added you to the wmf LDAP group, feel free to reopen if you need anything else.

May 18 2020, 9:59 PM · LDAP-Access-Requests, Operations
RLazarus closed T252887: LDAP access to the wmf group for Carly Bogen as Resolved.

Hi Carly, I've added you to the wmf LDAP group -- feel free to reopen if you need anything else.

May 18 2020, 9:44 PM · Operations, LDAP-Access-Requests
RLazarus removed a project from T252987: Requesting access to wikimedia namespace in packagist: SRE-Access-Requests.

Thanks @bd808! It doesn't sound like there's presently anything for the SRE clinic-duty person to do here, so I'm optimistically removing SRE-Access-Requests, but feel free to add it back if you need anything from me (or subsequent clinicians).

May 18 2020, 9:25 PM · Patch-For-Review, Composer, Librarization
RLazarus triaged T252705: Requesting access to sites from Google Search Console for soworu as Medium priority.
May 18 2020, 7:58 PM · Operations, SRE-Access-Requests
RLazarus claimed T252705: Requesting access to sites from Google Search Console for soworu.

Hi Segun, thanks for the clear and complete request! I've granted you "restricted user" access for almost all those domains, which I think is the right level of access for your needs. If you need "full user," let me know and I'll upgrade you (or file another task, if you find out later).

May 18 2020, 7:57 PM · Operations, SRE-Access-Requests

May 14 2020

RLazarus created P11196 (An Untitled Masterwork).
May 14 2020, 5:37 PM

May 6 2020

RLazarus claimed T244340: Reduce read pressure on mc* servers by adding a machine-local Memcached instance (on-host memcached).
May 6 2020, 5:11 PM · User-jijiki, Sustainability (Incident Followup), Performance-Team, Patch-For-Review, Operations, serviceops

May 5 2020

RLazarus created P11152 (An Untitled Masterwork).
May 5 2020, 2:58 PM
RLazarus created P11151 (An Untitled Masterwork).
May 5 2020, 2:44 PM