Page MenuHomePhabricator

jijiki (effie mouzeli)
is an animal

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Aug 14 2018, 10:50 AM (57 w, 4 d)
Availability
Available
IRC Nick
jijiki
LDAP User
Effie Mouzeli
MediaWiki User
EMouzeli (WMF) [ Global Accounts ]

Recent Activity

Fri, Sep 20

Jdforrester-WMF awarded T229792: Remove HHVM from production a Yellow Medal token.
Fri, Sep 20, 10:40 PM · Performance-Team (Radar), Patch-For-Review, serviceops, Operations, HHVM

Thu, Sep 19

jijiki closed T218323: reallocate former image scaler to thumbor use as Resolved.

We are planning to move Thumbor to k8s, T233196, thus I am closing this task

Thu, Sep 19, 2:26 PM · Operations, Thumbor, hardware-requests
jijiki added a comment to T232412: HTTPS-blog on blog.wikimedia.org is CRITICAL: SSL CRITICAL - failed to connect or SSL handshake:SSL connect attempt failed.

@CDanis @ayounsi @Vgutierrez Hello all, happy Monday! WordPress is still investigating what's happening, and they're wondering if we have a more detailed error message that we could share with them. I could also try to cc you on the WordPress ticket, if that would be more helpful. Thanks for your time—y'all are fantastic!

Thu, Sep 19, 2:23 PM · Wikimedia-Blog

Wed, Sep 18

jijiki added a subtask for T231433: Move cache upload cluster from nginx to ats-tls: T233205: Higher failed fetches error rate on some caching servers .
Wed, Sep 18, 11:01 AM · Patch-For-Review, Traffic, Operations
jijiki added a parent task for T233205: Higher failed fetches error rate on some caching servers : T231433: Move cache upload cluster from nginx to ats-tls.
Wed, Sep 18, 11:01 AM · Operations, Traffic
jijiki created T233205: Higher failed fetches error rate on some caching servers .
Wed, Sep 18, 11:01 AM · Operations, Traffic
jijiki added a comment to T192457: Reallocate former image scalers.

@Dzahn We will be moving Thumbor to k8s T233196, we can repurpose the spare server for something else:)

Wed, Sep 18, 9:35 AM · Patch-For-Review, serviceops, Operations
jijiki triaged T233196: Migrate thumbor to Kubernetes as Normal priority.
Wed, Sep 18, 9:34 AM · Operations, serviceops, Thumbor
jijiki created T233196: Migrate thumbor to Kubernetes .
Wed, Sep 18, 9:34 AM · Operations, serviceops, Thumbor
jijiki moved T226551: Package table_properties utility for Debian from Backlog to Next up on the serviceops board.
Wed, Sep 18, 8:06 AM · CPT Initiatives (Session Management Service (CDP2)), serviceops, User-Clarakosi
jijiki added a subtask for T220600: Remove PHP 7.0 from production application servers: T230024: Update component/php72 to 7.2.22.
Wed, Sep 18, 8:05 AM · serviceops, Operations
jijiki added a parent task for T230024: Update component/php72 to 7.2.22: T220600: Remove PHP 7.0 from production application servers.
Wed, Sep 18, 8:05 AM · serviceops, Operations
jijiki updated subscribers of T230024: Update component/php72 to 7.2.22.

@thcipriani is it ok if we update php7 to 7.2.22 on deploy* servers? Do you know if there are any dependencies ?

Wed, Sep 18, 8:04 AM · serviceops, Operations
jijiki updated the task description for T230024: Update component/php72 to 7.2.22.
Wed, Sep 18, 8:03 AM · serviceops, Operations
jijiki added a comment to T229980: Need help to create and deploy Debian-packaged Python 3 app.

@holger.knust What is the status of this? Just checking if I can help

Wed, Sep 18, 7:50 AM · serviceops, Operations, Packaging, CPT Initiatives (Session Management Service (CDP2))

Tue, Sep 17

jijiki added a comment to T232767: Netbox API Occasionally 500s and Netbox2001 dumpcsv fails.

@crusnov I have downtimed 'Check systemd state' checks until Sept 24 because it was polluting the alerts

Tue, Sep 17, 5:42 AM · SRE-tools
jijiki added a comment to T232412: HTTPS-blog on blog.wikimedia.org is CRITICAL: SSL CRITICAL - failed to connect or SSL handshake:SSL connect attempt failed.

I am downtiming the check for a couple of days because it is polluting our alerts

Tue, Sep 17, 4:54 AM · Wikimedia-Blog
jijiki added a comment to T215411: (OoW) thumbor1004 memory errors.

I have disabled this check for now

Tue, Sep 17, 4:16 AM · User-jijiki, Thumbor, ops-eqiad, serviceops, Operations

Fri, Sep 13

jijiki closed T195392: Switch cronjobs on maintenance hosts to PHP7, a subtask of T176370: Migrate to PHP 7 in WMF production, as Resolved.
Fri, Sep 13, 5:44 AM · MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), CPT Initiatives (PHP7 (TEC4)), Patch-For-Review, TechCom-RFC (TechCom-Approved), User-ArielGlenn, HHVM, Operations
jijiki closed T195392: Switch cronjobs on maintenance hosts to PHP7 as Resolved.
Fri, Sep 13, 5:44 AM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki closed T195392: Switch cronjobs on maintenance hosts to PHP7, a subtask of T219127: SRE FY19-20 Q1 goal: complete the transition to PHP7, as Resolved.
Fri, Sep 13, 5:44 AM · Operations, serviceops
jijiki updated the task description for T195392: Switch cronjobs on maintenance hosts to PHP7.
Fri, Sep 13, 5:43 AM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki moved T220600: Remove PHP 7.0 from production application servers from Backlog to Next up on the serviceops board.
Fri, Sep 13, 4:43 AM · serviceops, Operations
jijiki moved T195364: Remove pear/mail packages from WMF MW app servers from Backlog to Next up on the serviceops board.
Fri, Sep 13, 4:43 AM · serviceops, Patch-For-Review, Wikimedia-General-or-Unknown, Operations
jijiki closed T227592: Confd died on bast3002 as Resolved.

It has not happened again, Resolving for now.

Fri, Sep 13, 4:42 AM · Operations, serviceops
jijiki moved T230024: Update component/php72 to 7.2.22 from Backlog to Next up on the serviceops board.
Fri, Sep 13, 4:41 AM · serviceops, Operations
jijiki moved T229980: Need help to create and deploy Debian-packaged Python 3 app from Backlog to Next up on the serviceops board.
Fri, Sep 13, 4:36 AM · serviceops, Operations, Packaging, CPT Initiatives (Session Management Service (CDP2))
jijiki moved T231011: Mysterious, coordinated slowdowns every ~ 25 minutes on mw1347,mw1348 (php7 api servers) from Backlog to Next up on the serviceops board.
Fri, Sep 13, 4:34 AM · PHP 7.2 support, serviceops, Operations
jijiki moved T195392: Switch cronjobs on maintenance hosts to PHP7 from Next up to Doing on the serviceops board.
Fri, Sep 13, 4:30 AM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki moved T229286: "worker died, restarting" mobileapps issue from Backlog to Externally Blocked on the serviceops board.
Fri, Sep 13, 3:27 AM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops

Thu, Sep 12

jijiki renamed T229433: ResourceLoaderWikiModule.php: PHP Notice: Undefined index: from Investigate magically missing array key in ResourceLoaderWikiModule to ResourceLoaderWikiModule.php: PHP Notice: Undefined index: .
Thu, Sep 12, 2:26 PM · Performance-Team, Wikimedia-production-error, MediaWiki-ResourceLoader
jijiki renamed T232613: LBFactoryMulti.php: PHP Notice: Undefined index: from LBFactoryMulti.php PHP Notice: Undefined index: to LBFactoryMulti.php: PHP Notice: Undefined index: .
Thu, Sep 12, 2:26 PM · Patch-For-Review, MW-1.34-notes (1.34.0-wmf.22; 2019-09-10), Core Platform Team Workboards (Clinic Duty Team), Wikimedia-Rdbms, PHP 7.2 support, Wikimedia-production-error

Wed, Sep 11

jijiki claimed T180696: Terminate Thumbor with SSL.
Wed, Sep 11, 2:53 PM · User-jijiki, serviceops, Performance-Team (Radar), Thumbor
jijiki changed the status of Unknown Object (Task), a subtask of T215411: (OoW) thumbor1004 memory errors, from Open to Stalled.
Wed, Sep 11, 2:07 PM · User-jijiki, Thumbor, ops-eqiad, serviceops, Operations

Aug 22 2019

jijiki updated the task description for T231011: Mysterious, coordinated slowdowns every ~ 25 minutes on mw1347,mw1348 (php7 api servers).
Aug 22 2019, 2:23 PM · PHP 7.2 support, serviceops, Operations
jijiki added a comment to T224857: Enhance MediaWiki deployments for support of php7.x.

@thcipriani When a deployer run scap pull on mwdebug, they got sudo: [/usr/local/sbin/check-and-restart-php,: command not found (notice the extra comma), so we rolledback https://gerrit.wikimedia.org/r/531475

Aug 22 2019, 1:47 PM · Release-Engineering-Team-TODO (201909), Release-Engineering-Team (Deployment services), Patch-For-Review, User-jijiki, PHP 7.2 support, Scap, serviceops
jijiki updated the task description for T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.
Aug 22 2019, 1:45 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki added a comment to T230861: PHP 7.2 is very slow on an allocation-intensive benchmark.

We are on our way to finishing migration to PHP7, my opinion is to try the PHP7.2 bandaid rather than upgrading production to PHP7.3 right now, if we all agree of course.

Aug 22 2019, 11:27 AM · PHP 7.3 support, PHP 7.2 support, serviceops, Operations
jijiki updated the task description for T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.
Aug 22 2019, 11:19 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops

Aug 13 2019

jijiki added a comment to T229286: "worker died, restarting" mobileapps issue.

@Mholloway We are getting a number of alerts where scb* servers return 504 to healthchecks, what do you suggest?

Aug 13 2019, 2:19 PM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops
jijiki updated subscribers of T230240: purgeParserCache too slow to run once a day.
Aug 13 2019, 8:40 AM · MediaWiki-Maintenance-scripts
jijiki moved T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters from Next up to Doing on the serviceops board.
Aug 13 2019, 8:32 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki moved T195392: Switch cronjobs on maintenance hosts to PHP7 from Doing to Next up on the serviceops board.
Aug 13 2019, 8:32 AM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations

Aug 12 2019

jijiki updated the task description for T195392: Switch cronjobs on maintenance hosts to PHP7.
Aug 12 2019, 3:26 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki updated subscribers of T195392: Switch cronjobs on maintenance hosts to PHP7.

@Dzahn Thank you! We'll see how things are since we now have merged https://gerrit.wikimedia.org/r/425027, and continue with moving any transitional changes we did.

Aug 12 2019, 3:26 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki updated the task description for T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.
Aug 12 2019, 3:18 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki updated the task description for T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.
Aug 12 2019, 3:18 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki closed T224538: Socket Errors on PHP7 as Resolved.

Fixed!

Aug 12 2019, 1:41 PM · Patch-For-Review, serviceops, Operations, PHP 7.2 support

Aug 9 2019

jijiki added a project to T195392: Switch cronjobs on maintenance hosts to PHP7: MediaWiki-extensions-Mailgun.
Aug 9 2019, 5:27 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki awarded T195392: Switch cronjobs on maintenance hosts to PHP7 a Love token.
Aug 9 2019, 3:17 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki added a comment to T195392: Switch cronjobs on maintenance hosts to PHP7.

@Dzahn I think we should continue by switching all jobs on Monday and see what gives. We followed the same strategy with async jobs, i.e. we migrated a few jobs that stood out along with a few high traffic ones, and eventually we switched what was left with one go. What do you think? Thank you very much for the help!

Aug 9 2019, 3:17 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki triaged T230206: Remove nginx submodule from puppet as Normal priority.
Aug 9 2019, 2:44 PM · User-jijiki, Operations, puppet-compiler
jijiki created T230206: Remove nginx submodule from puppet.
Aug 9 2019, 2:44 PM · User-jijiki, Operations, puppet-compiler

Aug 8 2019

jijiki updated the task description for T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.
Aug 8 2019, 11:22 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops

Aug 7 2019

jijiki edited P8883 20190807-s8-cawiki-errors.
Aug 7 2019, 8:41 PM
jijiki created P8883 20190807-s8-cawiki-errors.
Aug 7 2019, 8:40 PM
jijiki updated the task description for T195392: Switch cronjobs on maintenance hosts to PHP7.
Aug 7 2019, 4:59 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki updated the task description for T195392: Switch cronjobs on maintenance hosts to PHP7.
Aug 7 2019, 3:17 PM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations

Aug 6 2019

jijiki moved T195392: Switch cronjobs on maintenance hosts to PHP7 from Next up to Doing on the serviceops board.
Aug 6 2019, 9:56 AM · MediaWiki-extensions-Mailgun, User-jijiki, cloud-services-team, Services (watching), serviceops, Core Platform Team Legacy (Watching / External), User-ArielGlenn, Operations
jijiki moved T151304: tmpreaper possible race condition from Next up to Externally Blocked on the serviceops board.
Aug 6 2019, 9:54 AM · serviceops, Operations
jijiki closed T228643: Don't monitor HHVM on PHP7 only servers, a subtask of T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters, as Invalid.
Aug 6 2019, 9:53 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki closed T228643: Don't monitor HHVM on PHP7 only servers as Invalid.

@Dzahn you are right, I am marking this as invalid.

Aug 6 2019, 9:53 AM · Operations, serviceops, PHP 7.2 support
jijiki added a comment to T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters.

We switched mw1270 to PHP7 but we came across the following issues

Aug 6 2019, 8:56 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki closed T227548: SSH to mw2269.mgmt not working as Resolved.

@Papaul closing, sorry this slipped through the cracks.

Aug 6 2019, 7:27 AM · Operations, ops-codfw

Aug 5 2019

jijiki closed T227408: (OoW) restbase2009 lockup as Resolved.

@Papaul I am marking this as resolved, thank you!

Aug 5 2019, 5:34 PM · serviceops, ops-codfw, Operations
jijiki closed T219148: Use PHP7 to run all async jobs, a subtask of T219127: SRE FY19-20 Q1 goal: complete the transition to PHP7, as Resolved.
Aug 5 2019, 2:29 PM · Operations, serviceops
jijiki closed T219148: Use PHP7 to run all async jobs as Resolved.
Aug 5 2019, 2:29 PM · User-WDoran, Core Platform Team Workboards (Clinic Duty Team), Patch-For-Review, User-jijiki, Services (watching), Operations, serviceops
jijiki added a comment to T219148: Use PHP7 to run all async jobs.

Removing HHVM and any leftovers are now part of T229792, we mark this as resolved 💃

Aug 5 2019, 2:29 PM · User-WDoran, Core Platform Team Workboards (Clinic Duty Team), Patch-For-Review, User-jijiki, Services (watching), Operations, serviceops
jijiki added a comment to T224538: Socket Errors on PHP7 .

For connection pooling purposes, when we want to access search.svc.eqiad.wmnet from php-fpm, we are doing so via nginx. This nginx is installed on each mw* server listening on *:80. Mediawiki is configured to do so via https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/ProductionServices.php#L147, and trying to resolve localhost. For every request, localhost first resolves to ::1 where nginx is not listening to, and then continues to 127.0.0.1, which is successful. The first failure increases the tcp attemptfails counter.

Aug 5 2019, 2:19 PM · Patch-For-Review, serviceops, Operations, PHP 7.2 support
jijiki added a comment to T227408: (OoW) restbase2009 lockup.

@Papaul Server is depooled, ping me when do pool it back, many thanks !

Aug 5 2019, 2:10 PM · serviceops, ops-codfw, Operations
jijiki added a comment to T151304: tmpreaper possible race condition.

@elukey @MoritzMuehlenhoff I have added tmpreaper's removal as an actionable in our HHVM removal process (T229792), shall we mark this as resolved or invalid?

Aug 5 2019, 10:02 AM · serviceops, Operations
jijiki updated the task description for T229792: Remove HHVM from production.
Aug 5 2019, 10:01 AM · Performance-Team (Radar), Patch-For-Review, serviceops, Operations, HHVM
jijiki added a subtask for T219127: SRE FY19-20 Q1 goal: complete the transition to PHP7: T229792: Remove HHVM from production.
Aug 5 2019, 9:56 AM · Operations, serviceops
jijiki added a parent task for T229792: Remove HHVM from production: T219127: SRE FY19-20 Q1 goal: complete the transition to PHP7.
Aug 5 2019, 9:56 AM · Performance-Team (Radar), Patch-For-Review, serviceops, Operations, HHVM
jijiki created T229792: Remove HHVM from production.
Aug 5 2019, 9:55 AM · Performance-Team (Radar), Patch-For-Review, serviceops, Operations, HHVM

Jul 31 2019

Mholloway awarded T229286: "worker died, restarting" mobileapps issue a The World Burns token.
Jul 31 2019, 6:38 PM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops

Jul 30 2019

jijiki updated the task description for T224538: Socket Errors on PHP7 .
Jul 30 2019, 3:52 PM · Patch-For-Review, serviceops, Operations, PHP 7.2 support
jijiki added a project to T229286: "worker died, restarting" mobileapps issue: Mobile-Content-Service.
Jul 30 2019, 3:51 PM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops
jijiki moved T151304: tmpreaper possible race condition from Backlog to Next up on the serviceops board.
Jul 30 2019, 9:52 AM · serviceops, Operations
jijiki added a project to T151304: tmpreaper possible race condition: serviceops.
Jul 30 2019, 9:52 AM · serviceops, Operations
jijiki added a comment to T151304: tmpreaper possible race condition.

@elukey I am up for attempting to patch it and upload to stretch-wikimedia, I will try to do it next week

Jul 30 2019, 9:51 AM · serviceops, Operations

Jul 29 2019

jijiki triaged T229286: "worker died, restarting" mobileapps issue as Normal priority.
Jul 29 2019, 10:12 PM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops
jijiki created T229286: "worker died, restarting" mobileapps issue.
Jul 29 2019, 10:11 PM · Patch-For-Review, Page Content Service, Product-Infrastructure-Team-Backlog (Kanban), Mobile-Content-Service, serviceops
jijiki added a comment to T228976: Allow to avoid installing HHVM from the mediawiki puppet module and profile.

Before making changes to profile::mediawiki::common and since we are in the process of removing hhvm from jobrunners/videoscalers (T219148), I think we should also consider the following:

Jul 29 2019, 12:40 PM · CPT Initiatives (Parsoid REST API in PHP (CDP2)), Patch-For-Review, serviceops, Parsoid-PHP
jijiki added a comment to T206339: Separate Traffic layer caches for PHP7/HHVM.

Just a heads up, we are planning to start migrating API servers to serve only via PHP7. For the time being, we have one in each DC.

Jul 29 2019, 9:30 AM · Traffic, Operations

Jul 26 2019

jijiki added a comment to T227408: (OoW) restbase2009 lockup.

@Papaul Can you let us know what are our options (if any?)

Jul 26 2019, 12:34 PM · serviceops, ops-codfw, Operations
jijiki updated the task description for T219127: SRE FY19-20 Q1 goal: complete the transition to PHP7.
Jul 26 2019, 10:28 AM · Operations, serviceops
jijiki moved T219150: Ramp up percentage of users on php7.2 to 100% on both API and appserver clusters from Backlog to Next up on the serviceops board.
Jul 26 2019, 10:27 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, Operations, serviceops
jijiki moved T180696: Terminate Thumbor with SSL from Backlog to Next up on the serviceops board.
Jul 26 2019, 10:27 AM · User-jijiki, serviceops, Performance-Team (Radar), Thumbor
jijiki moved T228643: Don't monitor HHVM on PHP7 only servers from Backlog to Next up on the serviceops board.
Jul 26 2019, 10:27 AM · Operations, serviceops, PHP 7.2 support
jijiki triaged T229070: phased rollout of dbctl, etcd-backed database configuration in Mediawiki as Normal priority.
Jul 26 2019, 10:26 AM · Operations, serviceops-radar, Performance-Team (Radar), DBA
jijiki moved T91474: Custom 404 error page should apply to *.php from Next up to Doing on the serviceops board.
Jul 26 2019, 10:24 AM · serviceops, User-jijiki, Regression, Wikimedia-General-or-Unknown

Jul 25 2019

jijiki moved T219148: Use PHP7 to run all async jobs from Doing to Next up on the serviceops board.
Jul 25 2019, 1:46 PM · User-WDoran, Core Platform Team Workboards (Clinic Duty Team), Patch-For-Review, User-jijiki, Services (watching), Operations, serviceops
jijiki added a comment to T219148: Use PHP7 to run all async jobs.

All async jobs run on PHP7, we will keep an eye for about a week, and then cleanup code leftovers

Jul 25 2019, 9:32 AM · User-WDoran, Core Platform Team Workboards (Clinic Duty Team), Patch-For-Review, User-jijiki, Services (watching), Operations, serviceops
jijiki updated the task description for T219148: Use PHP7 to run all async jobs.
Jul 25 2019, 9:29 AM · User-WDoran, Core Platform Team Workboards (Clinic Duty Team), Patch-For-Review, User-jijiki, Services (watching), Operations, serviceops
jijiki added a comment to T227408: (OoW) restbase2009 lockup.

@Eevans I was under the impression we have more work to be done on the server. Shall we mark this task as resolved?

Jul 25 2019, 9:24 AM · serviceops, ops-codfw, Operations

Jul 24 2019

jijiki added a comment to T228703: Keys from MediaWiki Redis Instances.

@holger.knust I accidentally copied the wrong dump to your directory yesterday, I uploaded a new dump today. Sorry for the confusion.

Jul 24 2019, 3:09 PM · serviceops, Core Platform Team Workboards (Green), Operations

Jul 23 2019

jijiki closed T228703: Keys from MediaWiki Redis Instances, a subtask of T228295: Multi-DC MainStash, as Resolved.
Jul 23 2019, 4:01 PM · CPT Initiatives (Mainstash Multi-DC)
jijiki closed T228703: Keys from MediaWiki Redis Instances as Resolved.

@holger.knust I copied a gzipped dump to a server you have access to, please reopen when you need newer one:)

Jul 23 2019, 4:01 PM · serviceops, Core Platform Team Workboards (Green), Operations
jijiki moved T228703: Keys from MediaWiki Redis Instances from Next up to Doing on the serviceops board.
Jul 23 2019, 1:34 PM · serviceops, Core Platform Team Workboards (Green), Operations