Page MenuHomePhabricator

Stashbot (Stashbot)
Bot that likes to stash things away for laterBot

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Feb 10 2016, 12:29 AM (162 w, 5 d)
Roles
Bot
Availability
Available
IRC Nick
stashbot
LDAP User
Stashbot
MediaWiki User
Unknown

Recent Activity

Today

Stashbot added a comment to T218022: decommission: labtestservices2001.wikimedia.org.

Mentioned in SAL (#wikimedia-operations) [2019-03-25T07:58:21Z] <vgutierrez> disable puppet and downtime host in icinga for labtestservices2001 - T218022

Mon, Mar 25, 7:58 AM · Operations, decommission, ops-codfw, cloud-services-team (Kanban)
Stashbot added a comment to T217152: Monitor and scale in the Trusty grid.

Mentioned in SAL (#wikimedia-cloud) [2019-03-25T02:34:28Z] <bd808> Disassociated floating IPs and deleted shutdown Trusty grid nodes tools-exec-14{33,34,35,36,37,38,39,40,41,42} (T217152)

Mon, Mar 25, 2:34 AM · Cloud-VPS (Ubuntu Trusty Deprecation), Toolforge, cloud-services-team (Kanban)
Stashbot added a comment to T217152: Monitor and scale in the Trusty grid.

Mentioned in SAL (#wikimedia-cloud) [2019-03-25T02:26:12Z] <bd808> Deleted shutdown Trusty grid nodes tools-webgrid-lighttpd-14{20,21,22,24,25,26,27,28} (T217152)

Mon, Mar 25, 2:26 AM · Cloud-VPS (Ubuntu Trusty Deprecation), Toolforge, cloud-services-team (Kanban)

Yesterday

Stashbot added a comment to T219087: Get rid of deployment-db0[34].

Mentioned in SAL (#wikimedia-releng) [2019-03-24T16:06:38Z] <Krenair> shut off old deployment-db03 instance per T219087

Sun, Mar 24, 4:06 PM · Patch-For-Review, Beta-Cluster-Infrastructure

Sat, Mar 23

Stashbot added a comment to T215562: npm 6 consistently fails with "Z_DATA_ERROR: invalid distance too far back" on some repos.

Mentioned in SAL (#wikimedia-releng) [2019-03-23T20:52:40Z] <Krinkle> Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/498664 / T215562

Sat, Mar 23, 8:52 PM · Patch-For-Review, Upstream, Continuous-Integration-Config

Fri, Mar 22

Stashbot added a comment to T216364: Upgrade heritage to Strech.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T21:40:43Z] <Lokal_Profil> Manually starting harvesting job to ensure migration (T216364) worked

Fri, Mar 22, 9:40 PM · Wiki-Loves-Monuments-Database
Stashbot added a comment to T219017: CI job beta-mediawiki-config-update-eqiad has stopped running.

Mentioned in SAL (#wikimedia-releng) [2019-03-22T19:59:45Z] <hashar> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/498466/ # T219017

Fri, Mar 22, 7:59 PM · Patch-For-Review, Beta-Cluster-Infrastructure, Release-Engineering-Team
Stashbot added a comment to T217835: PHP Fatal Error: Argument 1 passed to SpecialCollection::postZip() must be an instance of array, bool given.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T18:41:15Z] <krinkle@deploy1001> Synchronized php-1.33.0-wmf.22/extensions/Collection/: I2c4f5d005fc25c52 / T217835 (duration: 00m 52s)

Fri, Mar 22, 6:41 PM · MW-1.33-notes (1.33.0-wmf.23; 2019-03-26), Patch-For-Review, Collection, Wikimedia-production-error
Stashbot added a comment to T218918: Some interface messages (e.g. sitenotice, others) are loading old revisions of their messages.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T15:58:07Z] <James_F> UBN hot-deploy for T218918: Only load latest revision in MessageCache::loadFromDB

Fri, Mar 22, 3:58 PM · MW-1.33-notes (1.33.0-wmf.23; 2019-03-26), Patch-For-Review, User-notice, Multi-Content-Revisions (Reactive), Core Platform Team (MCR), Core Platform Team Kanban (Doing), Performance-Team, MediaWiki-Cache, MediaWiki-Internationalization, I18n
Stashbot added a comment to T218994: Deprecation warning on elasticsearch 6 expected [retry_on_conflict].

Mentioned in SAL (#wikimedia-operations) [2019-03-22T15:26:58Z] <gehel> restarting elasticsearch on elastic1046 for logging configuration change - T218994

Fri, Mar 22, 3:27 PM · Discovery-Search, CirrusSearch, Patch-For-Review, Operations
Stashbot added a comment to T218991: update elasticsearch curator to 5.6.0.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T14:33:34Z] <gehel> upgrading to elasticsearch-curator 5.6.0 on all elasticsearch nodes (including logstash) - T218991

Fri, Mar 22, 2:33 PM · Patch-For-Review, Discovery-Search (Current work), Logstash, Operations
Stashbot added a comment to T218126: LDAP: try how sssd works with our servers.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T13:59:03Z] <arturo> create VMs arturo-sgeexec-sssd-test-[12] for testing T218126

Fri, Mar 22, 1:59 PM · Patch-For-Review, cloud-services-team (Kanban)
Stashbot added a comment to T218133: Put our ldap servers behind LVS.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T11:22:07Z] <ema> lvs1002: bounce pybal to clear backends health icinga warning T218133

Fri, Mar 22, 11:22 AM · Patch-For-Review, LDAP
Stashbot added a comment to T218133: Put our ldap servers behind LVS.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T11:18:51Z] <ema> lvs1005: bounce pybal to clear backends health icinga warning T218133

Fri, Mar 22, 11:18 AM · Patch-For-Review, LDAP
Stashbot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T10:05:34Z] <ema> cp2005: repooled, serving traffic via ATS T213263

Fri, Mar 22, 10:05 AM · Patch-For-Review, Operations, Traffic
Stashbot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T09:47:53Z] <ema> cp2005: depool varnish-fe in preparation of traffic switch to ATS T213263

Fri, Mar 22, 9:48 AM · Patch-For-Review, Operations, Traffic
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T03:09:55Z] <bstorm_> T217280 depooled and rebooted 15 other nodes. Entire stretch grid is in a good state for now.

Fri, Mar 22, 3:10 AM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T02:31:04Z] <bstorm_> T217280 depooled and rebooted tools-sgeexec-0908 since it had no jobs but very high load from an NFS event that was no longer happening

Fri, Mar 22, 2:31 AM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T02:09:10Z] <bstorm_> T217280 depooled and rebooted tools-sgewebgrid-lighttpd-0924

Fri, Mar 22, 2:09 AM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T93110: Invalid parameter for message "logentry-massmessage-failure".

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:47:43Z] <Krinkle> krinkle@mwmaint1002 Fixing corrupt 'log_params' field of kawiki.logging row where log_id=1021367; T93110

Fri, Mar 22, 12:47 AM · Editing-team, MassMessage, Wikimedia-production-error
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-22T00:39:39Z] <bstorm_> T217280 depooled and rebooted tools-sgewebgrid-lighttpd-0902

Fri, Mar 22, 12:39 AM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T218963: CI outage for mediawiki/extensions/VisualEditor – job "mwext-node10-rundoc-docker" always fails and we can't merge code due to repo configuration.

Mentioned in SAL (#wikimedia-releng) [2019-03-22T00:36:43Z] <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/498276 / T218963

Fri, Mar 22, 12:36 AM · User-Ryasmeen, VisualEditor, Continuous-Integration-Infrastructure, Release-Engineering-Team (Backlog)
Stashbot added a comment to T215562: npm 6 consistently fails with "Z_DATA_ERROR: invalid distance too far back" on some repos.

Mentioned in SAL (#wikimedia-releng) [2019-03-22T00:36:19Z] <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/498276 / T215562)

Fri, Mar 22, 12:36 AM · Patch-For-Review, Upstream, Continuous-Integration-Config
Stashbot added a comment to T217932: Change log routing to ELK cluster to use rsyslog->kafka rather than talking directly to the ELK cluster.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:35:56Z] <bd808@deploy1001> Finished deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932) (duration: 01m 15s)

Fri, Mar 22, 12:36 AM · cloud-services-team (Kanban), Patch-For-Review, Striker
Stashbot added a comment to T176325: "No Phabricator accounts found for tool maintainers." while creating the a new diffusion repository.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:35:56Z] <bd808@deploy1001> Finished deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932) (duration: 01m 15s)

Fri, Mar 22, 12:36 AM · Patch-For-Review, Striker
Stashbot added a comment to T182142: Diffusion repository creation fails via toolsadmin.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:35:56Z] <bd808@deploy1001> Finished deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932) (duration: 01m 15s)

Fri, Mar 22, 12:36 AM · Patch-For-Review, Diffusion, Striker
Stashbot added a comment to T192487: Update Django to latest stable or LTS.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:35:56Z] <bd808@deploy1001> Finished deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932) (duration: 01m 15s)

Fri, Mar 22, 12:36 AM · Patch-For-Review, cloud-services-team (Kanban), Striker
Stashbot added a comment to T217932: Change log routing to ELK cluster to use rsyslog->kafka rather than talking directly to the ELK cluster.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:34:41Z] <bd808@deploy1001> Started deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932)

Fri, Mar 22, 12:35 AM · cloud-services-team (Kanban), Patch-For-Review, Striker
Stashbot added a comment to T176325: "No Phabricator accounts found for tool maintainers." while creating the a new diffusion repository.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:34:41Z] <bd808@deploy1001> Started deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932)

Fri, Mar 22, 12:35 AM · Patch-For-Review, Striker
Stashbot added a comment to T182142: Diffusion repository creation fails via toolsadmin.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:34:41Z] <bd808@deploy1001> Started deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932)

Fri, Mar 22, 12:35 AM · Patch-For-Review, Diffusion, Striker
Stashbot added a comment to T192487: Update Django to latest stable or LTS.

Mentioned in SAL (#wikimedia-operations) [2019-03-22T00:34:41Z] <bd808@deploy1001> Started deploy [striker/deploy@c4726e3]: Django upgrade and various bug fixes (T192487, T182142, T176325, T217932)

Fri, Mar 22, 12:34 AM · Patch-For-Review, cloud-services-team (Kanban), Striker

Thu, Mar 21

Stashbot added a comment to T218935: wikitech runJobs.php error.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T23:57:11Z] <gtirloni> downtimed systemd check in labweb1001/1002 (T218935)

Thu, Mar 21, 11:57 PM · Wikimedia-production-error, Discovery-Search, CirrusSearch, wikitech.wikimedia.org, cloud-services-team (Kanban)
Stashbot added a comment to T210818: Move admin cron jobs to systemd timers.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T23:53:49Z] <gtirloni> downtimed systemd check in labwen1001 (T210818)

Thu, Mar 21, 11:53 PM · Patch-For-Review, Epic, cloud-services-team (Kanban)
Stashbot added a comment to T218902: Log spam: "User account is not global" .

Mentioned in SAL (#wikimedia-operations) [2019-03-21T23:32:24Z] <jforrester@deploy1001> Synchronized php-1.33.0-wmf.22/extensions/ContentTranslation/api/ApiQueryContentTranslationSuggestions.php: SWAT T218902 CX: Return API error on anonymous suggestions queries (duration: 00m 51s)

Thu, Mar 21, 11:32 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Unplanned-Sprint-Work, Language-Team (Language-2019-January-March), User-zeljkofilipin, ContentTranslation, Wikimedia-production-error
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T23:28:12Z] <bstorm_> T217280 depooled, reloaded and repooled tools-sgeexec-0938

Thu, Mar 21, 11:28 PM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T217730: Connect wikimaniawiki to Wikidata.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T23:08:51Z] <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: SWAT T217730 Add wikimaniawiki to another special group in Wikibase client (duration: 00m 49s)

Thu, Mar 21, 11:09 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T21:53:31Z] <bstorm_> T217280 rebooted and cleared "unknown status" from tools-sgeexec-0914 after depooling

Thu, Mar 21, 9:53 PM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T21:51:04Z] <bstorm_> T217280 rebooted and cleared "unknown status" from tools-sgeexec-0909 after depooling

Thu, Mar 21, 9:51 PM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T190090: Offload pings to dedicated server.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T21:39:10Z] <XioNoX> Ping offload - replace test IP with text-lb.codfw IP on cr1/2-codfw - T190090

Thu, Mar 21, 9:39 PM · Patch-For-Review, netops, Operations, Traffic
Stashbot added a comment to T217280: LDAP server running out of memory frequently and disrupting Cloud VPS clients.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T21:26:00Z] <bstorm_> T217280 cleared error state from a couple queues and rebooted tools-sgeexec-0901 and 04 to clear other issues related

Thu, Mar 21, 9:26 PM · cloud-services-team (Kanban), Patch-For-Review, Operations, Cloud-VPS, LDAP, Toolforge
Stashbot added a comment to T218938: designate_floating_ip_ptr_records_updater is critical.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T19:58:54Z] <Krenair> Set proxy-eqiad.wmflabs.org to CNAME to proxy-eqiad1.wmflabs.org T218938

Thu, Mar 21, 7:58 PM · cloud-services-team (Kanban), Cloud-VPS
Stashbot added a comment to T213483: Fatal error: null EntityId argument passed to addEntityRedirect (Special:EntityData broken for some items).

Mentioned in SAL (#wikimedia-operations) [2019-03-21T19:45:55Z] <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: T213483 Disable RDF output of mediainfo Wikibase entities (duration: 00m 49s)

Thu, Mar 21, 7:46 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Patch-For-Review, Wikimedia-production-error, WikibaseMediaInfo, StructuredDataOnCommons
Stashbot added a comment to T213483: Fatal error: null EntityId argument passed to addEntityRedirect (Special:EntityData broken for some items).

Mentioned in SAL (#wikimedia-operations) [2019-03-21T19:40:45Z] <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: T213483 Read wmgWikibaseEntityTypesWithoutRdfOutput value (duration: 00m 50s)

Thu, Mar 21, 7:40 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Patch-For-Review, Wikimedia-production-error, WikibaseMediaInfo, StructuredDataOnCommons
Stashbot added a comment to T213483: Fatal error: null EntityId argument passed to addEntityRedirect (Special:EntityData broken for some items).

Mentioned in SAL (#wikimedia-operations) [2019-03-21T19:39:22Z] <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T213483 Set default wmgWikibaseEntityTypesWithoutRdfOutput value (duration: 00m 51s)

Thu, Mar 21, 7:39 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Patch-For-Review, Wikimedia-production-error, WikibaseMediaInfo, StructuredDataOnCommons
Stashbot added a comment to T216365: Upgrade wikiloves to Strech.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T18:50:58Z] <JeanFred> Deleted crons from old Bastion, restored them on new Bastion (T216365)

Thu, Mar 21, 6:51 PM · Tool-wikiloves
Stashbot added a comment to T218879: Upgrade to elasticsearch 6.5.4 for cirrus / eqiad.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T18:49:04Z] <gehel> resetting archived settings on elasticsearch cirrus eqiad - T218879

Thu, Mar 21, 6:49 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T216365: Upgrade wikiloves to Strech.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T18:01:01Z] <JeanFred> Deleted virtualenv, recreated in interactive container shell, reinstalled requirements, restarted webservice (T216365)

Thu, Mar 21, 6:01 PM · Tool-wikiloves
Stashbot added a comment to T216365: Upgrade wikiloves to Strech.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T17:39:16Z] <JeanFred> Stopping webservice for Trusty migration (T216365)

Thu, Mar 21, 5:39 PM · Tool-wikiloves
Stashbot added a comment to T218336: rack/setup/deploy codfw dedicated backup recovery/provisioning hosts.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T16:01:02Z] <marostegui> Poweroff db2096 for onsite maintenance T218336

Thu, Mar 21, 4:01 PM · Patch-For-Review, Operations, DBA, ops-codfw
Stashbot added a comment to T218881: labweb - wikitech_run_jobs.service in failed state.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T14:18:04Z] <gtirloni> downtimed labtestweb2001 (T218881)

Thu, Mar 21, 2:18 PM · Patch-For-Review, Operations, wikitech.wikimedia.org, cloud-services-team (Kanban)
Stashbot added a comment to T218862: acme-chief >0.13 generates wrong metadata for the endpoint used in file based deployment layout.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T14:11:29Z] <vgutierrez> re-enabling puppet in acme-chief clients - T218862

Thu, Mar 21, 2:11 PM · Acme-chief
Stashbot added a comment to T218024: decommmision: labtestweb2001.wikimedia.org.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T14:09:25Z] <arturo> T218024 disabled icinga checks for labtestweb2001

Thu, Mar 21, 2:09 PM · Patch-For-Review, decommission, ops-codfw, Operations, cloud-services-team (Kanban)
Stashbot added a comment to T218862: acme-chief >0.13 generates wrong metadata for the endpoint used in file based deployment layout.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T13:58:12Z] <vgutierrez> update acme-chief to version 0.15 in acmechief1001 - T218862

Thu, Mar 21, 1:58 PM · Acme-chief
Stashbot added a comment to T218862: acme-chief >0.13 generates wrong metadata for the endpoint used in file based deployment layout.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T13:54:18Z] <vgutierrez> disabling puppet in acme-chief clients - T218862

Thu, Mar 21, 1:54 PM · Acme-chief
Stashbot added a comment to T210818: Move admin cron jobs to systemd timers.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T13:49:10Z] <gtirloni> converted openstack cronjobs to systemd timers (T210818)

Thu, Mar 21, 1:49 PM · Patch-For-Review, Epic, cloud-services-team (Kanban)
Stashbot added a comment to T210818: Move admin cron jobs to systemd timers.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T13:18:00Z] <gtirloni> downtimed cloudcontrol*, cloudservices*, labcontrol*, labweb* (T210818)

Thu, Mar 21, 1:18 PM · Patch-For-Review, Epic, cloud-services-team (Kanban)
Stashbot added a comment to T216497: CloudVPS: workaround archival of jessie-backports repo.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T12:40:16Z] <arturo> T216497 remove python-cliff from jessie-wikimedia/openstack-mitaka-jessie

Thu, Mar 21, 12:40 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
Stashbot added a comment to T216497: CloudVPS: workaround archival of jessie-backports repo.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T12:08:31Z] <arturo> T216497 add python-cliff to jessie-wikimedia/openstack-mitaka-jessie

Thu, Mar 21, 12:08 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
Stashbot added a comment to T218862: acme-chief >0.13 generates wrong metadata for the endpoint used in file based deployment layout.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T12:02:29Z] <vgutierrez> uploaded acme-chief 0.15 to apt.wikimedia.org (buster) - T218862

Thu, Mar 21, 12:02 PM · Acme-chief
Stashbot added a comment to T217929: Wikimedia-Site-requests: LMU Edit-a-thon.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T11:04:21Z] <zfilipin@deploy1001> Synchronized wmf-config/throttle.php: SWAT: [[gerrit:495494|Add new throttle rule for LMU Edit-a-thon (T217929)]] (duration: 00m 57s)

Thu, Mar 21, 11:04 AM · Patch-For-Review, Wikimedia-Site-requests
Stashbot added a comment to T213195: Migrate cxserver to kubernetes.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T09:42:20Z] <jijiki> Depool scb* in codfw from serving cxserver, finishing its migration to k8s - T213195

Thu, Mar 21, 9:42 AM · User-jijiki, Patch-For-Review, Services (watching), Release-Engineering-Team (Next), serviceops, CX-cxserver, Release Pipeline
Stashbot added a comment to T207295: Expose not-yet-live certs to clients so they can handle OCSP stapling.

Mentioned in SAL (#wikimedia-operations) [2019-03-21T08:01:35Z] <vgutierrez> deploying directory based certificates in acme-chief clients - T207295

Thu, Mar 21, 8:01 AM · Patch-For-Review, Acme-chief
Stashbot added a comment to T217086: Investigate why the new Son of Grid Engine grid landed in a worse state when NFS was filled than the old Sun Grid Engine grid did.

Mentioned in SAL (#wikimedia-cloud) [2019-03-21T00:49:50Z] <gtirloni> nfs-exportd interval changes from 60 to 300s (T217086)

Thu, Mar 21, 12:49 AM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)

Wed, Mar 20

Stashbot added a comment to T218830: selectandother widget broken, particular on Special:GlobalBlock since 1.33.0-wmf.22.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T23:49:24Z] <jforrester@deploy1001> Synchronized php-1.33.0-wmf.22/resources/lib/ooui/oojs-ui-core.js: SWAT T218722 T218830 Bring forward UBN OOUI fix (duration: 00m 57s)

Wed, Mar 20, 11:49 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), MediaWiki-General-or-Unknown, Regression
Stashbot added a comment to T218722: DropdownInputWidget with MenuSectionOptionWidget triggers endless change events.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T23:49:24Z] <jforrester@deploy1001> Synchronized php-1.33.0-wmf.22/resources/lib/ooui/oojs-ui-core.js: SWAT T218722 T218830 Bring forward UBN OOUI fix (duration: 00m 57s)

Wed, Mar 20, 11:49 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), OOUI (OOUI-0.31.1)
Stashbot added a comment to T214075: Enable federated access to entities and properties from Wikidata to Commons.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T22:29:11Z] <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T214075 Enable federation of Wikidata items and properties on Test Commons (duration: 00m 57s)

Wed, Mar 20, 10:29 PM · Multimedia-Current-Work, SDC-Depicts (Depicts-Crawling), SDC Engineering (Depicts-Crawling), Patch-For-Review, Wikidata, Multimedia
Stashbot added a comment to T190090: Offload pings to dedicated server.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T21:37:44Z] <XioNoX> apply transit-in4 term offload-ping4 with test IP to cr1/2-codfw - T190090

Wed, Mar 20, 9:37 PM · Patch-For-Review, netops, Operations, Traffic
Stashbot added a comment to T190090: Offload pings to dedicated server.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T21:00:50Z] <XioNoX> apply icmp redirect on cr1-codfw:xe-5/0/2 (to cr4-ulsfo) for test IP 208.80.154.225 - T190090

Wed, Mar 20, 9:01 PM · Patch-For-Review, netops, Operations, Traffic
Stashbot added a comment to T217730: Connect wikimaniawiki to Wikidata.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T11:44:57Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:497578|Add wikimania as a special group to wikidata sitelinks (T217730)]] (duration: 00m 50s)

Wed, Mar 20, 11:45 AM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Stashbot added a comment to T217730: Connect wikimaniawiki to Wikidata.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T11:40:54Z] <ladsgroup@deploy1001> Synchronized dblists/wikidataclient.dblist: SWAT: [[gerrit:497579|Add wikimaniawiki to wikidataclient.dblist (T217730)]] (duration: 00m 50s)

Wed, Mar 20, 11:41 AM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Stashbot added a comment to T217643: Deploy AMC pt 1 to Arabic, Indonesian, and Spanish Wikipedias.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T11:34:45Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:497500|Enable Advanced Mobile Contributions mode for ar,id,es and test wikis (T217643)]] (duration: 00m 50s)

Wed, Mar 20, 11:34 AM · Patch-For-Review, Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q3), Advanced Mobile Contributions
Stashbot added a comment to T218535: Score extension now leaving large amounts of space on rendering.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T11:28:30Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:497433|Partially revert "Enable musical notation datatype in wikidata" (T218535)]] (duration: 00m 50s)

Wed, Mar 20, 11:28 AM · MediaWiki-Configuration, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Patch-For-Review, MediaWiki-extensions-Score, Regression
Stashbot added a comment to T217739: Set maxSerializedEntitySize to 3000.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T11:16:15Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:496161|Increased maxSerializedEntitySize from 2500 to 3000 (T217739)]] (duration: 01m 47s)

Wed, Mar 20, 11:16 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Africa-Wikimedia-Developers, Patch-For-Review, Wikidata, Wikimedia-Site-requests
Stashbot added a comment to T205870: Provision >= 50% of statsd/Graphite-only metrics in Prometheus.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T10:26:57Z] <godog> reimage prometheus1003 with stretch - T205870

Wed, Mar 20, 10:27 AM · Performance-Team (Radar), Patch-For-Review, monitoring, Operations
Stashbot added a comment to T218546: ToolForge Stretch grid: not able to restart wsexport webservice.

Mentioned in SAL (#wikimedia-cloud) [2019-03-20T10:10:39Z] <arturo> manually killing zombie procs in tools-sgewebgrid-lightttpd-0920 (T218546)

Wed, Mar 20, 10:10 AM · cloud-services-team (Kanban), Toolforge
Stashbot added a comment to T218546: ToolForge Stretch grid: not able to restart wsexport webservice.

Mentioned in SAL (#wikimedia-cloud) [2019-03-20T10:03:13Z] <arturo> T218546 force job deletion 788036

Wed, Mar 20, 10:03 AM · cloud-services-team (Kanban), Toolforge
Stashbot added a comment to T187960: Rack/cable/configure asw2-a-eqiad switch stack.

Mentioned in SAL (#wikimedia-operations) [2019-03-20T06:09:58Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool databases in row A - T187960 (duration: 00m 49s)

Wed, Mar 20, 6:10 AM · Wikidata, wikidata-tech-focus, Reading-Infrastructure-Team-Backlog, Cognate, Language-Team, Growth-Team, Patch-For-Review, Operations, ops-eqiad, netops

Tue, Mar 19

Stashbot added a comment to T218706: Deployed repos search returns no results.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T23:58:42Z] <legoktm> rolling restart of everything, issues probably due to Gerrit downtime (T218706)

Tue, Mar 19, 11:58 PM · VPS-project-codesearch
Stashbot added a comment to T218553: Wikibase CI appears to be broken.

Mentioned in SAL (#wikimedia-releng) [2019-03-19T22:47:44Z] <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/497322 / T218553

Tue, Mar 19, 10:47 PM · MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Addshore, Wikidata
Stashbot added a comment to T218559: puppet broken on integration WMCS instances due to openstack Debian packages.

Mentioned in SAL (#wikimedia-releng) [2019-03-19T20:23:19Z] <hashar> integration: sudo cumin --force '*' 'rm /etc/apt/preferences.d/jessie_mitaka_pinning_*' # T218559

Tue, Mar 19, 8:23 PM · Patch-For-Review, cloud-services-team (Kanban), Operations-Software-Development, Continuous-Integration-Infrastructure
Stashbot added a comment to T218559: puppet broken on integration WMCS instances due to openstack Debian packages.

Mentioned in SAL (#wikimedia-releng) [2019-03-19T19:56:45Z] <hashar> integration: sudo cumin --force '*' 'rm /etc/apt/sources.list.d/openstack-mitaka-jessie.list' # T218559

Tue, Mar 19, 7:56 PM · Patch-For-Review, cloud-services-team (Kanban), Operations-Software-Development, Continuous-Integration-Infrastructure
Stashbot added a comment to T218633: Sort out orphaned proxy entries.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T18:00:43Z] <Krenair> Removed old proxy-eqiad A record pointing at no-longer-used IP T218633

Tue, Mar 19, 6:00 PM · cloud-services-team
Stashbot added a comment to T218633: Sort out orphaned proxy entries.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T18:00:24Z] <Krenair> Updated m and www records to CNAME to proxy-eqiad1 instead of proxy-eqiad (which was an A record to a now unused IP), found in review at T218633

Tue, Mar 19, 6:00 PM · cloud-services-team
Stashbot added a comment to T132839: [RfC] Property suggester suggests human properties for non-human items.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:47:47Z] <Lucas_WMDE> Updated the Wikidata property suggester with data from last Monday's JSON dump and applied the T132839 workarounds (T216270)

Tue, Mar 19, 5:48 PM · MediaWiki-extensions-PropertySuggester, Patch-For-Review, Proposal, Wikidata-Sprint-2016-11-08, Wikidata-Sprint-2016-10-12, Wikidata-Sprint-2016-05-24, Wikidata-Sprint-2016-05-10, Wikidata-Sprint-2016-04-26, Wikidata
Stashbot added a comment to T216270: Update property suggester data.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:47:47Z] <Lucas_WMDE> Updated the Wikidata property suggester with data from last Monday's JSON dump and applied the T132839 workarounds (T216270)

Tue, Mar 19, 5:47 PM · Wikidata-Campsite, MediaWiki-extensions-PropertySuggester, Wikidata
Stashbot added a comment to T218633: Sort out orphaned proxy entries.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T17:46:53Z] <Krenair> Deleted orphaned DNS records ase.wikipedia, math.testme, mathoid.testme, mws.testme, openid-wiki.instance-proxy, structured.wikiquote T218633

Tue, Mar 19, 5:46 PM · cloud-services-team
Stashbot added a comment to T218633: Sort out orphaned proxy entries.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T17:44:16Z] <Krenair> Deleted orphaned DNS records codesearch-sourcegraph1, migrat4-testone, migrat4-testtwo, sourcegraph1, sourcegraph11 T218633

Tue, Mar 19, 5:44 PM · cloud-services-team
Stashbot added a comment to T150377: Use 'outside' instead of 'Inside' for list-style in Mobile version so that long line bullet list items are more scannable.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:33:12Z] <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@64f09a0]: T150377: Bump wikimedia-page-library to 6.3.0 (duration: 01m 50s)

Tue, Mar 19, 5:33 PM · Readers-Web-Backlog, MW-1.33-notes (1.33.0-wmf.19; 2019-02-26), Patch-For-Review, Unplanned-Sprint-Work, MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), User-Jdlrobson, MinervaNeue, Design, WMF-Design
Stashbot added a comment to T150377: Use 'outside' instead of 'Inside' for list-style in Mobile version so that long line bullet list items are more scannable.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:31:22Z] <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@64f09a0]: T150377: Bump wikimedia-page-library to 6.3.0

Tue, Mar 19, 5:31 PM · Readers-Web-Backlog, MW-1.33-notes (1.33.0-wmf.19; 2019-02-26), Patch-For-Review, Unplanned-Sprint-Work, MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), User-Jdlrobson, MinervaNeue, Design, WMF-Design
Stashbot added a comment to T150377: Use 'outside' instead of 'Inside' for list-style in Mobile version so that long line bullet list items are more scannable.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:18:11Z] <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@64f09a0]: T150377: Bump wikimedia-page-library to 6.3.0 (duration: 03m 47s)

Tue, Mar 19, 5:18 PM · Readers-Web-Backlog, MW-1.33-notes (1.33.0-wmf.19; 2019-02-26), Patch-For-Review, Unplanned-Sprint-Work, MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), User-Jdlrobson, MinervaNeue, Design, WMF-Design
Stashbot added a comment to T150377: Use 'outside' instead of 'Inside' for list-style in Mobile version so that long line bullet list items are more scannable.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T17:14:24Z] <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@64f09a0]: T150377: Bump wikimedia-page-library to 6.3.0

Tue, Mar 19, 5:14 PM · Readers-Web-Backlog, MW-1.33-notes (1.33.0-wmf.19; 2019-02-26), Patch-For-Review, Unplanned-Sprint-Work, MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), User-Jdlrobson, MinervaNeue, Design, WMF-Design
Stashbot added a comment to T207295: Expose not-yet-live certs to clients so they can handle OCSP stapling.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T16:45:15Z] <vgutierrez> uploaded acme-chief 0.14 to apt.wikimedia.org (buster) - T218685 T218418 T207295

Tue, Mar 19, 4:45 PM · Patch-For-Review, Acme-chief
Stashbot added a comment to T218418: CN + SNI list on config file doesn't match issued certificate on some scenarios.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T16:45:15Z] <vgutierrez> uploaded acme-chief 0.14 to apt.wikimedia.org (buster) - T218685 T218418 T207295

Tue, Mar 19, 4:45 PM · Patch-For-Review, Acme-chief
Stashbot added a comment to T218685: acme-chief creates absolute symlinks for "live" and relative for "new" on certificate renewal.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T16:45:15Z] <vgutierrez> uploaded acme-chief 0.14 to apt.wikimedia.org (buster) - T218685 T218418 T207295

Tue, Mar 19, 4:45 PM · Acme-chief
Stashbot added a comment to T187960: Rack/cable/configure asw2-a-eqiad switch stack.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:27:56Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Set s2 read only OFF - T187960 (duration: 00m 26s)

Tue, Mar 19, 3:28 PM · Wikidata, wikidata-tech-focus, Reading-Infrastructure-Team-Backlog, Cognate, Language-Team, Growth-Team, Patch-For-Review, Operations, ops-eqiad, netops
Stashbot added a comment to T187960: Rack/cable/configure asw2-a-eqiad switch stack.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:21:10Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Set s2 database master on read only - T187960 (duration: 00m 48s)

Tue, Mar 19, 3:21 PM · Wikidata, wikidata-tech-focus, Reading-Infrastructure-Team-Backlog, Cognate, Language-Team, Growth-Team, Patch-For-Review, Operations, ops-eqiad, netops
Stashbot added a comment to T187960: Rack/cable/configure asw2-a-eqiad switch stack.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T15:12:24Z] <XioNoX> eqiad A7 servers uplink move - T187960

Tue, Mar 19, 3:12 PM · Wikidata, wikidata-tech-focus, Reading-Infrastructure-Team-Backlog, Cognate, Language-Team, Growth-Team, Patch-For-Review, Operations, ops-eqiad, netops
Stashbot added a comment to T187960: Rack/cable/configure asw2-a-eqiad switch stack.

Mentioned in SAL (#wikimedia-operations) [2019-03-19T14:44:51Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool databases in row A - T187960 (duration: 00m 48s)

Tue, Mar 19, 2:45 PM · Wikidata, wikidata-tech-focus, Reading-Infrastructure-Team-Backlog, Cognate, Language-Team, Growth-Team, Patch-For-Review, Operations, ops-eqiad, netops
Stashbot added a comment to T217922: Migrate Wikilabels from labsdb1004 to clouddb1002.

Mentioned in SAL (#wikimedia-cloud) [2019-03-19T14:02:01Z] <halfak> stopping the web service https://phabricator.wikimedia.org/T217922

Tue, Mar 19, 2:02 PM · Scoring-platform-team (Current), Wikilabels, Cloud-VPS
Stashbot added a comment to T192217: Remove the "check" pipeline and Zuul's user-filter.

Mentioned in SAL (#wikimedia-releng) [2019-03-19T14:01:13Z] <hashar> Removed Zuul "check" pipeline | https://gerrit.wikimedia.org/r/#/c/integration/config/+/493188/ | T192217

Tue, Mar 19, 2:01 PM · Patch-For-Review, Release-Engineering-Team (Backlog), Continuous-Integration-Infrastructure