Page MenuHomePhabricator

thcipriani (Tyler Cipriani)
¯\_(ツ)_/¯Administrator

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Feb 9 2015, 10:04 PM (297 w, 5 d)
Roles
Administrator
Availability
Available
IRC Nick
thcipriani
LDAP User
Unknown
MediaWiki User
TCipriani (WMF) [ Global Accounts ]

Recent Activity

Fri, Oct 23

thcipriani added a comment to D1185: Fix regex in isValidGitShallowCloneResponse.

The output of the shallow clone from the "git-http-backend" is:

Fri, Oct 23, 11:57 PM · Phabricator, Release-Engineering-Team

Wed, Oct 21

thcipriani moved T265814: Special:MathStatus fails on api.wikimedia.org from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:55 PM · Patch-For-Review, API-Portal, User-DannyS712, Platform Team Initiatives (API Gateway), Math, Wikimedia-production-error
thcipriani moved T266129: Invariant failed: Bad UTF-8 (full string verification) -- bad UTF-8 from database from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:54 PM · Parsing-Active-Work, Parsoid, Wikimedia-production-error
thcipriani added a comment to T266129: Invariant failed: Bad UTF-8 (full string verification) -- bad UTF-8 from database.

Similar to T242298: Invariant failed: Bad UTF-8 at end of string (2 byte sequence)

Wed, Oct 21, 6:54 PM · Parsing-Active-Work, Parsoid, Wikimedia-production-error
thcipriani moved T266162: TypeError: null is not an object (evaluating 'extraData.userTopics.length') from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:53 PM · JavaScript, GrowthExperiments, Growth-Team, Wikimedia-production-error
thcipriani moved T266163: PHP fatal/ Js exception TypeError: haystack is undefined in stringEndsWith from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:53 PM · Advanced-Search, Wikimedia-production-error
thcipriani moved T266165: TypeError: lastCssBuffer is null at addEmbeddedCSS in resourceloader startup module from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:52 PM · JavaScript, MediaWiki-ResourceLoader, Performance-Team, Wikimedia-production-error
thcipriani moved T249742: Call to a member function getAttribute() on null from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:52 PM · Wikimedia-production-error, Parsoid
thcipriani moved T266171: Return value of Wikimedia\Parsoid\Config\Env::getDOMFragment() must be an instance of DOMDocumentFragment, null returned from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:52 PM · Parsoid, Parsing-Team--ARCHIVED, Wikimedia-production-error
thcipriani moved T266179: Use of SpecialBlock::canBlockEmail was deprecated in MediaWiki 1.36. [Called from SpecialCheckUser::getBlockForm] from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:52 PM · Platform Engineering, CheckUser, Wikimedia-production-error
thcipriani moved T266181: Argument 1 passed to LinkRenderer::makeKnownLink() must implement interface LinkTarget, null given, called in CologneBlueTemplate.php from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 21, 6:51 PM · MW-1.36-notes (1.36.0-wmf.16; 2020-11-03), CologneBlue, Release-Engineering-Team (Logspam), Wikimedia-production-error
thcipriani added a project to T249742: Call to a member function getAttribute() on null: Wikimedia-production-error.
Wed, Oct 21, 6:31 PM · Wikimedia-production-error, Parsoid
thcipriani moved T114488: Automate the recurring management of wikitech:Deployments and phab:#train_deployments from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:50 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Deployments, User-MModell
thcipriani moved T247186: check gerrit-replica healthcheck alerts from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:50 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Gerrit
thcipriani moved T255596: PipelineLib helm test log output from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:39 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release-Engineering-Team (Pipeline), Release Pipeline
thcipriani moved T264236: Make Pipelinelib promote step use service name instead of repo name from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:39 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release Pipeline, Release-Engineering-Team (Pipeline)
thcipriani moved T253773: Add Timed media handler to local-charts from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:38 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), User-brennen, TimedMediaHandler, local-charts
thcipriani moved T261809: Migrate mathoid and other remaining services to use PipelineLib from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:38 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Mathoid, Release-Engineering-Team (Pipeline), Math
thcipriani moved T266081: Evaluate how we might run buildpack's `pack` in CI from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:33 PM · Continuous-Integration-Infrastructure, Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2))
thcipriani moved T259817: Sketch MediaWiki production image composition and workflows from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:33 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), MW-on-K8s, Release-Engineering-Team (Pipeline)
thcipriani moved T259319: Ensure failed helm deployments are cleaned up by PipelineLib from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:33 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release-Engineering-Team (Pipeline), Release Pipeline
thcipriani moved T260828: Experiment with PipelineLib/Blubber driven MediaWiki container image pipeline from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:32 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release-Engineering-Team (Pipeline)
thcipriani moved T258630: Improve scheduling of CI jobs invoked by zuul from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:18 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Upstream, Release-Engineering-Team (CI & Testing services), Patch-For-Review, Continuous-Integration-Infrastructure, Zuul, User-brennen, User-dancy
thcipriani moved T258821: Monitor /var/log/zuul/error.log from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:18 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), observability, Continuous-Integration-Infrastructure, User-brennen, Zuul, User-dancy
thcipriani moved T260827: Investigate/trace rebuildLocalisationCache process for MW containerization from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Wed, Oct 21, 5:17 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release-Engineering-Team (Pipeline)
thcipriani archived Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)).
Wed, Oct 21, 1:08 AM

Tue, Oct 20

thcipriani triaged T266055: Update Scap to perform rolling restart for all MW deploy as Medium priority.
Tue, Oct 20, 6:08 PM · Patch-For-Review, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Scap
thcipriani moved T266055: Update Scap to perform rolling restart for all MW deploy from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Tue, Oct 20, 6:08 PM · Patch-For-Review, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Scap
thcipriani assigned T266055: Update Scap to perform rolling restart for all MW deploy to dancy.

Assigning per brief discussion in IRC. @dancy feel free to loop in others (@LarsWirzenius will be helpful for actual release) as needed.

Tue, Oct 20, 6:07 PM · Patch-For-Review, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Scap
thcipriani added a comment to T243009: Add option in Scap to restart php-fpm for emergency deployments, and skip depooling/pooling servers.

Open questions (I guess for @Joe):

  • Would "simply" restarting php-fpm directly abort on-going requests? I assume so, and we're fine with this. Just checking so that we can document this as expected.

Yes, restarting php-fpm will kill all in-flight requests

Tue, Oct 20, 5:59 PM · Patch-For-Review, Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Sustainability (Incident Followup), Release-Engineering-Team (Deployment services), Scap
thcipriani updated the task description for T266055: Update Scap to perform rolling restart for all MW deploy.
Tue, Oct 20, 5:46 PM · Patch-For-Review, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Scap
thcipriani added a comment to T263008: Gerrit out of heap.

Gerrit tuning infos: https://ctf-dev-environment-vagrant.s3.amazonaws.com/Gerrit-Performance-Tuning-Cheat-Sheet.pdf

We probably want to look at jgit config:

modules/gerrit/templates/gerrit.config.erb
[core]
    packedGitOpenFiles = <%= @git_open_files %>  # 20000
    packedGitLimit = 4g
Tue, Oct 20, 5:33 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), git-protocol-v2, Release-Engineering-Team (Development services), Gerrit

Wed, Oct 14

thcipriani moved T265525: Wikibase\Client\Usage\Sql\SqlSubscriptionManager::subscribe: Expected mass rollback of all peer transactions (DBO_TRX set) from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 14, 6:52 PM · Platform Engineering, Wikidata-Campsite, Wikidata, Wikimedia-production-error
thcipriani created T265525: Wikibase\Client\Usage\Sql\SqlSubscriptionManager::subscribe: Expected mass rollback of all peer transactions (DBO_TRX set).
Wed, Oct 14, 6:52 PM · Platform Engineering, Wikidata-Campsite, Wikidata, Wikimedia-production-error
thcipriani moved T265524: Cannot diff content types other than MassMessageListContent from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 14, 6:46 PM · Release-Engineering-Team (Logspam), MassMessage, Wikimedia-production-error
thcipriani updated the task description for T265524: Cannot diff content types other than MassMessageListContent.
Wed, Oct 14, 6:46 PM · Release-Engineering-Team (Logspam), MassMessage, Wikimedia-production-error
thcipriani created T265524: Cannot diff content types other than MassMessageListContent.
Wed, Oct 14, 6:45 PM · Release-Engineering-Team (Logspam), MassMessage, Wikimedia-production-error
thcipriani raised the priority of T203075: Warning: MediaWiki\Storage\SqlBlobStore::fetchBlob: Bad data in text row from Medium to High.

Happening on hrwiki as of today (2020-10-14) with wmf.11: Failed to load data blob from tt:1677927: Bad data in text row 1677927. Use findBadBlobs.php to remedy.. If this problem persist, use the findBadBlobs maintenance script to investigate the issue and mark bad blobs.

Wed, Oct 14, 6:41 PM · MediaWiki-Revision-backend, Platform Team Workboards (Clinic Duty Team), Wikimedia-production-error
thcipriani moved T265523: PHP Notice: Undefined index: href (from ContentTranslationSpecialPage.php) from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 14, 6:35 PM · Language-Team (Language-2020-October-December), MW-1.36-notes (1.36.0-wmf.16; 2020-11-03), ContentTranslation, Wikimedia-production-error
thcipriani created T265523: PHP Notice: Undefined index: href (from ContentTranslationSpecialPage.php).
Wed, Oct 14, 6:35 PM · Language-Team (Language-2020-October-December), MW-1.36-notes (1.36.0-wmf.16; 2020-11-03), ContentTranslation, Wikimedia-production-error
thcipriani moved T265521: No formatter defined for value type unknown. from Untriaged to Oct2020/1.36.0-wmf.11+ on the Wikimedia-production-error board.
Wed, Oct 14, 6:28 PM · Wikidata-Campsite, Wikibase, Wikimedia-production-error, Wikidata
thcipriani updated the task description for T265521: No formatter defined for value type unknown..
Wed, Oct 14, 6:28 PM · Wikidata-Campsite, Wikibase, Wikimedia-production-error, Wikidata
thcipriani created T265521: No formatter defined for value type unknown..
Wed, Oct 14, 6:28 PM · Wikidata-Campsite, Wikibase, Wikimedia-production-error, Wikidata
thcipriani added a comment to T237467: Invariant failed: Bad UTF-8 (full string verification).

Still seen in production today for wmf.11:

Wed, Oct 14, 6:22 PM · MW-1.36-notes (1.36.0-wmf.16; 2020-11-03), User-notice, Parsing-Active-Work, WMDE-QWERTY-Sprint-2020-08-26, Patch-For-Review, Wikimedia-production-error, User-brennen, Parsoid
thcipriani merged task T263132: Could not enqueue jobs from stream mediawiki.job.cirrusSearchIncomingLinkCount into T249745: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable".
Wed, Oct 14, 6:19 PM · Analytics-Kanban, Operations, Analytics, Event-Platform, Wikimedia-production-error
thcipriani merged T263132: Could not enqueue jobs from stream mediawiki.job.cirrusSearchIncomingLinkCount into T249745: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable".
Wed, Oct 14, 6:19 PM · Patch-For-Review, User-brennen, serviceops, Platform Team Workboards (Clinic Duty Team), WMF-JobQueue, Wikimedia-production-error
thcipriani added a comment to T249745: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable".

There was a spike in events related to "Could not enqueue jobs from stream" for various streams at ~12pm UTC: https://logstash.wikimedia.org/goto/7bbe517376de0be509c6a07dd8d027bc

Wed, Oct 14, 6:18 PM · Patch-For-Review, User-brennen, serviceops, Platform Team Workboards (Clinic Duty Team), WMF-JobQueue, Wikimedia-production-error
thcipriani closed T263181: 1.36.0-wmf.15 deployment blockers as Declined.

Week of data center switchover

Wed, Oct 14, 12:53 AM · User-brennen, Release-Engineering-Team-TODO, Release, Train Deployments

Tue, Oct 13

thcipriani added a comment to T261403: Move dedcode/mwaddlink from github to gerrit.

if you help me with requesting a gerrit-repo that would be great (no experience with that yet). also

Pinging Release-Engineering-Team and serviceops with some questions about this process:

First, as I understand it, since we are using the Deployment pipeline we just need a single gerrit repository that will have the .pipeline directory (and not a separate deploy repo as it appears ORES has), is that right? (Question for serviceops I think)

Tue, Oct 13, 11:05 PM · serviceops, Release-Engineering-Team, Growth-Structured-Tasks, Growth-Team
thcipriani added a comment to T263179: 1.36.0-wmf.13 deployment blockers.

Per https://wikitech.wikimedia.org/wiki/Deployments, 8 hours from now 1.36.0-wmf.13 will be deployed to group 0. But, T263177: 1.36.0-wmf.11 deployment blockers hasn't been reattempted yet, and everything is currently on wmf.10. Is there a plan for rolling out .11 and then .13, or are we going straight to .13?

Tue, Oct 13, 1:55 PM · Patch-For-Review, Release-Engineering-Team-TODO, Release, Train Deployments

Fri, Oct 9

thcipriani created T265183: In a k8s world: where does MediaWiki code live?.
Fri, Oct 9, 10:45 PM · MW-on-K8s

Wed, Oct 7

thcipriani updated subscribers of T248180: parser tests running under phpunit are broken in extensions.

None of the patches for GrowthExperiments pass CI because of this. Does anyone have ideas on what caused this problem?

Wed, Oct 7, 2:12 PM · Release-Engineering-Team, MediaWiki-Core-Testing, ci-test-error
thcipriani updated the task description for T256367: WMF-NDA access for DannyS712.
Wed, Oct 7, 1:54 PM · WMF-NDA-Requests, User-DannyS712

Tue, Oct 6

thcipriani added a comment to T256367: WMF-NDA access for DannyS712.

I also support this proposal. @DannyS712 has been very helpful in all sorts of production firefighting recently -- of particular note, T264369

@greg @thcipriani @faidon @mark Can one of you please help move this along?

Tue, Oct 6, 10:00 PM · WMF-NDA-Requests, User-DannyS712
thcipriani created T264801: 1.36.0-wmf.21 deployment blockers.
Tue, Oct 6, 8:52 PM · Release, Train Deployments

Fri, Oct 2

thcipriani closed T264383: 500 Internal Server Error when logging in as Resolved.
sudo apt-get clean
Fri, Oct 2, 1:16 PM · Release-Engineering-Team, GitLab-Test
thcipriani added a comment to T264383: 500 Internal Server Error when logging in.

hrm, seems like we ran out of disk space. Investigating.

Fri, Oct 2, 1:13 PM · Release-Engineering-Team, GitLab-Test

Thu, Oct 1

thcipriani claimed T264362: Scap feature: restart php-fpm on deployment.
Thu, Oct 1, 8:47 PM · Patch-For-Review, User-brennen, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Sustainability (Incident Followup)
thcipriani moved T264362: Scap feature: restart php-fpm on deployment from INBOX to Maintenance on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.
Thu, Oct 1, 8:47 PM · Patch-For-Review, User-brennen, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Sustainability (Incident Followup)
thcipriani created T264362: Scap feature: restart php-fpm on deployment.
Thu, Oct 1, 8:46 PM · Patch-For-Review, User-brennen, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Sustainability (Incident Followup)

Wed, Sep 30

thcipriani moved T261809: Migrate mathoid and other remaining services to use PipelineLib from INBOX to New Work on the Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)) board.
Wed, Sep 30, 5:41 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Mathoid, Release-Engineering-Team (Pipeline), Math

Sun, Sep 27

thcipriani updated subscribers of T263872: Localisation cache must be purged after or during train deploy, not (just) before.

One other thing to note: there are two purges that happen as part of the scap sync-world and one seems to happen way too soon. @dduvall pointed this out to me recently that rebuildLocalisationCache.php does a purge:

Sun, Sep 27, 7:57 PM · Performance-Team (Radar), Scap, Release-Engineering-Team

Fri, Sep 25

thcipriani added a comment to T252124: Scap configuration for WDQS should get server groups from a known source or truth.

@thcipriani I have the change in puppet (https://gerrit.wikimedia.org/r/630081). What else would I need to do to guarantee that generated lists in /etc/dsh/groups are picked up? I rather leave everything else configured as it is right now (https://github.com/wikimedia/wikidata-query-deploy/blob/master/scap/scap.cfg)

Fri, Sep 25, 8:25 PM · Discovery-Search (Current work), Scap, Wikidata, Operations, Wikidata-Query-Service

Sep 24 2020

thcipriani awarded Blog Post: CI now updates your deployment-charts a Party Time token.
Sep 24 2020, 8:19 PM
thcipriani added a comment to T263721: Add hashar to `archiva-deployers` LDAP group.

Approved as Release-Engineering-Team manager

Sep 24 2020, 6:08 PM · Gerrit, Operations, LDAP-Access-Requests
thcipriani added a comment to T257978: 1.36.0-wmf.10 deployment blockers.

Hi @DannyS712. @thcipriani instructed me to remove the non-blocking tasks from the subtasks list. An empty subtasks lists is the indication that there are no outstanding train blockers. Hope that helps!

Sep 24 2020, 12:35 AM · Patch-For-Review, User-brennen, Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release, Train Deployments

Sep 23 2020

thcipriani added a comment to T263682: gerrit yields Error 400 (Bad Request): Error in operator parentproject:integration.

I deleted a repository on disk (for T257775) which seemed to work fine but Gerrit internal caches were not invalidated as a result.

Solved it by flushing the project_list cache.

Sep 23 2020, 7:19 PM · Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release-Engineering-Team (Development services), Gerrit
thcipriani triaged T263675: Undefined class constant 'LIMIT' as Unbreak Now! priority.

Added as a train blocker

Sep 23 2020, 6:18 PM · MW-1.36-notes (1.36.0-wmf.10; 2020-09-22), User-DannyS712, MobileFrontend, Readers-Web-Backlog, Wikimedia-production-error
thcipriani added a parent task for T263675: Undefined class constant 'LIMIT': T257978: 1.36.0-wmf.10 deployment blockers.
Sep 23 2020, 6:18 PM · MW-1.36-notes (1.36.0-wmf.10; 2020-09-22), User-DannyS712, MobileFrontend, Readers-Web-Backlog, Wikimedia-production-error
thcipriani added a subtask for T257978: 1.36.0-wmf.10 deployment blockers: T263675: Undefined class constant 'LIMIT'.
Sep 23 2020, 6:18 PM · Patch-For-Review, User-brennen, Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release, Train Deployments
thcipriani moved T263500: phan-taint-check-plugin: Undefined constant 'ast\AST_LIST' from INBOX to New Work on the Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)) board.
Sep 23 2020, 5:30 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Patch-For-Review, Release-Engineering-Team (CI & Testing services), Quibble, Platform Team Initiatives (Parsoid REST API in PHP (CDP2)), phan-taint-check-plugin
thcipriani added a comment to T263601: ApiQueryGlobalUsage.php Undefined index error when accessing $pageIds.

Requests are generating a lot of noise here. For example the 1 request-id in this task generated 164 notices https://logstash.wikimedia.org/goto/0aa65e4883b273a1497897615ab6a081

Sep 23 2020, 3:38 PM · Patch-For-Review, Platform Team Workboards (Clinic Duty Team), MW-1.36-notes (1.36.0-wmf.11; 2020-09-29), GlobalUsage, Release-Engineering-Team (Logspam), Wikimedia-production-error
thcipriani triaged T263592: Use of Language::commafy with a non-numeric string was deprecated in MediaWiki 1.36. [Called from Language::formatNum] as Unbreak Now! priority.

train blockers are UBNs

Sep 23 2020, 3:33 PM · Commons, MediaWiki-File-management, MW-1.36-notes (1.36.0-wmf.16; 2020-11-03), MediaWiki-extensions-Scribunto, MediaWiki-Parser, MediaWiki-General, Patch-For-Review
thcipriani raised the priority of T263601: ApiQueryGlobalUsage.php Undefined index error when accessing $pageIds from Low to Unbreak Now!.

train blockers are UBNs

Sep 23 2020, 3:31 PM · Patch-For-Review, Platform Team Workboards (Clinic Duty Team), MW-1.36-notes (1.36.0-wmf.11; 2020-09-29), GlobalUsage, Release-Engineering-Team (Logspam), Wikimedia-production-error

Sep 22 2020

thcipriani assigned T263184: 1.36.0-wmf.18 deployment blockers to hashar.
Sep 22 2020, 8:24 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263183: 1.36.0-wmf.17 deployment blockers to dancy.
Sep 22 2020, 8:23 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263182: 1.36.0-wmf.16 deployment blockers to brennen.
Sep 22 2020, 8:22 PM · User-brennen, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263181: 1.36.0-wmf.15 deployment blockers to LarsWirzenius.
Sep 22 2020, 8:21 PM · User-brennen, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263180: 1.36.0-wmf.14 deployment blockers to jeena.
Sep 22 2020, 8:21 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263179: 1.36.0-wmf.13 deployment blockers to dduvall.
Sep 22 2020, 8:20 PM · Patch-For-Review, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani assigned T263178: 1.36.0-wmf.12 deployment blockers to hashar.
Sep 22 2020, 8:19 PM · Patch-For-Review, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani moved T263177: 1.36.0-wmf.11 deployment blockers from INBOX to Deployment on the Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)) board.
Sep 22 2020, 8:18 PM · Patch-For-Review, Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release, Train Deployments
thcipriani assigned T263177: 1.36.0-wmf.11 deployment blockers to mmodell.
Sep 22 2020, 8:17 PM · Patch-For-Review, Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release, Train Deployments

Sep 21 2020

thcipriani added a comment to T262867: Story idea for Blog: GitLab consultation.
Sep 21 2020, 11:12 PM · Technical-blog-posts
thcipriani awarded T262867: Story idea for Blog: GitLab consultation a Party Time token.
Sep 21 2020, 11:12 PM · Technical-blog-posts
thcipriani added a comment to T262867: Story idea for Blog: GitLab consultation.

One quick thing:

Sep 21 2020, 10:00 PM · Technical-blog-posts
thcipriani edited projects for T263295: Setup Git repo and CI for shellbox, added: Continuous-Integration-Config, Release-Engineering-Team (CI & Testing services), Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)); removed Release-Engineering-Team.
Sep 21 2020, 3:18 PM · Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, Platform Team Workboards (Clinic Duty Team), Patch-For-Review

Sep 19 2020

thcipriani lowered the priority of T263132: Could not enqueue jobs from stream mediawiki.job.cirrusSearchIncomingLinkCount from Unbreak Now! to High.

After a few rounds of spikes, messages seem to have subsided, keeping an eye on logs for the time being.

Sep 19 2020, 12:00 AM · Analytics-Kanban, Operations, Analytics, Event-Platform, Wikimedia-production-error

Sep 18 2020

thcipriani raised the priority of T263132: Could not enqueue jobs from stream mediawiki.job.cirrusSearchIncomingLinkCount from High to Unbreak Now!.

Various jobenqueue errors happened today in the past 6 hours with spikes of over 10k. I didn't see any k8s deployments correlated with them.

Sep 18 2020, 10:34 PM · Analytics-Kanban, Operations, Analytics, Event-Platform, Wikimedia-production-error
thcipriani added a comment to T262867: Story idea for Blog: GitLab consultation.

Couple of photos appeal to me:

Sep 18 2020, 5:26 PM · Technical-blog-posts
thcipriani added a comment to T262867: Story idea for Blog: GitLab consultation.

@thcipriani do you have any thoughts about an image you would like to include for the featured image?

Here is some info about picking an image: https://www.mediawiki.org/wiki/Wikimedia_technical_blog_editorial_guidelines#Images_used_in_your_post

Sep 18 2020, 5:09 PM · Technical-blog-posts

Sep 17 2020

thcipriani added a comment to T262516: Unable to push repository to remote, hostname is inaccessible.
~
(/^ヮ^)/*:・゚✧ /usr/bin/ssh -T git@gitlab-test.wmcloud.org
Host key fingerprint is SHA256:ivD4/XKLYZyNxua0clumaPUosFXGEeBD+CRZvYwI1Hc
+---[ECDSA 256]---+
|...++o..         |
|. ++..oE         |
| . =++.o         |
|  . o.*          |
|  .  o  S        |
|  .+.+.=         |
|  .+o.%o+        |
|  ..+O+Bo        |
|   .o+B=o.       |
+----[SHA256]-----+
Welcome to GitLab, @thcipriani!
Sep 17 2020, 8:46 PM · Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)), Release-Engineering-Team (CI & Testing services), User-brennen, GitLab-Test
thcipriani created T263186: 1.36.0-wmf.20 deployment blockers.
Sep 17 2020, 7:30 PM · Release, Train Deployments
thcipriani created T263185: 1.36.0-wmf.19 deployment blockers.
Sep 17 2020, 7:30 PM · Release, Train Deployments
thcipriani created T263184: 1.36.0-wmf.18 deployment blockers.
Sep 17 2020, 7:30 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani created T263183: 1.36.0-wmf.17 deployment blockers.
Sep 17 2020, 7:30 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani created T263182: 1.36.0-wmf.16 deployment blockers.
Sep 17 2020, 7:30 PM · User-brennen, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani created T263181: 1.36.0-wmf.15 deployment blockers.
Sep 17 2020, 7:30 PM · User-brennen, Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani created T263180: 1.36.0-wmf.14 deployment blockers.
Sep 17 2020, 7:30 PM · Release-Engineering-Team-TODO, Release, Train Deployments
thcipriani created T263179: 1.36.0-wmf.13 deployment blockers.
Sep 17 2020, 7:30 PM · Patch-For-Review, Release-Engineering-Team-TODO, Release, Train Deployments