Page MenuHomePhabricator
Feed Advanced Search

Wed, Apr 24

dduvall committed rGBLBRa260ee40f28a: experimental: Support buildkit (authored by dduvall).
experimental: Support buildkit
Wed, Apr 24, 1:33 PM

Fri, Apr 19

dduvall committed rGBLBR5853dfac8258: experimental: Support buildkit (authored by dduvall).
experimental: Support buildkit
Fri, Apr 19, 11:05 PM
dduvall committed rGBLBRbbf4e93072f4: experimental: Support buildkit (authored by dduvall).
experimental: Support buildkit
Fri, Apr 19, 11:05 PM

Wed, Apr 17

dduvall committed rGBLBR11dab2e54a22: experimental: Support LLB output format (authored by dduvall).
experimental: Support LLB output format
Wed, Apr 17, 7:01 PM

Tue, Apr 9

dduvall edited P8379 analyze-thread-dumps.sh.
Tue, Apr 9, 9:47 PM

Mon, Apr 8

dduvall closed T206678: 1.33.0-wmf.24 deployment blockers as Resolved.
Mon, Apr 8, 8:00 PM · Release-Engineering-Team (Kanban), Release, Train Deployments
dduvall closed T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons as Resolved.
Mon, Apr 8, 8:00 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall closed T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons, a subtask of T206678: 1.33.0-wmf.24 deployment blockers, as Resolved.
Mon, Apr 8, 8:00 PM · Release-Engineering-Team (Kanban), Release, Train Deployments

Thu, Apr 4

dduvall added a comment to T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons.
{
  "_index": "logstash-2019.04.04",
  "_type": "mediawiki",
  "_id": "AWnqCdqV8aQffZ3HyCjK",
  "_version": 1,
  "_score": null,
  "_source": {
    "exception": {
      "trace": "#0 /srv/mediawiki/php-1.33.0-wmf.24/extensions/EventBus/includes/JobExecutor.php(289): Wikimedia\\Rdbms\\LBFactory->commitMasterChanges(string, array)\n#1 /srv/mediawiki/php-1.33.0-wmf.24/extensions/EventBus/includes/JobExecutor.php(67): JobExecutor->commitMasterChanges(Wikimedia\\Rdbms\\LBFactoryMulti, string)\n#2 /srv/mediawiki/rpc/RunSingleJob.php(77): JobExecutor->execute(array)\n#3 {main}",
      "code": 0,
      "file": "/srv/mediawiki/php-1.33.0-wmf.24/includes/libs/rdbms/lbfactory/LBFactory.php:261",
      "message": "RefreshLinksJob::run: transaction round 'RefreshLinksJob::runForTitle' still running.",
      "class": "Wikimedia\\Rdbms\\DBTransactionError"
    },
    "server": "jobrunner.discovery.wmnet",
    "phpversion": "5.6.99-hhvm",
    "wiki": "enwiki",
    "channel": "exception",
    "exception_id": "XKW0twpAAEUAAFjwWmIAAAAA",
    "program": "mediawiki",
    "type": "mediawiki",
    "message_checksum": "40dee3e70084b7a6f50133fd69b72f74",
    "caught_by": "mwe_handler",
    "exception_url": "/rpc/RunSingleJob.php",
    "http_method": "POST",
    "host": "mw1302",
    "@version": 1,
    "shard": "s1",
    "timestamp": "2019-04-04T20:28:27+00:00",
    "severity": "err",
    "level": "ERROR",
    "ip": "10.64.16.67",
    "mwversion": "1.33.0-wmf.24",
    "logsource": "mw1302",
    "message": "[XKW0twpAAEUAAFjwWmIAAAAA] /rpc/RunSingleJob.php   Wikimedia\\Rdbms\\DBTransactionError from line 261 of /srv/mediawiki/php-1.33.0-wmf.24/includes/libs/rdbms/lbfactory/LBFactory.php: RefreshLinksJob::run: transaction round 'RefreshLinksJob::runForTitle' still running.",
    "normalized_message": "[{exception_id}] {exception_url}   Wikimedia\\Rdbms\\DBTransactionError from line 261 of /srv/mediawiki/php-1.33.0-wmf.24/includes/libs/rdbms/lbfactory/LBFactory.php: RefreshLinksJob::run: transaction round 'RefreshLinksJob::runForTitle' still running.",
    "url": "/rpc/RunSingleJob.php",
    "reqId": "XKW0twpAAEUAAFjwWmIAAAAA",
    "tags": [
      "input-kafka-rsyslog-udp-localhost",
      "rsyslog-udp-localhost",
      "kafka",
      "es"
    ],
    "referrer": null,
    "@timestamp": "2019-04-04T20:28:27.000Z",
    "facility": "user"
  },
  "fields": {
    "@timestamp": [
      1554409707000
    ]
  },
  "sort": [
    1554409707000
  ]
}
Thu, Apr 4, 8:29 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall added a comment to T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons.

Following syncs and promotion to all wikis, the error rate is much lower than the previously seen spike, but it's still rather high, average of 800/min.

Thu, Apr 4, 8:28 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus

Wed, Apr 3

dduvall edited projects for T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons, added: EventBus, Core Platform Team; removed MediaWiki-JobQueue.
Wed, Apr 3, 8:17 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall renamed T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons from Spike in DBTransactionError following 1.33.0-wmf.24 group1 promotion to RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons.
Wed, Apr 3, 8:16 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall triaged T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons as Unbreak Now! priority.
Wed, Apr 3, 8:07 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall created T220037: RefreshLinksJob::runForTitle: transaction round 'RefreshLinksJob::run' already started on commons.
Wed, Apr 3, 8:06 PM · Core Platform Team Kanban (Done with CPT), Services (done), MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Core Platform Team (Security, stability, performance and scalability (TEC1)), Analytics, EventBus
dduvall added a comment to T219510: Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if configured.

@Mvolz I'm currently holding the group1 train deployment. Do you have an update or ETA on a fix?

I'm not 100% it's worth holding the train over - it'll disable citoid on at least one small language wiki. But the fix isn't done yet, no. Alternatively we can just revert it but note it requires reverting the change in the cite extension as well.

Wed, Apr 3, 7:42 PM · MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), VisualEditor (Current work), Patch-For-Review, Citoid
dduvall added a comment to T219510: Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if configured.

@Mvolz I'm currently holding the group1 train deployment. Do you have an update or ETA on a fix?

Wed, Apr 3, 7:28 PM · MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), VisualEditor (Current work), Patch-For-Review, Citoid
dduvall triaged T219510: Citoid should only usurp "<ref" / ctrl+shift+k shortcuts if configured as Unbreak Now! priority.

Marking UBN as is our policy with train blockers.

Wed, Apr 3, 7:03 PM · MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), VisualEditor (Current work), Patch-For-Review, Citoid

Tue, Apr 2

dduvall assigned T219738: PHP Warning: Array key should be either a string or an integer to Pchelolo.

The https://gerrit.wikimedia.org/r/500363 fixes it. Don't want to self-merge my own patch though.

Tue, Apr 2, 6:10 PM · Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Kanban (Done with CPT), MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Beta-Cluster-reproducible, Analytics, EventBus, Wikimedia-production-error
dduvall triaged T219738: PHP Warning: Array key should be either a string or an integer as Unbreak Now! priority.

Marking this UBN as is policy for all deployment blockers.

Tue, Apr 2, 5:46 PM · Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Kanban (Done with CPT), MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Beta-Cluster-reproducible, Analytics, EventBus, Wikimedia-production-error
dduvall raised the priority of T217087: Error "A non well formed numeric value encountered" (from ImageMap) from Normal to Unbreak Now!.

Thanks for catching this prior to deployment. Marking UBN as is policy with all deployment blockers.

Tue, Apr 2, 5:44 PM · MW-1.34-notes (1.34.0-wmf.1; 2019-04-16), MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), User-notice, User-Ryasmeen, Editing-team, VisualEditor, PHP 7.2 support, ImageMap, Wikimedia-production-error

Mon, Apr 1

dduvall added a comment to T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish.

Nominally fixed -> not UBN.

Mon, Apr 1, 4:45 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), Patch-For-Review, MediaWiki-Logging, Wikimedia-production-error
dduvall removed a parent task for T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish: T206677: 1.33.0-wmf.23 deployment blockers.
Mon, Apr 1, 4:44 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), Patch-For-Review, MediaWiki-Logging, Wikimedia-production-error
dduvall removed a subtask for T206677: 1.33.0-wmf.23 deployment blockers: T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish.
Mon, Apr 1, 4:44 PM · Release-Engineering-Team (Kanban), Release, Train Deployments

Thu, Mar 28

dduvall closed T206677: 1.33.0-wmf.23 deployment blockers as Resolved.
Thu, Mar 28, 7:42 PM · Release-Engineering-Team (Kanban), Release, Train Deployments

Mar 26 2019

dduvall added a comment to T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish.

@pmiazga And more importantly: thanks for the fix! :)

Mar 26 2019, 5:24 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), Patch-For-Review, MediaWiki-Logging, Wikimedia-production-error
dduvall added a comment to T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish.

@dduvall we're very close to merge. Can you give us couple more minutes, please?

Mar 26 2019, 5:20 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), Patch-For-Review, MediaWiki-Logging, Wikimedia-production-error
dduvall added a comment to T218940: Exception "At least one of: RCID, revision ID, and log ID MUST be specified" from ManualLogEntry::publish.

FYI I'll be starting the 1.33.0-wmf.23 branch cut shortly. If https://gerrit.wikimedia.org/r/c/mediawiki/core/+/498809 doesn't make it in I will plan on backporting it prior to actual deployment.

Mar 26 2019, 5:06 PM · MW-1.33-notes (1.33.0-wmf.25; 2019-04-09), Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q4), Patch-For-Review, MediaWiki-Logging, Wikimedia-production-error

Mar 22 2019

dduvall renamed T218827: Evaluate Argo from Evaluate Argo Workflow to Evaluate Argo.
Mar 22 2019, 8:07 PM · Release-Engineering-Team (Kanban)
dduvall closed T218827: Evaluate Argo as Resolved.

First, some clarification about the various Argo projects.

Mar 22 2019, 7:05 PM · Release-Engineering-Team (Kanban)
dduvall closed T218827: Evaluate Argo, a subtask of T217325: Consider and evaluate possible new CI tooling, as Resolved.
Mar 22 2019, 7:05 PM · User-zeljkofilipin, Release-Engineering-Team (Kanban)
dduvall renamed T218827: Evaluate Argo from Evaluate Argo CD to Evaluate Argo Workflow.
Mar 22 2019, 5:51 PM · Release-Engineering-Team (Kanban)

Mar 21 2019

dduvall closed T217912: Evaluate Tekton as Resolved.
Mar 21 2019, 7:25 PM · Release-Engineering-Team (Kanban)
dduvall closed T217912: Evaluate Tekton, a subtask of T217325: Consider and evaluate possible new CI tooling, as Resolved.
Mar 21 2019, 7:25 PM · User-zeljkofilipin, Release-Engineering-Team (Kanban)
dduvall added a comment to T217912: Evaluate Tekton.

Tekton is narrow in scope but it seems to do what it does well: It provides a coherent set of Custom Resource Definitions (CRD) necessary to get CI type workloads running on k8s efficiently and quickly. Its narrowness in scope and CRD nature yield these benefits and drawbacks:

Mar 21 2019, 7:10 PM · Release-Engineering-Team (Kanban)
dduvall moved T218827: Evaluate Argo from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
Mar 21 2019, 6:12 PM · Release-Engineering-Team (Kanban)
dduvall claimed T218827: Evaluate Argo.
Mar 21 2019, 6:12 PM · Release-Engineering-Team (Kanban)

Mar 20 2019

dduvall created T218827: Evaluate Argo.
Mar 20 2019, 9:31 PM · Release-Engineering-Team (Kanban)
dduvall closed T218334: Evaluate Jenkins X as Resolved.

I followed the Getting Started section of the Jenkins X documentation to get it installed locally using minikube.

Mar 20 2019, 4:08 PM · Jenkins, Release-Engineering-Team (Kanban)
dduvall closed T218334: Evaluate Jenkins X, a subtask of T217325: Consider and evaluate possible new CI tooling, as Resolved.
Mar 20 2019, 4:08 PM · User-zeljkofilipin, Release-Engineering-Team (Kanban)

Mar 19 2019

dduvall edited P8237 Blubber built on Tekton + minikube.
Mar 19 2019, 10:30 PM

Mar 15 2019

dduvall added a comment to T205911: Track and install additional npm packages for all service container images.

Or: Isn't packages-lock.json supposed to be used for this kind of thing? Is that not the official way to declare an authoritative list of dependencies that a Node project requires in a production environment? (That thought is informed by a Ruby background, however, as Gemfile.lock is such a solution in that world.)

Mar 15 2019, 9:32 PM · Core Platform Team Kanban (Done with CPT), Services (done), Release-Engineering-Team (Watching / External), Operations, Release Pipeline
dduvall added a comment to T205911: Track and install additional npm packages for all service container images.

Having services depend on a meta package that itself depends on the real package is a recipe for deps hell, and I'd really like to avoid that. Not only does it create more manual work, but it also adds complexity that is delegated to humans, and, as we know, PEBKAC is the number #1 reason things go wrong.

Mar 15 2019, 8:01 PM · Core Platform Team Kanban (Done with CPT), Services (done), Release-Engineering-Team (Watching / External), Operations, Release Pipeline

Mar 14 2019

dduvall added a comment to T205911: Track and install additional npm packages for all service container images.

I'm pushing back on the patchset to Blubber for a couple of reasons:

Mar 14 2019, 10:25 PM · Core Platform Team Kanban (Done with CPT), Services (done), Release-Engineering-Team (Watching / External), Operations, Release Pipeline

Mar 8 2019

dduvall created T217912: Evaluate Tekton.
Mar 8 2019, 5:58 PM · Release-Engineering-Team (Kanban)

Mar 5 2019

dduvall committed rGBLBRec0af37c1e41: Include a basic go.mod that declares the package name (authored by dduvall).
Include a basic go.mod that declares the package name
Mar 5 2019, 7:37 PM
dduvall committed rGBLBR71f4eb2581e6: Include a basic go.mod that declares the package name (authored by dduvall).
Include a basic go.mod that declares the package name
Mar 5 2019, 7:16 PM

Feb 15 2019

dduvall updated subscribers of T209106: Setup session storage service testing/continuous integration.

Just wanted to offer some feedback based on the patches you already have and some of the things @Eevans, @thcipriani and I talked about during All Hands.

Feb 15 2019, 11:43 PM · Patch-For-Review, User-Clarakosi, Core Platform Team Backlog (Next), Core Platform Team (Session Management Service (CDP2)), User-Eevans
dduvall added a comment to T210267: The continuous release pipeline should support more than one service per repo.

I went a little crazy with a new config proposal in anticipation of us implementing T216272: The pipeline should provide a way to save artifacts from a stage. It's more loosely coupled, like what @thcipriani proposed earlier, with some extra fields for clearly defining the way in which stages should be executed and different methods for publishing artifacts. We'd likely want some basic policy/validation that enforces sanity (e.g. if it's publishing an image in a stage, it must also specify testDeploy, etc.). Useful defaults would also be important to cut down on configuration duplication.

Feb 15 2019, 8:51 PM · Patch-For-Review, Release Pipeline, Release-Engineering-Team (Backlog), Operations, ORES, Scoring-platform-team
dduvall edited P8093 .pipeline/config.yaml.
Feb 15 2019, 8:35 PM
dduvall edited P8093 .pipeline/config.yaml.
Feb 15 2019, 8:14 PM
dduvall closed T216204: contint1001 and contint2001 should allow port 9418 from their ipv6 addresses as Resolved.

Deployed and tested.

Feb 15 2019, 12:24 AM · Patch-For-Review, Release-Engineering-Team (Kanban), Continuous-Integration-Infrastructure

Feb 14 2019

dduvall moved T216204: contint1001 and contint2001 should allow port 9418 from their ipv6 addresses from Untriaged to In-progress on the Continuous-Integration-Infrastructure board.
Feb 14 2019, 11:31 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Continuous-Integration-Infrastructure
dduvall claimed T216204: contint1001 and contint2001 should allow port 9418 from their ipv6 addresses.
Feb 14 2019, 11:31 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Continuous-Integration-Infrastructure
dduvall created T216204: contint1001 and contint2001 should allow port 9418 from their ipv6 addresses.
Feb 14 2019, 11:20 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Continuous-Integration-Infrastructure

Feb 13 2019

dduvall claimed T177867: Pipeline image build cleanup.
Feb 13 2019, 6:58 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline

Feb 7 2019

dduvall renamed T212247: Refactor integration/pipelinelib to use blubberoid.wikimedia.org from Refactor integration/pipelinelib to use blubberoid.discovery.wmnet to Refactor integration/pipelinelib to use blubberoid.wikimedia.org.
Feb 7 2019, 11:48 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline
dduvall committed rMSCAa29a6c6f43cd: Rename and simplify some git deploy functions (authored by dduvall).
Rename and simplify some git deploy functions
Feb 7 2019, 12:09 PM
dduvall committed rMSCAd8bcae58233f: Execute distinct stages of deployment separately (authored by dduvall).
Execute distinct stages of deployment separately
Feb 7 2019, 12:09 PM
dduvall committed rMSCA4db5964f67fa: Execute distinct stages of deployment separately (authored by dduvall).
Execute distinct stages of deployment separately
Feb 7 2019, 12:09 PM
dduvall committed rMSCA2f3a15eae70a: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:09 PM
dduvall committed rMSCA32b482a51de5: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:09 PM
dduvall committed rMSCAd2b1d69d41a1: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:09 PM
dduvall committed rMSCA43e39fd7ae9a: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:09 PM
dduvall committed rMSCAe6a551829112: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:08 PM
dduvall committed rMSCA813daa3f9193: Support atomic promotion and rollback (authored by dduvall).
Support atomic promotion and rollback
Feb 7 2019, 12:08 PM
dduvall committed rMSCAb8ab63f8003c: Filter target host logging from stdout of main process (authored by dduvall).
Filter target host logging from stdout of main process
Feb 7 2019, 12:08 PM

Jan 28 2019

dduvall committed rGBLBR764c402034b6: Comment about copies config expansion inefficiency (authored by dduvall).
Comment about copies config expansion inefficiency
Jan 28 2019, 11:43 PM
dduvall committed rGBLBRfd20d346ddfe: Unify `copies` and `artifacts` configuration (authored by dduvall).
Unify `copies` and `artifacts` configuration
Jan 28 2019, 11:26 PM
dduvall committed rGBLBR20e9b739e56f: Bump version number (authored by thcipriani).
Bump version number
Jan 28 2019, 11:08 PM

Jan 19 2019

dduvall committed rGBLBRd10e4b3e00c6: Unify `copies` and `artifacts` configuration (authored by dduvall).
Unify `copies` and `artifacts` configuration
Jan 19 2019, 12:01 AM
dduvall committed rGBLBR6cc05320b12f: Unify `copies` and `artifacts` configuration (authored by dduvall).
Unify `copies` and `artifacts` configuration
Jan 19 2019, 12:01 AM
dduvall committed rGBLBR341794300c01: Unify `copies` and `artifacts` configuration (authored by dduvall).
Unify `copies` and `artifacts` configuration
Jan 19 2019, 12:01 AM

Jan 18 2019

dduvall renamed T211625: Unify configuration for local build-context copies and variant artifacts from Manually defining artifacts results in default copy of all project files to Unify configuration for local build-context copies and variant artifacts.
Jan 18 2019, 10:46 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline (Blubber)

Jan 17 2019

dduvall closed T206667: 1.33.0-wmf.13 deployment blockers as Resolved.
Jan 17 2019, 8:28 PM · Release-Engineering-Team (Kanban), Release, Train Deployments
dduvall awarded Blog Post: Gerrit now automatically adds reviewers a 100 token.
Jan 17 2019, 5:58 PM · Release-Engineering-Team, Gerrit

Jan 16 2019

dduvall added a comment to T204871: Investigate the spikes of "web request took longer than 60 seconds and timed out" during deployments.

Just a note that for today's promotion of group1 to 1.33.0-wmf.13 (T206667), I segmented the group1 error-log dashboard to have a view of just these timeout errors and a view excluding them. It was very helpful in keeping on both the rise in timeouts and side effects or unrelated errors. I plan on saving the dashboards and adding links in the train docs.

Jan 16 2019, 11:36 PM · Release-Engineering-Team (Kanban), User-zeljkofilipin, Wikimedia-Incident, Wikimedia-production-error
dduvall awarded D1138: pipeline: test scap with CD pipeline a Love token.
Jan 16 2019, 8:48 PM · Release-Engineering-Team

Jan 15 2019

dduvall claimed T206667: 1.33.0-wmf.13 deployment blockers.

Cutting the branch.

Jan 15 2019, 6:04 PM · Release-Engineering-Team (Kanban), Release, Train Deployments

Jan 12 2019

dduvall added a comment to T210267: The continuous release pipeline should support more than one service per repo.

What new problems does this create?

  • Potential that one merge can use a lot of CI executors (also currently the case, but RelEng has some ability to mitigate currently)
  • Others?
Jan 12 2019, 1:09 AM · Patch-For-Review, Release Pipeline, Release-Engineering-Team (Backlog), Operations, ORES, Scoring-platform-team

Jan 10 2019

dduvall closed T206666: 1.33.0-wmf.12 deployment blockers as Resolved.

Today's all-wiki deployment went much like yesterday's: We saw an increase in the MediaWiki error rate due to a flood of "timed out" errors and what seemed like a side effect of 500s caused by nginx/apache/hhvm socket timeouts. The rate increases lasted from approximately 20:11 until 20:34 UTC at which point they subsided to pre-deployment levels.

Jan 10 2019, 8:59 PM · Release, Train Deployments
dduvall moved T211625: Unify configuration for local build-context copies and variant artifacts from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
Jan 10 2019, 7:36 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall added a watcher for Release-Engineering-Team (Kanban): dduvall.
Jan 10 2019, 7:35 PM

Jan 9 2019

dduvall committed rGBLBR867a9943d94e: Tweaked logo to play nice with preview renderer on commons (authored by dduvall).
Tweaked logo to play nice with preview renderer on commons
Jan 9 2019, 11:32 PM

Jan 8 2019

dduvall added a comment to T212427: No namespace configured for entity type `form`.

@greg @dduvall train is unblocked now, verified on beta.

Jan 8 2019, 6:34 PM · MW-1.33-notes (1.33.0-wmf.12; 2019-01-08), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikimedia-production-error, Wikidata

Dec 20 2018

dduvall committed rGBLBR1ac4655cefaf: Provide a new Blubber logo (authored by dduvall).
Provide a new Blubber logo
Dec 20 2018, 9:25 PM
dduvall committed rGBLBR013ab435a354: Provide a new Blubber logo (authored by dduvall).
Provide a new Blubber logo
Dec 20 2018, 9:25 PM

Dec 19 2018

dduvall added a comment to T212251: Expose blubberoid to the public allowing CI in WMCS to be able to reach out as well to it.

As SREs I don't think we would like to expose a public service that is not going to receive any traffic any time soon, mostly for operational and complexity reasons. I guess we don't really have that need yet, do we?

But that is exactly the point of this task? Open the service to the outside world so we can start using it. Instances on WMCS can not reach production internal network (10.0.0.0/8), they get out via PAT/NAT and are considered to be just like any internet traffic (== untrusted).

Is it? Cause I gather that it is specifically for CI. In which case an instance in CI is architecturally a better construct.

Dec 19 2018, 6:36 PM · Patch-For-Review, serviceops, Release-Engineering-Team (Kanban), Release Pipeline

Dec 18 2018

dduvall updated the task description for T212251: Expose blubberoid to the public allowing CI in WMCS to be able to reach out as well to it.
Dec 18 2018, 8:48 PM · Patch-For-Review, serviceops, Release-Engineering-Team (Kanban), Release Pipeline
dduvall moved T212251: Expose blubberoid to the public allowing CI in WMCS to be able to reach out as well to it from Backlog to Blocked (externally) on the Release-Engineering-Team (Kanban) board.
Dec 18 2018, 8:44 PM · Patch-For-Review, serviceops, Release-Engineering-Team (Kanban), Release Pipeline
dduvall moved T212247: Refactor integration/pipelinelib to use blubberoid.wikimedia.org from Blocked (externally) to In-progress on the Release-Engineering-Team (Kanban) board.
Dec 18 2018, 8:44 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline
dduvall moved T212247: Refactor integration/pipelinelib to use blubberoid.wikimedia.org from Backlog to Blocked (externally) on the Release-Engineering-Team (Kanban) board.
Dec 18 2018, 8:44 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline
dduvall triaged T212247: Refactor integration/pipelinelib to use blubberoid.wikimedia.org as Normal priority.
Dec 18 2018, 8:43 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline
dduvall created T212251: Expose blubberoid to the public allowing CI in WMCS to be able to reach out as well to it.
Dec 18 2018, 8:43 PM · Patch-For-Review, serviceops, Release-Engineering-Team (Kanban), Release Pipeline
dduvall created T212247: Refactor integration/pipelinelib to use blubberoid.wikimedia.org.
Dec 18 2018, 8:24 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Release Pipeline
dduvall updated images of M271: Blubber(oid) logos.
Dec 18 2018, 8:05 PM · Release Pipeline (Blubber), Release-Engineering-Team
dduvall updated images of M271: Blubber(oid) logos.
Dec 18 2018, 8:03 PM · Release Pipeline (Blubber), Release-Engineering-Team

Dec 17 2018

dduvall updated images of M271: Blubber(oid) logos.
Dec 17 2018, 6:43 PM · Release Pipeline (Blubber), Release-Engineering-Team
dduvall created M271: Blubber(oid) logos.
Dec 17 2018, 6:39 PM · Release Pipeline (Blubber), Release-Engineering-Team

Dec 15 2018

dduvall committed rGBLBRdb27311386af: WIP Playing with new logos (authored by dduvall).
WIP Playing with new logos
Dec 15 2018, 12:51 AM

Dec 14 2018

dduvall committed rGBLBR752aaa2bab4e: WIP Playing with new logos (authored by dduvall).
WIP Playing with new logos
Dec 14 2018, 11:51 PM