dduvall (Dan Duvall)
Automation Engineer

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Oct 7 2014, 4:24 PM (141 w, 2 d)
Availability
Available
IRC Nick
marxarelli
LDAP User
Dduvall
MediaWiki User
DDuvall (WMF)

Recent Activity

Today

dduvall updated the diff for D691: Support environment variables.

Added task reference.

Thu, Jun 22, 9:44 PM · Release-Engineering-Team
dduvall added a revision to T168425: Support environment variables in configuration: D691: Support environment variables.
Thu, Jun 22, 9:44 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall created D691: Support environment variables.
Thu, Jun 22, 9:38 PM · Release-Engineering-Team

Tue, Jun 20

dduvall committed rGBLBR77b95b1f94de: Set HOME environment variable for runs-as user (authored by dduvall).
Set HOME environment variable for runs-as user
Tue, Jun 20, 5:20 PM
dduvall closed D686: Set HOME environment variable for runs-as user by committing rGBLBR77b95b1f94de: Set HOME environment variable for runs-as user.
Tue, Jun 20, 5:20 PM · Release-Engineering-Team
dduvall committed rGBLBRbbb5eba26ae3: Use correct COPY syntax for quoted paths (authored by dduvall).
Use correct COPY syntax for quoted paths
Tue, Jun 20, 5:20 PM
dduvall closed D685: Use correct COPY syntax for quoted paths by committing rGBLBRbbb5eba26ae3: Use correct COPY syntax for quoted paths.
Tue, Jun 20, 5:19 PM · Release-Engineering-Team
dduvall moved T168425: Support environment variables in configuration from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
Tue, Jun 20, 5:15 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall lowered the priority of T168001: Improve Blubber unit test coverage from High to Normal.
Tue, Jun 20, 5:13 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall moved T168425: Support environment variables in configuration from Backlog to Doing on the Release Pipeline (Blubber) board.
Tue, Jun 20, 5:13 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall triaged T168425: Support environment variables in configuration as Normal priority.
Tue, Jun 20, 5:13 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall created T168425: Support environment variables in configuration.
Tue, Jun 20, 5:12 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)

Fri, Jun 16

dduvall created D686: Set HOME environment variable for runs-as user.
Fri, Jun 16, 12:41 AM · Release-Engineering-Team

Thu, Jun 15

dduvall updated the diff for D685: Use correct COPY syntax for quoted paths.

Fixed main source COPY instruction as well

Thu, Jun 15, 11:27 PM · Release-Engineering-Team
dduvall created D685: Use correct COPY syntax for quoted paths.
Thu, Jun 15, 11:11 PM · Release-Engineering-Team
dduvall committed rGBLBR58c8e9718c97: Support single-stage builds for older Docker versions (authored by dduvall).
Support single-stage builds for older Docker versions
Thu, Jun 15, 9:51 PM
dduvall closed D684: Support single-stage builds for older Docker versions by committing rGBLBR58c8e9718c97: Support single-stage builds for older Docker versions.
Thu, Jun 15, 9:51 PM · Release-Engineering-Team
dduvall moved T168001: Improve Blubber unit test coverage from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
Thu, Jun 15, 9:25 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall claimed T168001: Improve Blubber unit test coverage.
Thu, Jun 15, 6:34 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall moved T168001: Improve Blubber unit test coverage from Backlog to Doing on the Release Pipeline (Blubber) board.
Thu, Jun 15, 6:34 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall moved T167999: Escape Blubber config values when compiling to Dockerfile from Backlog to Doing on the Release Pipeline (Blubber) board.
Thu, Jun 15, 6:33 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall triaged T168001: Improve Blubber unit test coverage as High priority.
Thu, Jun 15, 6:31 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall triaged T168000: Document current Blubber types and functions as Low priority.
Thu, Jun 15, 6:31 PM · Release Pipeline (Blubber)
dduvall triaged T167999: Escape Blubber config values when compiling to Dockerfile as Normal priority.
Thu, Jun 15, 6:31 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall created T168001: Improve Blubber unit test coverage.
Thu, Jun 15, 6:28 PM · Release-Engineering-Team (Kanban), Release Pipeline (Blubber)
dduvall created T168000: Document current Blubber types and functions.
Thu, Jun 15, 6:27 PM · Release Pipeline (Blubber)
dduvall created T167999: Escape Blubber config values when compiling to Dockerfile.
Thu, Jun 15, 6:25 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)

Wed, Jun 14

dduvall created D684: Support single-stage builds for older Docker versions.
Wed, Jun 14, 8:40 PM · Release-Engineering-Team

Mon, Jun 12

dduvall closed D680: Fix variant expansion for bool config fields by committing rGBLBRb6c3ad2aac52: Fix variant expansion for bool config fields.
Mon, Jun 12, 4:24 PM · Release-Engineering-Team
dduvall closed T166353: Fix Blubber variant expansion for boolean/int config properties as Resolved by committing rGBLBRb6c3ad2aac52: Fix variant expansion for bool config fields.
Mon, Jun 12, 4:24 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall committed rGBLBRb6c3ad2aac52: Fix variant expansion for bool config fields (authored by dduvall).
Fix variant expansion for bool config fields
Mon, Jun 12, 4:24 PM

Wed, Jun 7

dduvall updated the diff for D680: Fix variant expansion for bool config fields.

Added task to summary

Wed, Jun 7, 8:13 PM · Release-Engineering-Team
dduvall added a revision to T166353: Fix Blubber variant expansion for boolean/int config properties: D680: Fix variant expansion for bool config fields.
Wed, Jun 7, 8:13 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall created D680: Fix variant expansion for bool config fields.
Wed, Jun 7, 7:48 PM · Release-Engineering-Team

Thu, Jun 1

dduvall moved T166353: Fix Blubber variant expansion for boolean/int config properties from Backlog to Build PoC on the Release Pipeline board.
Thu, Jun 1, 5:17 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)

Wed, May 31

dduvall edited projects for T130037: Implement a smoke + parallel strategy for running end-to-end tests, added: Release Pipeline; removed Browser-Tests-Infrastructure.
Wed, May 31, 5:13 PM · Release Pipeline, Release-Engineering-Team (Backlog)
dduvall added a comment to T130037: Implement a smoke + parallel strategy for running end-to-end tests.

@dduvall is this task replaced by release pipeline?

Wed, May 31, 5:12 PM · Release Pipeline, Release-Engineering-Team (Backlog)

Tue, May 30

dduvall claimed T166353: Fix Blubber variant expansion for boolean/int config properties.
Tue, May 30, 4:27 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall moved T166353: Fix Blubber variant expansion for boolean/int config properties from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
Tue, May 30, 4:26 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall triaged T166353: Fix Blubber variant expansion for boolean/int config properties as Normal priority.
Tue, May 30, 4:26 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)

Thu, May 25

dduvall updated the task description for T166353: Fix Blubber variant expansion for boolean/int config properties.
Thu, May 25, 11:49 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)
dduvall created T166353: Fix Blubber variant expansion for boolean/int config properties.
Thu, May 25, 11:47 PM · Release Pipeline (Blubber), Release-Engineering-Team (Kanban)

May 23 2017

dduvall committed rGBLBRd0266214e0da: Use Phabricator import URIs (authored by dduvall).
Use Phabricator import URIs
May 23 2017, 9:14 PM
dduvall closed D657: Use Phabricator import URIs by committing rGBLBRd0266214e0da: Use Phabricator import URIs.
May 23 2017, 9:14 PM · Release-Engineering-Team
dduvall updated the diff for D657: Use Phabricator import URIs.

Fixed docker/compiler import paths

May 23 2017, 6:57 PM · Release-Engineering-Team
dduvall created D657: Use Phabricator import URIs.
May 23 2017, 6:19 PM · Release-Engineering-Team

May 11 2017

dduvall claimed T164964: Move blubber to phabricator repo.

Hmm... now to wrestle with import paths.

May 11 2017, 6:21 PM · Release-Engineering-Team (Kanban), Release Pipeline
dduvall updated subscribers of T164965: ci-staging registry base image for mathoid PoC.

@mobrovac and I discussed Blubber requirements today regarding local development use cases. Among other things, we discussed the need for a Node base image with Wikimedia APT sources included and nodejs and related packages installed.

May 11 2017, 5:41 PM · User-mobrovac, Release Pipeline

Apr 20 2017

dduvall added a comment to T158812: Gather requirements for build manifest specification.

Over the past few days I've been experimenting with a general build configuration format and wrote a build tool in Go—because, well, I unabashedly wanted to learn something new while experimenting; please don't judge my awful Go code :)—codenamed Blubber that spits out a Dockerfile.

Apr 20 2017, 6:39 PM · Release Pipeline (Blubber), User-Joe, Services (designing), User-mobrovac

Apr 19 2017

dduvall added a comment to P5294 output of go build && ./blubber blubber.example.json production.

Result of:

Apr 19 2017, 6:52 PM
dduvall updated the title for P5294 output of go build && ./blubber blubber.example.json production from Masterwork From Distant Lands to output of go build && ./blubber blubber.example.json production.
Apr 19 2017, 6:50 PM
dduvall edited P5294 output of go build && ./blubber blubber.example.json production.
Apr 19 2017, 6:48 PM

Apr 17 2017

dduvall created T163159: Archive Malu project and repo.
Apr 17 2017, 9:48 PM · User-zeljkofilipin, Malu

Mar 29 2017

dduvall added a comment to T161659: Port existing `differential-docker-test` job to Jenkinsfile/Groovy.

Looks like there's a Docker Pipeline plugin that might provide what we need for this.

Mar 29 2017, 10:28 PM · Release Pipeline
dduvall added a comment to T161657: Set up Docker registry in ci-staging.

There's now a ci-staging-docker-registry instance running in ci-staging with the new role::ci::docker::registry applied. I've verified that it's working by pulling down the latest Debian image from dockerhub and pushing it to ci-staging-docker-registry.ci-staging.eqiad.wmflabs:

Mar 29 2017, 9:08 PM · Patch-For-Review, Release Pipeline
dduvall lowered the priority of T158812: Gather requirements for build manifest specification from Normal to Low.
Mar 29 2017, 7:42 PM · Release Pipeline (Blubber), User-Joe, Services (designing), User-mobrovac
dduvall triaged T161661: Install Jenkins Pipeline Plugin for ci-staging and verify its basic functionality as Normal priority.
Mar 29 2017, 7:42 PM · Release Pipeline
dduvall moved T161661: Install Jenkins Pipeline Plugin for ci-staging and verify its basic functionality from Backlog to Build PoC on the Release Pipeline board.
Mar 29 2017, 7:42 PM · Release Pipeline

Mar 28 2017

dduvall added a parent task for T161659: Port existing `differential-docker-test` job to Jenkinsfile/Groovy: T161661: Install Jenkins Pipeline Plugin for ci-staging and verify its basic functionality.
Mar 28 2017, 11:08 PM · Release Pipeline
dduvall added a subtask for T161661: Install Jenkins Pipeline Plugin for ci-staging and verify its basic functionality: T161659: Port existing `differential-docker-test` job to Jenkinsfile/Groovy.
Mar 28 2017, 11:08 PM · Release Pipeline
dduvall created T161661: Install Jenkins Pipeline Plugin for ci-staging and verify its basic functionality.
Mar 28 2017, 11:08 PM · Release Pipeline
dduvall claimed T161657: Set up Docker registry in ci-staging.
Mar 28 2017, 11:02 PM · Patch-For-Review, Release Pipeline
dduvall moved T161657: Set up Docker registry in ci-staging from Backlog to Build PoC on the Release Pipeline board.
Mar 28 2017, 11:02 PM · Patch-For-Review, Release Pipeline
dduvall moved T159864: Build testing k8s setup in ci-staging from Build PoC to Backlog on the Release Pipeline board.
Mar 28 2017, 11:02 PM · Release-Engineering-Team (Kanban), Patch-For-Review, Release Pipeline
dduvall closed T153363: Spike: Evaluate containerized CI builds using Kubernetes as Declined.

This idea is essentially superseded by the Release Pipeline which will undoubtedly incorporate or interact with a k8s cluster in one way or another.

Mar 28 2017, 11:01 PM · Release Pipeline, Release-Engineering-Team, Continuous-Integration-Infrastructure
dduvall created T161659: Port existing `differential-docker-test` job to Jenkinsfile/Groovy.
Mar 28 2017, 10:57 PM · Release Pipeline
dduvall created T161657: Set up Docker registry in ci-staging.
Mar 28 2017, 10:47 PM · Patch-For-Review, Release Pipeline
dduvall added a comment to T159864: Build testing k8s setup in ci-staging.

Also needed add an annotation to the default namespace to satisfy our custom RegistryEnforcer admin controller:

Mar 28 2017, 6:49 PM · Release-Engineering-Team (Kanban), Patch-For-Review, Release Pipeline
dduvall added a comment to T159864: Build testing k8s setup in ci-staging.

So I think I got most of what was required for setup into the submitted puppet patch. The only other manual steps include:

Mar 28 2017, 6:43 PM · Release-Engineering-Team (Kanban), Patch-For-Review, Release Pipeline

Mar 16 2017

dduvall added a comment to T158812: Gather requirements for build manifest specification.

Also, it's probably worth pointing out at this point that 'manifest' is not really the right word for what we're talking about here; a manifest as it's defined in all of the existing specifications is actually a bundled filesystem image (agnostic to how it was built) and runtime metadata about how to set up the container and how to invoke some application binary. What we've been discussing is a slightly higher level abstraction that covers the metadata but also system level package dependencies. The manifest + image will be a product of our build step.

Mar 16 2017, 5:34 PM · Release Pipeline (Blubber), User-Joe, Services (designing), User-mobrovac
dduvall added a comment to T158812: Gather requirements for build manifest specification.

Some related reading material on container standards (both de facto and burgeoning) that illustrates their separation of concerns and overlap.

Mar 16 2017, 5:10 PM · Release Pipeline (Blubber), User-Joe, Services (designing), User-mobrovac
dduvall triaged T159864: Build testing k8s setup in ci-staging as Normal priority.
Mar 16 2017, 5:07 PM · Release-Engineering-Team (Kanban), Patch-For-Review, Release Pipeline
dduvall moved T159864: Build testing k8s setup in ci-staging from Backlog to Build PoC on the Release Pipeline board.
Mar 16 2017, 5:07 PM · Release-Engineering-Team (Kanban), Patch-For-Review, Release Pipeline

Feb 22 2017

dduvall created T158812: Gather requirements for build manifest specification.
Feb 22 2017, 9:40 PM · Release Pipeline (Blubber), User-Joe, Services (designing), User-mobrovac

Feb 16 2017

dduvall abandoned D418: Use USER environment variable for default ssh user.
Feb 16 2017, 10:21 PM · Release-Engineering-Team

Feb 14 2017

dduvall closed D558: Fix `failure_rate` percentages by committing rMSCA292d89e6da9a: Fix `failure_rate` percentages.
Feb 14 2017, 12:41 AM · Release-Engineering-Team
dduvall committed rMSCA292d89e6da9a: Fix `failure_rate` percentages (authored by dduvall).
Fix `failure_rate` percentages
Feb 14 2017, 12:41 AM
dduvall committed rMSCAcc58ae9f2891: Exclude empty deploy groups (authored by dduvall).
Exclude empty deploy groups
Feb 14 2017, 12:41 AM
dduvall closed T157136: Error after "Finished deploy": <ValueError> xrange() arg 3 must not be zero as Resolved by committing rMSCAcc58ae9f2891: Exclude empty deploy groups.
Feb 14 2017, 12:41 AM · Scoring-platform-team, ORES, Deployment-Systems, Release-Engineering-Team, scap2, Scap
dduvall closed D566: Exclude empty deploy groups by committing rMSCAcc58ae9f2891: Exclude empty deploy groups.
Feb 14 2017, 12:41 AM · Release-Engineering-Team

Feb 13 2017

dduvall added a comment to D561: Adds check to prevent xrange related crash. Fixes T157136.

@Halfak: The Deployment Cabal discussed this a bit more in our meeting this morning and decided that disallowing/excluding empty groups may indeed be a better long-term fix since empty input could lead to further unexpected behavior if left unvalidated.

Feb 13 2017, 9:30 PM · Release-Engineering-Team
dduvall created D566: Exclude empty deploy groups.
Feb 13 2017, 9:09 PM · Release-Engineering-Team
dduvall added a revision to T157136: Error after "Finished deploy": <ValueError> xrange() arg 3 must not be zero: D566: Exclude empty deploy groups.
Feb 13 2017, 9:09 PM · Scoring-platform-team, ORES, Deployment-Systems, Release-Engineering-Team, scap2, Scap

Feb 9 2017

dduvall resigned from D561: Adds check to prevent xrange related crash. Fixes T157136.
Feb 9 2017, 8:45 PM · Release-Engineering-Team
dduvall added a comment to D561: Adds check to prevent xrange related crash. Fixes T157136.
In D561#11106, @dduvall wrote:
In D561#11102, @demon wrote:
In D561#11085, @Halfak wrote:

Not sure if we'd prefer this solution or to use max(self.size, 1) and skip the conditional.

I think this will be better, because it means we'll be doing an extra useless stage that we've already done. I'd like @thcipriani's input here though.

Perhaps put that in the constructor. A non-zero group size may cause other issues down the line and it would be best to sanitize the input early.

Feb 9 2017, 8:23 PM · Release-Engineering-Team
dduvall added a comment to D561: Adds check to prevent xrange related crash. Fixes T157136.
In D561#11102, @demon wrote:
In D561#11085, @Halfak wrote:

Not sure if we'd prefer this solution or to use max(self.size, 1) and skip the conditional.

I think this will be better, because it means we'll be doing an extra useless stage that we've already done. I'd like @thcipriani's input here though.

Feb 9 2017, 8:04 PM · Release-Engineering-Team
dduvall added a task to D561: Adds check to prevent xrange related crash. Fixes T157136: T123: Turn on "diffusion.allow-http-auth".

Oh and one little nit about the commit message: Please use the "Refs T123" or "Fixes T123" auto-close syntax to reference tasks. See https://secure.phabricator.com/book/phabricator/article/diffusion_autoclose/

Feb 9 2017, 8:02 PM · Release-Engineering-Team
dduvall added a revision to T123: Turn on "diffusion.allow-http-auth": D561: Adds check to prevent xrange related crash. Fixes T157136.
Feb 9 2017, 8:02 PM · Gerrit-Migration, Wikimedia Phabricator RfC
dduvall requested changes to D561: Adds check to prevent xrange related crash. Fixes T157136.

To clarify: I think your changes are beneficial and should stay as well. :) Can you please add a unit test to tests/scap/test_targets.py to verify the edge case? Thanks!

Feb 9 2017, 7:54 PM · Release-Engineering-Team
dduvall added a comment to D561: Adds check to prevent xrange related crash. Fixes T157136.

I think the correct fix here would be to ensure a non-zero group size in the first place. Any idea how that may have happened? I suspect a percentage failure_rate that was converted to an integer and floored to 0 perhaps...

Feb 9 2017, 7:52 PM · Release-Engineering-Team

Feb 8 2017

dduvall created D558: Fix `failure_rate` percentages.
Feb 8 2017, 9:06 PM · Release-Engineering-Team
dduvall committed rMSCAbf6443a7ba1b: Fix regression of deploy group continue prompt (authored by dduvall).
Fix regression of deploy group continue prompt
Feb 8 2017, 8:41 PM
dduvall closed T156839: Saying yes (y) continues to all groups as Resolved by committing rMSCAbf6443a7ba1b: Fix regression of deploy group continue prompt.
Feb 8 2017, 8:41 PM · Scap, Parsoid
dduvall closed D555: Fix regression of deploy group continue prompt by committing rMSCAbf6443a7ba1b: Fix regression of deploy group continue prompt.
Feb 8 2017, 8:40 PM · Release-Engineering-Team
dduvall added a comment to D555: Fix regression of deploy group continue prompt.
  • I can't remember what we decided about failure percentage for groups where group_size is defined. In D490#10152 you said:
In D490#10152, @dduvall wrote:
  • Stage execution methods now considers the failure_limit for the entire original group, not individual subgroups

but that doesn't seem to be the case when you use a string like '50%' because of the line: https://github.com/wikimedia/scap/blob/master/scap/targets.py#L257

I may file a task for that 2nd one...

Feb 8 2017, 8:40 PM · Release-Engineering-Team

Feb 7 2017

dduvall updated the diff for D555: Fix regression of deploy group continue prompt.

An additional follow up to make the prompt conditional more clear

Feb 7 2017, 7:29 PM · Release-Engineering-Team
dduvall updated the diff for D555: Fix regression of deploy group continue prompt.

Implemented suggested re-prompting following bad prompt input

Feb 7 2017, 7:19 PM · Release-Engineering-Team
dduvall created D555: Fix regression of deploy group continue prompt.
Feb 7 2017, 2:23 AM · Release-Engineering-Team
dduvall added a revision to T156839: Saying yes (y) continues to all groups: D555: Fix regression of deploy group continue prompt.
Feb 7 2017, 2:23 AM · Scap, Parsoid

Feb 6 2017

dduvall added a comment to T156839: Saying yes (y) continues to all groups.

Looks like I caused the regression rMSCA98247477db5d: Improve failure handling and rollback behavior and called it out in the commit message as a "behavior change". :)

Feb 6 2017, 7:42 PM · Scap, Parsoid