Deploy the MachineVision extension to production
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Mholloway
	Jul 5 2019, 5:14 PM

Description

Task tracking requirements to deploying the new machine vision middleware extension to production.

https://www.mediawiki.org/wiki/Review_queue

At least three weeks prior to target deployment date, add to make-wmf-branch
Deploy to beta cluster T227348 (done)
DBA review T227355 (done)
Security readiness review T227346 (complete, follow-up tasks created, no deployment blockers identified)
Performance review T230813 (complete)
Release Manager signoff

Target deployment date: November 5, 2019 (testcommonswiki), November 6, 2019 (commonswiki)

Details

Subject	Repo	Branch	Lines +/-
Enable MachineVision on commonswiki	operations/mediawiki-config	master	+1 -0
Configure and enable MachineVision on testcommonswiki	operations/mediawiki-config	master	+69 -0
MachineVision: Update filtered_tables.txt	operations/puppet	production	+5 -3

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• Ramsey-WMF	T225964 [SDC] Build a depicts tag suggestion tool that is powered by machine vision platforms
Resolved	• Mholloway	T226119 Build middleware to utilize machine vision API for structured data on commons depicts tag suggestion tool
Resolved	• Mholloway	T227349 Deploy the MachineVision extension to production
Resolved	• Mholloway	T227348 Deploy MachineVision extension to the Beta Cluster
Resolved	sbassett	T227346 Security readiness review for the MachineVision extension
Resolved	• Mholloway	T227347 Create a diagram of the machine vision middleware architecture
Resolved	Marostegui	T227355 DBA review for the MachineVision extension
Resolved	• Mholloway	T227351 Define DB table schemas for MV depicts suggestions
Resolved	Krinkle	T230813 Performance review for the MachineVision extension
Resolved	• Mholloway	T240210 Fallback text for non-JS-supporting clients on Special:SuggestedTags
Resolved	None	T242662 "machinevision-machineaidedtagging-intro" message should not refer to production interwiki codes
Resolved	• Ramsey-WMF	T242667 Reduce flash of empty page and hoisted footer when loading Special:SuggestedTags (FOUC)
Resolved	• Mholloway	T236797 How should the MachineVision extension interact with external APIs from production?
Resolved	• Mholloway	T236426 Configure Google Cloud Vision credentials in production
Resolved	• Mholloway	T236843 Add HTTP proxy support to the MachineVision extension
Resolved	• Mholloway	T237596 Add MachineVision dependencies to vendor
Declined	Reedy	T237588 Security review for MachineVision libraries

Event Timeline

• Mholloway created this task.Jul 5 2019, 5:14 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 5 2019, 5:14 PM

• Mholloway added a subtask: T227348: Deploy MachineVision extension to the Beta Cluster.Jul 5 2019, 5:15 PM

• Mholloway added a subtask: T227346: Security readiness review for the MachineVision extension.Jul 5 2019, 5:34 PM

• Mholloway added a subtask: T227355: DBA review for the MachineVision extension.Jul 5 2019, 5:37 PM

• Mholloway added a parent task: T226119: Build middleware to utilize machine vision API for structured data on commons depicts tag suggestion tool.Jul 17 2019, 3:43 PM

LGoto moved this task from Needs triage to Backlog on the Product-Infrastructure-Team-Backlog-Deprecated board.Jul 17 2019, 3:44 PM

• Mholloway closed subtask T227348: Deploy MachineVision extension to the Beta Cluster as Resolved.Aug 13 2019, 8:40 PM

• Mholloway renamed this task from Deploy machine vision middleware to production to Deploy the MachineVision extension to production.Aug 20 2019, 3:12 PM

• Mholloway updated the task description. (Show Details)

• Mholloway updated the task description. (Show Details)Aug 31 2019, 5:32 PM

• Mholloway added a subtask: T230813: Performance review for the MachineVision extension.

• Mholloway updated the task description. (Show Details)Aug 31 2019, 5:45 PM

• Mholloway updated the task description. (Show Details)

• Mholloway updated the task description. (Show Details)Sep 10 2019, 12:34 AM

• Mholloway updated the task description. (Show Details)

• Mholloway updated the task description. (Show Details)Sep 10 2019, 12:40 AM

• Mholloway updated the task description. (Show Details)

• Mholloway updated the task description. (Show Details)Sep 10 2019, 1:08 AM

• Mholloway triaged this task as Medium priority.Sep 10 2019, 1:22 AM

Hi @greg, I'm wondering, is there any difference between the requirements for enabling a new extension on a production test wiki and a "real" production wiki? We are hoping to enable this on testcommonswiki ASAP, but not necessarily full commonswiki until closer to the feature launch date.

Going by past experience, DBA and performance reviews will go pretty quickly, but security review will take some time. On the linked security readiness review ticket, they indicate that they'll probably only be able to do a basic review in the next 30 days, and will take longer to look in-depth. (Note, however, that I previously requested a security concept review for the extension, in which it was judged to be low-risk.) Is that sufficient for launching to testcommonswiki and/or commonswiki?

The target date for launching the Computer-Aided Tagging feature is the end of October. The October 9 deadline here and in the linked tickets is to allow a few weeks on testcommonswiki and commonswiki for testing before launch.

Thank you!

• Mholloway updated the task description. (Show Details)Sep 10 2019, 9:51 PM

• Mholloway closed subtask T227355: DBA review for the MachineVision extension as Resolved.Oct 18 2019, 3:41 PM

• Mholloway updated the task description. (Show Details)

• Mholloway raised the priority of this task from Medium to High.Oct 24 2019, 12:49 AM

In T227349#5481210, @Mholloway wrote:

Hi @greg, I'm wondering, is there any difference between the requirements for enabling a new extension on a production test wiki and a "real" production wiki? We are hoping to enable this on testcommonswiki ASAP, but not necessarily full commonswiki until closer to the feature launch date.

Going by past experience, DBA and performance reviews will go pretty quickly, but security review will take some time. On the linked security readiness review ticket, they indicate that they'll probably only be able to do a basic review in the next 30 days, and will take longer to look in-depth. (Note, however, that I previously requested a security concept review for the extension, in which it was judged to be low-risk.) Is that sufficient for launching to testcommonswiki and/or commonswiki?

The target date for launching the Computer-Aided Tagging feature is the end of October. The October 9 deadline here and in the linked tickets is to allow a few weeks on testcommonswiki and commonswiki for testing before launch.

Thank you!

Hi, please do ping out of band if I don't respond on Phab in a reasonable time.

tl;dr: production is production, same process.

• Mholloway added a subtask: T236797: How should the MachineVision extension interact with external APIs from production?.Oct 29 2019, 3:08 PM

• Mholloway updated the task description. (Show Details)Nov 2 2019, 12:25 AM

In T227349#5601466, @greg wrote:

Hi, please do ping out of band if I don't respond on Phab in a reasonable time.

tl;dr: production is production, same process.

Yeah, that's what I figured the answer was anyhow. :)

We are aiming to enable the extension in production next week, after wrapping up the remaining follow-ups from security and performance review on Monday. (It will be hidden behind a feature flag and invisible to all but a few users with special permissions for user testing for a couple of weeks after that.) Do we have your signoff?

• Mholloway updated the task description. (Show Details)Nov 5 2019, 2:40 PM

• Mholloway closed subtask T230813: Performance review for the MachineVision extension as Resolved.Nov 5 2019, 2:42 PM

• Mholloway closed subtask T227346: Security readiness review for the MachineVision extension as Resolved.Nov 5 2019, 3:15 PM

• Mholloway closed subtask T236797: How should the MachineVision extension interact with external APIs from production? as Resolved.Nov 5 2019, 3:19 PM

• Mholloway claimed this task.Nov 5 2019, 3:25 PM

• Mholloway updated the task description. (Show Details)

In T227349#5628203, @Mholloway wrote:

In T227349#5601466, @greg wrote:

Hi, please do ping out of band if I don't respond on Phab in a reasonable time.

tl;dr: production is production, same process.

Yeah, that's what I figured the answer was anyhow. :)

We are aiming to enable the extension in production next week, after wrapping up the remaining follow-ups from security and performance review on Monday. (It will be hidden behind a feature flag and invisible to all but a few users with special permissions for user testing for a couple of weeks after that.) Do we have your signoff?

Looks good. Thanks.

• Mholloway updated the task description. (Show Details)Nov 5 2019, 4:54 PM

Change 548869 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[operations/mediawiki-config@master] Configure and enable MachineVision on testcommonswiki

https://gerrit.wikimedia.org/r/548869

gerritbot added a project: Patch-For-Review.Nov 5 2019, 8:06 PM

• Mholloway edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.Nov 5 2019, 8:32 PM

• Mholloway moved this task from To Do to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.

Change 548886 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[operations/puppet@production] MachineVision: Update filtered_tables.txt

https://gerrit.wikimedia.org/r/548886

Change 548886 merged by Jcrespo:
[operations/puppet@production] MachineVision: Update filtered_tables.txt

https://gerrit.wikimedia.org/r/548886

Change 548869 merged by Mholloway:
[operations/mediawiki-config@master] Configure and enable MachineVision on testcommonswiki

https://gerrit.wikimedia.org/r/548869

Mentioned in SAL (#wikimedia-operations) [2019-11-06T15:52:21Z] <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Configure MachineVision and enable on testcommonswiki (T227349) (duration: 01m 00s)

Mentioned in SAL (#wikimedia-operations) [2019-11-06T15:58:45Z] <mdholloway> created MachineVision tables on testcommonswiki (T227349)

Mentioned in SAL (#wikimedia-operations) [2019-11-06T16:11:19Z] <mdholloway> MachineVision: Imported Freebase to Wikidata ID mappings on testcommonswiki (T227349)

Maintenance_bot removed a project: Patch-For-Review.Nov 6 2019, 4:11 PM

sbassett reopened subtask T227346: Security readiness review for the MachineVision extension as Open.Nov 6 2019, 8:06 PM

Change 549216 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[operations/mediawiki-config@master] Enable MachineVision on commonswiki

https://gerrit.wikimedia.org/r/549216

gerritbot added a project: Patch-For-Review.Nov 6 2019, 10:19 PM

Change 549216 merged by Mholloway:
[operations/mediawiki-config@master] Enable MachineVision on commonswiki

https://gerrit.wikimedia.org/r/549216

Mentioned in SAL (#wikimedia-operations) [2019-11-06T22:29:05Z] <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable MachineVision on commonswiki (T227349) (duration: 01m 00s)

Mentioned in SAL (#wikimedia-operations) [2019-11-06T22:36:53Z] <mdholloway> MachineVision: Imported Freebase to Wikidata ID mappings on commonswiki (T227349)

Maintenance_bot removed a project: Patch-For-Review.Nov 6 2019, 11:10 PM

Mentioned in SAL (#wikimedia-operations) [2019-11-07T17:25:28Z] <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Drop currently unsupported external dependencies (T227349) (duration: 05m 19s)