Page MenuHomePhabricator
Create Task
Maniphest T343856

Move whitelist.txt from WDQS deploy repo into puppet and rename it to "allow list"
Closed, ResolvedPublic
Actions

Description

Federating with a new endpoint (example ticket here ) currently requires changing a file in the WDQS deploy repo (whitelist.txt), and rolling a new deploy, which is time-consuming and needlessly risky.

Move whitelist.txt into our Puppet code so it can be updated more quickly and safely and rename to allowlist.txt.

Details

SubjectRepoBranchLines +/-
wdqs: Add allowlist.txtoperations/puppetproduction+94 -0
query_service: move allowlist file resourceoperations/puppetproduction+0 -1
runBlazegraph.sh: Add ALLOWLIST optionwikidata/query/deploymaster+2 -1
Allow overriding allowlist file pathwikidata/query/rdfmaster+2 -1
query_service: let puppet manage whitelistoperations/puppetproduction+98 -3
query_service: let puppet manage whitelistoperations/puppetproduction+98 -3
query_service: rollback allowlist changesoperations/puppetproduction+0 -0
query_service: move allowlist file resourceoperations/puppetproduction+10 -10
query_service: Set correct path for allowlist in blazegraph.ppoperations/puppetproduction+1 -1
query_service: Set correct path for allowlistoperations/puppetproduction+3 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

bking created this task.Aug 8 2023, 8:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 8 2023, 8:33 PM
bking updated the task description. (Show Details)Aug 8 2023, 8:33 PM
Reedy subscribed.Aug 9 2023, 12:28 PM

Would be a good time to potentially rename the file in support of T254646: Reconsidering how we name things.

Gehel subscribed.Aug 10 2023, 12:50 PM
In T343856#9080017, @Reedy wrote:

Would be a good time to potentially rename the file in support of T254646: Reconsidering how we name things.

Yes!

Gehel edited projects, added Wikidata-Query-Service; removed Discovery-Search.Aug 10 2023, 12:51 PM
Maintenance_bot added a project: Wikidata.Aug 10 2023, 1:30 PM
Gehel renamed this task from Move whitelist.txt from WDQS deploy repo into puppet to Move whitelist.txt from WDQS deploy repo into puppet and rename it to "allow list".Aug 14 2023, 3:12 PM
Gehel moved this task from Incoming to Watching / Waiting on the Wikidata-Query-Service board.
gerritbot added a comment.Aug 15 2023, 7:33 PM

Change 949101 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query_service: let puppet manage whitelist

https://gerrit.wikimedia.org/r/949101

gerritbot added a project: Patch-For-Review.Aug 15 2023, 7:33 PM
gerritbot added a comment.Aug 15 2023, 7:37 PM

Change 949102 had a related patch set uploaded (by Bking; author: Bking):

[wikidata/query/rdf@master] Allow overriding allowlist file path

https://gerrit.wikimedia.org/r/949102

bking updated the task description. (Show Details)Aug 15 2023, 8:40 PM
bking mentioned this in T344291: Puppet: consider skipping SPDX enforcement on text files.Aug 15 2023, 8:49 PM
bking mentioned this in T342361: Examine/refactor WDQS startup scripts.Aug 15 2023, 9:52 PM
Gehel moved this task from Incoming to In Progress on the Data-Platform-SRE board.Aug 16 2023, 12:58 PM
Gehel assigned this task to RKemper.Aug 22 2023, 3:37 PM
bking updated the task description. (Show Details)Aug 22 2023, 6:31 PM
gerritbot added a comment.Aug 22 2023, 6:55 PM

Change 949101 merged by Bking:

[operations/puppet@production] query_service: let puppet manage whitelist

https://gerrit.wikimedia.org/r/949101

gerritbot added a comment.Aug 22 2023, 7:09 PM

Change 951562 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query_service: Set correct path for allowlist

https://gerrit.wikimedia.org/r/951562

gerritbot added a comment.Aug 22 2023, 7:10 PM

Change 951562 merged by Bking:

[operations/puppet@production] query_service: Set correct path for allowlist

https://gerrit.wikimedia.org/r/951562

gerritbot added a comment.Aug 22 2023, 7:16 PM

Change 951563 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query_service: Set correct path for allowlist in blazegraph.pp

https://gerrit.wikimedia.org/r/951563

gerritbot added a comment.Aug 22 2023, 7:18 PM

Change 951563 merged by Bking:

[operations/puppet@production] query_service: Set correct path for allowlist in blazegraph.pp

https://gerrit.wikimedia.org/r/951563

gerritbot added a comment.Aug 22 2023, 7:26 PM

Change 951565 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query_service: move allowlist file resource

https://gerrit.wikimedia.org/r/951565

gerritbot added a comment.Aug 22 2023, 7:29 PM

Change 951565 merged by Bking:

[operations/puppet@production] query_service: move allowlist file resource

https://gerrit.wikimedia.org/r/951565

gerritbot added a comment.Aug 22 2023, 7:33 PM

Change 951566 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query_service: move allowlist file resource

https://gerrit.wikimedia.org/r/951566

gerritbot added a comment.Aug 22 2023, 7:48 PM

Change 951576 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] query service: rollback allowlist changes

https://gerrit.wikimedia.org/r/951576

gerritbot added a comment.Aug 22 2023, 8:20 PM

Change 951576 abandoned by Bking:

[operations/puppet@production] query_service: rollback allowlist changes

Reason:

already rolled back via Iea8ca1caf661f4105ac7816f69c613f86e41825d

https://gerrit.wikimedia.org/r/951576

Stashbot added a comment.Aug 22 2023, 8:29 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-22T20:29:32Z] <inflatador> bking@cumin1001 enable/run puppet on hosts after rollback T343856

gerritbot added a comment.Aug 24 2023, 1:44 PM

Change 952215 had a related patch set uploaded (by Btullis; author: Bking):

[operations/puppet@production] query_service: let puppet manage whitelist

https://gerrit.wikimedia.org/r/952215

gerritbot added a comment.Aug 24 2023, 2:02 PM

Change 952221 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] wdqs: Add allowlist.txt

https://gerrit.wikimedia.org/r/952221

gerritbot added a comment.Aug 24 2023, 4:05 PM

Change 952215 abandoned by Btullis:

[operations/puppet@production] query_service: let puppet manage whitelist

Reason:

This was just a trial for running pcc. Now addressed in https://gerrit.wikimedia.org/r/c/operations/puppet/+/952221

https://gerrit.wikimedia.org/r/952215

gerritbot added a comment.Aug 24 2023, 6:53 PM

Change 952221 merged by Bking:

[operations/puppet@production] wdqs: Add allowlist.txt

https://gerrit.wikimedia.org/r/952221

gerritbot added a comment.Aug 24 2023, 7:39 PM

Change 949102 merged by jenkins-bot:

[wikidata/query/rdf@master] Allow overriding allowlist file path

https://gerrit.wikimedia.org/r/949102

Stashbot added a comment.Aug 24 2023, 8:34 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T20:34:26Z] <inflatador> bking@deploy1002 'scap deploy new wdqs T343856'

gerritbot added a comment.Aug 24 2023, 8:42 PM

Change 952275 had a related patch set uploaded (by Bking; author: Bking):

[wikidata/query/deploy@master] runBlazegraph.sh: Add ALLOWLIST option

https://gerrit.wikimedia.org/r/952275

gerritbot added a comment.Aug 24 2023, 8:43 PM

Change 952275 merged by Ryan Kemper:

[wikidata/query/deploy@master] runBlazegraph.sh: Add ALLOWLIST option

https://gerrit.wikimedia.org/r/952275

Stashbot added a comment.Aug 24 2023, 8:43 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T20:43:58Z] <bking@deploy1002> Started deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125

Stashbot added a comment.Aug 24 2023, 9:06 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:06:02Z] <bking@deploy1002> Finished deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125 (duration: 22m 03s)

Stashbot added a comment.Aug 24 2023, 9:06 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:06:51Z] <bking@deploy1002> Started deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125

Stashbot added a comment.Aug 24 2023, 9:09 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:09:48Z] <bking@deploy1002> Finished deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125 (duration: 02m 56s)

Stashbot added a comment.Aug 24 2023, 9:16 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:16:02Z] <bking@deploy1002> Started deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125

Stashbot added a comment.Aug 24 2023, 9:18 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:18:19Z] <bking@deploy1002> Finished deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125 (duration: 02m 17s)

Stashbot added a comment.Aug 24 2023, 9:19 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:19:55Z] <bking@deploy1002> Started deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125

Stashbot added a comment.Aug 24 2023, 9:28 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:28:13Z] <bking@deploy1002> Finished deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125 (duration: 08m 18s)

Stashbot added a comment.Aug 24 2023, 9:29 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:29:16Z] <bking@deploy1002> Started deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125

Stashbot added a comment.Aug 24 2023, 9:29 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-24T21:29:31Z] <bking@deploy1002> Finished deploy [wdqs/wdqs@16e3dcf]: allow list changes T343856 0.3.125 (duration: 00m 15s)

Gehel moved this task from In Progress to Needs Review on the Data-Platform-SRE board.Aug 29 2023, 3:37 PM
Gehel moved this task from Needs Review to Done on the Data-Platform-SRE board.Sep 4 2023, 3:00 PM

Tests on T337296 indicate that this change is successful. There are other issues in T337296, but the allow list is working as expected.

Gehel closed this task as Resolved.Sep 4 2023, 3:01 PM
gerritbot added a comment.Sep 6 2023, 9:23 PM

Change 951566 abandoned by Ryan Kemper:

[operations/puppet@production] query_service: move allowlist file resource

Reason:

Patch redundant (issue(s) addressed in other patches)

https://gerrit.wikimedia.org/r/951566

Maintenance_bot removed a project: Patch-For-Review.Sep 6 2023, 9:30 PM
Gehel closed subtask T344284: Rename usages of whitelist to allowlist in query service rdf repo as Resolved.Sep 22 2023, 9:49 AM
Physikerwelt added a subtask: T355893: Rename WDQS whitelist to allowlist.Jan 25 2024, 6:36 PM
GreenReaper mentioned this in T355893: Rename WDQS whitelist to allowlist.Mar 26 2024, 12:56 AM
RickiJay-WMDE changed the status of subtask T355893: Rename WDQS whitelist to allowlist from Open to In Progress.Mar 27 2024, 4:29 PM
RickiJay-WMDE closed subtask T355893: Rename WDQS whitelist to allowlist as Resolved.Apr 3 2024, 10:56 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits