Page MenuHomePhabricator

Set up the application authentication for WCQS on commons-query.wikimedia.org
Closed, ResolvedPublic5 Estimated Story Points

Description

As a WCQS maintainer I want users of the service to be authenticated so that we can react to actions that affect WCQS negatively.

This task depends on T282117 - domain has to be set up first.

AC:

  • Authentication is working on commons-query.wikimedia.org in the same way as it does for the WCQS beta

Details

Show related patches Customize query in gerrit

Event Timeline

Zbyszko renamed this task from Set up the application authentication for WCQS to Set up the application authentication for WCQS on commons-query.wikimedia.org.May 6 2021, 12:56 PM
Zbyszko updated the task description. (Show Details)
Zbyszko updated the task description. (Show Details)
MPhamWMF moved this task from Incoming to SDAW on the Wikidata-Query-Service board.

Plan right now assumes that WCQS API will be provided via https://api.wikimedia.org/ - technical feasibility is still to be assessed. This will allow us to provide an OAuth 2.0 flow and standardised experience similar to other future wikimedia APIs.

In the future, WDQS UI (used for WCQS as well) could be adapted to use the same endpoint.

Zbyszko, Erik, and I discussed the current status of this effort. We've realized that there's some open questions around how to actually implement the oauth, particularly with stuff like the callback URL given the way our routing logic works currently.

Currently, for WDQS (which does not use oauth), users visit the "microsite" for the UI, but their requests end up hitting nginx (which forwards them to blazegraph).

For WCQS, there won't be that pattern of a request hitting blazegraph directly. They'll be talking to the microsite, and the user is never going to do an interactive request against the backend server (e.g. nginx->blazegraph).

ATS at the edge [*before* the request hits the internal loadbalancer (lvs)] will look at request and decide, based off the directory, whether it should go to the microsite or nginx. There's one special directory that will route to nginx, whereas everything else will go to the microsite.

Where this breaks down is with oauth in the picture, this pattern won't work.

  • One approach would be for all traffic to route to nginx, and then have nginx decide what to forward to the microsite. As a general rule we were trying to avoid this approach though because we want this routing logic handled at the "infra" level, not the app layer.
  • Another approach might be to sort of lightly "integrate" oauth into the microsite UI, so that there's basically one magic directory that goes to nginx instead of routing all traffic like the above approach

We'll need to figure out an approach here.

We should figure out who is the holder of the auth consumer - we probably should have a service account for mediawiki

Change 724821 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/puppet@production] wcqs: enable oauth

https://gerrit.wikimedia.org/r/724821

Change 724829 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[operations/puppet@production] query_service: Split oauth secret from settings

https://gerrit.wikimedia.org/r/724829

Change 724830 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/puppet@production] query_service: Parameterize url redirected to after oauth success

https://gerrit.wikimedia.org/r/724830

Change 724831 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[labs/private@master] query_service: Split consumer secret out of oauth_settings

https://gerrit.wikimedia.org/r/724831

Change 724831 merged by Ryan Kemper:

[labs/private@master] query_service: Split consumer secret out of oauth_settings

https://gerrit.wikimedia.org/r/724831

Change 724832 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[labs/private@master] query_service: Remove non-secret values from secrets repo

https://gerrit.wikimedia.org/r/724832

Change 725049 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[labs/private@master] Move query_service secrets to profile-specific file

https://gerrit.wikimedia.org/r/725049

Change 725049 merged by Ryan Kemper:

[labs/private@master] Move query_service secrets to profile-specific file

https://gerrit.wikimedia.org/r/725049

Change 725084 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[labs/private@master] query_service: keep oauth secret in both paths

https://gerrit.wikimedia.org/r/725084

Change 725084 merged by Ryan Kemper:

[labs/private@master] query_service: keep oauth secret in both paths

https://gerrit.wikimedia.org/r/725084

Change 724832 abandoned by Ryan Kemper:

[labs/private@master] query_service: Remove non-secret values from secrets repo

Reason:

obsoleted by https://gerrit.wikimedia.org/r/c/labs/private/+/725049/1/hieradata/common.yaml

https://gerrit.wikimedia.org/r/724832

Change 724829 merged by Ryan Kemper:

[operations/puppet@production] query_service: Split oauth secret from settings

https://gerrit.wikimedia.org/r/724829

Change 724830 merged by Ryan Kemper:

[operations/puppet@production] query_service: Parameterize oauth redirect url

https://gerrit.wikimedia.org/r/724830

Change 725104 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/puppet@production] query_service: default oauth_settings to {}

https://gerrit.wikimedia.org/r/725104

Change 725104 merged by Ryan Kemper:

[operations/puppet@production] query_service: default oauth_settings in gui to {}

https://gerrit.wikimedia.org/r/725104

Change 725110 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[operations/puppet@production] wcqs: disable oauth while fixing readiness probe

https://gerrit.wikimedia.org/r/725110

Change 725110 merged by Ryan Kemper:

[operations/puppet@production] wcqs: disable oauth while fixing readiness probe

https://gerrit.wikimedia.org/r/725110

Change 725120 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[operations/puppet@production] query_service: Exempt health check url from oauth

https://gerrit.wikimedia.org/r/725120

Change 725120 merged by Ryan Kemper:

[operations/puppet@production] query_service: Exempt health check url from oauth

https://gerrit.wikimedia.org/r/725120

Change 732735 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[wikidata/query/deploy@master] Add wcqs group for scap deployment

https://gerrit.wikimedia.org/r/732735

Change 732742 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[wikidata/query/deploy@master] deploy version 0.3.90

https://gerrit.wikimedia.org/r/732742

Change 732735 merged by Ryan Kemper:

[wikidata/query/deploy@master] Add wcqs group for scap deployment

https://gerrit.wikimedia.org/r/732735

Change 732742 merged by Ryan Kemper:

[wikidata/query/deploy@master] deploy version 0.3.90

https://gerrit.wikimedia.org/r/732742

Change 732772 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[wikidata/query/deploy@master] Use a separate scap environment for wcqs deploy

https://gerrit.wikimedia.org/r/732772

Change 732772 merged by Ryan Kemper:

[wikidata/query/deploy@master] Use a separate scap environment for wcqs deploy

https://gerrit.wikimedia.org/r/732772

Change 732797 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[wikidata/query/deploy@master] Reduce config in wcqs environment to only that necessary

https://gerrit.wikimedia.org/r/732797

Change 732797 merged by Ryan Kemper:

[wikidata/query/deploy@master] Reduce config in wcqs environment to only that necessary

https://gerrit.wikimedia.org/r/732797

Change 732800 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[wikidata/query/deploy@master] runBlazegraph.sh: Add env vars for new oauth properties

https://gerrit.wikimedia.org/r/732800

Change 732801 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/puppet@production] query_service: Add new oauth related configuration

https://gerrit.wikimedia.org/r/732801

Change 732800 merged by Ryan Kemper:

[wikidata/query/deploy@master] runBlazegraph.sh: Add env vars for new oauth properties

https://gerrit.wikimedia.org/r/732800

Change 734418 had a related patch set uploaded (by Ryan Kemper; author: Ryan Kemper):

[labs/private@master] wcqs: add dummy oauth_access_token_secret

https://gerrit.wikimedia.org/r/734418

Change 734418 merged by Ryan Kemper:

[labs/private@master] wcqs: add dummy oauth_access_token_secret

https://gerrit.wikimedia.org/r/734418

Change 732801 merged by Ryan Kemper:

[operations/puppet@production] query_service: Add new oauth related configuration

https://gerrit.wikimedia.org/r/732801

Change 734702 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/puppet@production] query_service: jvm defines are provided with -D

https://gerrit.wikimedia.org/r/734702

Change 734702 merged by Ryan Kemper:

[operations/puppet@production] query_service: jvm defines are provided with -D

https://gerrit.wikimedia.org/r/734702

Authentication is deployed and appears to be working as desired.

Change 743499 had a related patch set uploaded (by Ryan Kemper; author: Ebernhardson):

[operations/puppet@production] rdf-query-service: Allow logback config to load outside the blazegraph war

https://gerrit.wikimedia.org/r/743499

Change 743499 merged by Ryan Kemper:

[operations/puppet@production] rdf-query-service: Allow logback config to load outside the blazegraph war

https://gerrit.wikimedia.org/r/743499

Change 724821 abandoned by Ryan Kemper:

[operations/puppet@production] wcqs: enable oauth

Reason:

https://gerrit.wikimedia.org/r/724821

Phabricator task to remove authentication: T297995