Page MenuHomePhabricator
Create Task
Maniphest T368760

Configure airflow webserver under Kubernetes to use OIDC authentication
Closed, ResolvedPublic
Actions

Description

As part of our project to T362788: Migrate Airflow to the dse-k8s cluster we would like to be able to improve the security model for Airflow, such that is uses the CAS/SSO system as a login authentication source.

Specifically, we would like to use the OIDC protocol.

Given that the Airflow webserver is a flask application, built with FAB (flask application builder) the process of configuring OIDC should be quite similar to that we undertook for Superset in T353794: Configure OIDC Authentication for Superset on K8S.

Details

Other Assignee
bking
SubjectRepoBranchLines +/-
airflow: implement SSO authoperations/deployment-chartsmaster+103 -7
Update airflow-test-k8s image to include authliboperations/deployment-chartsmaster+1 -1
trafficserver: add airflow-test-k8s discovery recordoperations/puppetproduction+5 -0
dns: provision airflow-test-k8s temp domainoperations/dnsmaster+3 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

BTullis created this task.Jun 28 2024, 4:35 PM
Gehel triaged this task as High priority.Jul 9 2024, 8:03 AM
Gehel moved this task from Incoming to Scratch on the Data-Platform-SRE board.
Gehel edited projects, added Data-Platform-SRE (2024.07.08 - 2024.07.28); removed Data-Platform-SRE.Jul 9 2024, 8:06 AM
Stevemunene claimed this task.Jul 23 2024, 10:17 AM
Stevemunene moved this task from Backlog to In Progress on the Data-Platform-SRE (2024.07.08 - 2024.07.28) board.
Stevemunene added a comment.Jul 29 2024, 7:25 AM

According to the Airflow: High-Availability Strategy, The proposed solution is to make a public URL for each instance, for example: https://airflow-analytics.wikimedia.org https://airflow-research.wikimedia.org
This will require the idp entries for all instances we have that is available https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/Airflow/Instances urls pending confirmation.

airflow-analytics
airflow-search
airflow-platform-eng
airflow-research
airflow-analytics-product
airflow-wmde

Written up on T371208

Since the test instances are the first to be deployed we shall start with airflow-analytics-test domains and OIDC setup

Stevemunene closed subtask T371210: Create the airflow-analytics-test domain and certs as Resolved.Jul 30 2024, 10:56 AM
bking updated Other Assignee, added: bking.Jul 31 2024, 1:11 PM
bking edited projects, added Data-Platform-SRE (2024.07.29 - 2024.08.16); removed Data-Platform-SRE (2024.07.08 - 2024.07.28).Jul 31 2024, 2:47 PM
bking moved this task from Backlog to In Progress on the Data-Platform-SRE (2024.07.29 - 2024.08.16) board.
Stevemunene moved this task from In Progress to Backlog on the Data-Platform-SRE (2024.07.29 - 2024.08.16) board.Aug 6 2024, 1:59 PM
BTullis added subscribers: brouberol, bking.Aug 7 2024, 4:46 PM

Since the test instances are the first to be deployed we shall start with airflow-analytics-test domains and OIDC setup

@Stevemunene @bking @brouberol - I think we decided to create a new Airflow instance called test-k8s to support this testing phase, didn't we?
See the Airflow on Kubernetes - Test and migration plan doc for details.

I'm fine to leave the airflow-analytics-testDNS records in place because we will need them in due course, but I think we will need additional configs for this new test-k8s instance.

gerritbot added a comment.Aug 12 2024, 4:02 PM

Change #1062048 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/dns@master] dns: provision airflow-test-k8s temp domain

https://gerrit.wikimedia.org/r/1062048

gerritbot added a project: Patch-For-Review.Aug 12 2024, 4:02 PM
gerritbot added a comment.Aug 15 2024, 9:35 AM

Change #1062048 merged by Stevemunene:

[operations/dns@master] dns: provision airflow-test-k8s temp domain

https://gerrit.wikimedia.org/r/1062048

Maintenance_bot removed a project: Patch-For-Review.Aug 15 2024, 10:30 AM
Gehel edited projects, added Data-Platform-SRE (2024.08.17 - 2024.09.06); removed Data-Platform-SRE (2024.07.29 - 2024.08.16).Aug 16 2024, 9:45 AM
gerritbot added a comment.Aug 16 2024, 1:52 PM

Change #1063195 had a related patch set uploaded (by Bking; author: Bking):

[operations/deployment-charts@master] WIP: airflow: implement SSO auth

https://gerrit.wikimedia.org/r/1063195

gerritbot added a project: Patch-For-Review.Aug 16 2024, 1:52 PM
gerritbot added a comment.Aug 19 2024, 4:05 PM

Change #1063848 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/puppet@production] trafficserver: add airflow-test-k8s discovery record

https://gerrit.wikimedia.org/r/1063848

gerritbot added a comment.Aug 20 2024, 2:47 PM

Change #1063848 merged by Stevemunene:

[operations/puppet@production] trafficserver: add airflow-test-k8s discovery record

https://gerrit.wikimedia.org/r/1063848

brouberol moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2024.08.17 - 2024.09.06) board.Aug 27 2024, 2:32 PM
CodeReviewBot added a comment.Aug 29 2024, 4:08 PM

stevemunene opened https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/merge_requests/13

Add Authlib module for SSO

CodeReviewBot added a comment.Aug 29 2024, 7:39 PM

bking merged https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/merge_requests/13

Add Authlib module for SSO

CodeReviewBot added a comment.Aug 30 2024, 7:18 AM

stevemunene opened https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/merge_requests/14

Add authlib to production build

CodeReviewBot added a comment.Aug 30 2024, 12:29 PM

stevemunene merged https://gitlab.wikimedia.org/repos/data-engineering/airflow/-/merge_requests/14

Add authlib to production build

gerritbot added a comment.Aug 30 2024, 12:41 PM

Change #1069166 had a related patch set uploaded (by Stevemunene; author: Stevemunene):

[operations/deployment-charts@master] Update airflow-test-k8s image to include authlib

https://gerrit.wikimedia.org/r/1069166

gerritbot added a comment.Aug 30 2024, 2:44 PM

Change #1069166 merged by jenkins-bot:

[operations/deployment-charts@master] Update airflow-test-k8s image to include authlib

https://gerrit.wikimedia.org/r/1069166

Stevemunene added a comment.Sep 4 2024, 8:58 AM

We have setup oidc config for testing on https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1063195 which follows the standard flask procedure with FAB as used in superset as well.
The OIDC redirect is working, however we do have an error when trying to authenticate with CAS "Application Not Authorized to Use CAS" as below;

image.png (728×3 px, 158 KB)

Upon digging this is due to how we defined the redirect url ie.

/var/log/cas# tail -f cas.log | grep "ERROR"
2024-09-04 08:49:27,407 ERROR [org.apereo.cas.support.oauth.util.OAuth20Utils] - <Unsupported [redirect_uri]: [https://airflow-test-k8s.wikimedia.org/oauth-authorized/CAS] does not match what is defined for registered service: [https://airflow-test-k8s\.wikimedia\.org\/[\w\/]*]. Service is considered unauthorized. Verify the service matching strategy used in the service definition is correct and does in fact match the client [https://airflow-test-k8s.wikimedia.org/oauth-authorized/CAS]>
2024-09-04 08:49:27,412 ERROR [org.apereo.cas.support.oauth.util.OAuth20Utils] - <Unsupported [redirect_uri]: [https://airflow-test-k8s.wikimedia.org/oauth-authorized/CAS] does not match what is defined for registered service: [https://airflow-test-k8s\.wikimedia\.org\/[\w\/]*]. Service is considered unauthorized. Verify the service matching strategy used in the service definition is correct and does in fact match the client [https://airflow-test-k8s.wikimedia.org/oauth-authorized/CAS]>

Implementing a fix for this

gerritbot added a comment.Sep 5 2024, 2:07 PM

Change #1063195 merged by jenkins-bot:

[operations/deployment-charts@master] airflow: implement SSO auth

https://gerrit.wikimedia.org/r/1063195

Maintenance_bot removed a project: Patch-For-Review.Sep 5 2024, 2:31 PM
Stevemunene closed this task as Resolved.Sep 5 2024, 4:15 PM
Stevemunene moved this task from In Progress to Done on the Data-Platform-SRE (2024.08.17 - 2024.09.06) board.

We have setup SSO authentication via CAS for the airflow webserver on our initial airflow-test-k8s instance. We have some primary role for new users and an admin group as well. This will act as a solid foundation for the other airflow instances we shall deploy.

Stevemunene closed subtask T371209: Create airflow-test-k8s OIDC configuration as Resolved.Sep 5 2024, 4:16 PM
Stevemunene mentioned this in T371209: Create airflow-test-k8s OIDC configuration.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits