The Developer Hub prototype must feature three APIs across three sections:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| common: do not inject principal/keytab in dag_default_args in k8s devenvs | repos/data-engineering/airflow-dags!1606 | brouberol | simple-skein-operator-k8s-devenv | main | |
| main: ingest Superset data into datahub | repos/data-engineering/airflow-dags!1605 | brouberol | T306903 | main | |
| d/changelog: bump to 16.1.7 | repos/cloud/toolforge/jobs-cli!84 | raymond-ndibe | bump_jobs-cli | main | |
| Enable Spark Lineage on even more dags | repos/data-engineering/airflow-dags!1059 | tchin | enable-even-more-spark-lineage | main | |
| Disable Spark Lineage on questionable Spark operations | repos/data-engineering/airflow-dags!991 | tchin | disable-spark-lineage | main | |
| Enable Spark Lineage on a DAG that generates temp files | repos/data-engineering/airflow-dags!976 | tchin | temp-file-lineage | main | |
| scap prep: set pushInsteadOf in the submodules config | repos/releng/scap!607 | hashar | scap-prep-pushInsteadOf | master | |
| Enable spark lineage on more dags | repos/data-engineering/airflow-dags!935 | tchin | enable-more-lineage | main | |
| scheduled job: add timeout parameter | repos/cloud/toolforge/jobs-cli!76 | dcaro | add_timeout_for_scheduled_job | main | |
| scheduled jobs: add timeout option | repos/cloud/toolforge/jobs-api!129 | dcaro | add_timeout_for_scheduled_job | main | |
| Add Datahub Spark lineage support (no iceberg) | repos/data-engineering/airflow-dags!820 | tchin | support-spark-lineage | main | |
| Add pyflakes to CI and fix associated errors: | repos/abstract-wiki/wikifunctions/function-evaluator!151 | apine | apine-pyflakes | main | |
| Insert officewiki webrequests into destination | repos/data-engineering/airflow-dags!83 | milimetric | officewiki-webrequest | main | |
| Update spark3 analytics for SparkSQLNoCLIDriver | repos/data-engineering/airflow-dags!62 | joal | spark3_sql | main |
@Spage, since you are looking for the first APIs to work on, please check the subtasks of this task defining the three APIs Sumana thought that we should focus on. These are your defaults today, but feel free to change them if you think that we should prioritize other APIs.
The early version covers all these!
There are articles featuring these APIs and improvements I've made to their reference material.
The exception is T335: there's a link to explore RESTBase to "Request HTML of Wikimedia articles", but no showcase/inspire yet.
Is this because you didn't have time to write those yet, or because there are no good showcases? If the latter, I think we can resolve this task and move forward.
Rendering HTML of a wiki page is faster. But talking to partnership people that's not an important use case (partners want search, extracts, snippets). Declaring victory, more articles on workboard.