HomePhabricator
Diffusion wikimedia/discovery/analytics 8d71e72c9a0e

Configuration for imagerec data shipping
8d71e72c9a0e
Actions

Description

Configuration for imagerec data shipping

Bug: T285816
Change-Id: I1b25f96ed28b311bfe3cd667a3179cdeeb390b58

Details

Provenance
EBernhardsonAuthored on Aug 17 2021, 9:51 PM
Parents
rWDAN563f87614198: Increase parallelism of process_sparql_query_hourly
Branches
Unknown
Tags
Unknown
ChangeId
I1b25f96ed28b311bfe3cd667a3179cdeeb390b58

Changes (5)

PathSize
airflow/
dags/
tests/
fixtures/
skein_operator_spec/
imagerec_manual-upload_to_swift.expected
ores_predictions_bulk_ingest-upload_to_swift.expected
spark_submit_operator/
imagerec_manual_convert_to_esbulk.expected
transfer_to_es_hourly_convert_to_esbulk.expected
spark/

rWDAN8d71e72c9a0e

View Options

airflow/dags/transfer_to_es.py

Loading...
View Options

airflow/tests/conftest.py

Loading...
View Options

airflow/tests/fixtures/skein_operator_spec/imagerec_manual-upload_to_swift.expected

Loading...
View Options

airflow/tests/fixtures/spark_submit_operator/imagerec_manual_convert_to_esbulk.expected

Loading...
View Options

spark/convert_to_esbulk.py

Loading...
Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL