We need to do a new ingestion to introduce main image into our snapshots API, so that we can offer it in all of our product offerings.
Acceptance criteria
Image object is in the snapshots.
ToDo
- automate the ingestion process
- add the ability to bulk-ingestion DAG to upscale articlebulk service
- add the ability to bulk-ingestion DAG to upscale bulk-ingestion service
- add the ability to bulk-ingestion DAG to monitor messages lag for article-names topic
- check how it's going to influence snapshots (might have duplications in the topic for the time before compaction runs)
- double check structured-data-articlebulk check in there any changes needs to be introduced
- notify SRE about incoming traffic from us
- deploy changes to prod
- structured data article bulk service deployment
- scheduler deployment
- infrastructure deployment
- run the bulk-ingestion DAG in prod