Avoid s3 hot partition issues:
Existing prefix: articles_v2/enwiki/The_Answer_to_Both_Your_Questions.json
Extended load on s3 causes partitioning, thus increases throughput but having the articles directly under enwiki leads to excessive throttling because s3 does not partition them well compared to articles_v2/enwiki/b/b1/The_Answer_to_Both_Your_Questions.json
This follows the WMF’s nomenclature to avoid the same issue.
- Design doc for this ticket
- Ensure the files are stored in s3 following the agreed structure
- Adapt APIs to fetch files using this structure