Currently pickle files are stored in flat directories per language:
en/100000.p, en/100002.p, en/100003.p ... This creates over 7 million files in single directory for enwiki which causes filesystem performance issues. Proposed solution: Use subdirectories based on floor(page_id/1000) Examples: - page_id 100000 → en/100/100000.p - page_id 100002 → en/100/100002.p - page_id 200005 → en/200/200005.p This reduces files per directory from ~7M to ~7K. Implementation will include: - Helper functions for path calculation - Support for the new directory structure - Optional migration utility for existing files
Related to T414071