As a Wikisource user, I want the team to see if we can prevent web crawlers from downloading books, so the ebook exports can only be done by real users (and, therefore, the queue will be smaller & more efficient).
Background: This is a follow-up of T256018. As we have discussed, if we keep the download links in the sidebar, we will still have web crawlers. However, if we add a download button at the top right of the book, it will not have web crawler access. This brings up a question: Can we replace all current download links to the new system, so we can prevent automated downloads and therefore increase reliability?
- Investigate if we can prevent automated downloads via bots & webcrawlers
- Investigate how we can prevent automated downloads via bots & webcrawlers
- Investigate the main challenges, risks, and dependencies associated with such work
- Provide a general estimate/idea, if possible, of the potential impact it may have on ebook export reliability
- Provide a general estimation/rough sense of the level of difficulty of effort required in doing such work
- Can there be a system to allow approved bots to download books?
- Discuss with @Prtksxna how UX changes may prevent bots from downloading books
- When discussing potential options and solutions, consider that people may want a way to download a bulk or mass of books as well (rather than only being provided an option to download books one at a time).
- Share findings with the team