Page MenuHomePhabricator

Enable OPDS catalog for English Wikisource
Closed, ResolvedPublic


Please enable the WS-export OPDS catalog for the English Wikisource, using [[Category:Ready for export]]

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The OPDS functionality is not yet live again on production, but it is on staging: the above OPDS export looks like this: (the URL will be similar for prod).

It validates with one warning:

One thing we need to know is what frequency of regeneration is suitable. Once a week?

It works in my ereader, so that's wonderful!

How long does it take to generate? I think it would ideally be at least daily so people can see their work?

Yeah, daily would be better for newly added works. For changes to existing works the frequency could be much lower with not much problem I think. Alternatively new works could be manually triggered (we have lots of manual processes already) given an interface for it.

Yes it sounds like we should try to be cleverer about it: the current system rebuilds the whole thing, and it takes quite a while. The above English one is only a couple of seconds, but the French equivalent is taking about a minute and the resulting file is 9.2 MB. It does seem to load completely fine in FBReader, for example, but surely there will be a point at which it's too big? I wonder if we'd be better off having different categories e.g. fiction/non-fiction or something? The current system allows any category name to be given, and it'll create a matching file at<lang>/<category>.xml I guess the main issue with that is that it doesn't currently descend into subcategories, but that could be fixed.

One way to deal with huge catalogues is to segment alphabetically. Scrolling the fr catalogue on koreader is bit of a faff because it has over 300 pages and you don't know where, say, R, starts, so you have to do a manual binary search.

Having a way to categorise within the catalog would be good too, so you can browse by topic, author, recently added, most downloaded, etc.

I've set up the cronjob to generate two OPDS files (en and fr) daily.

Moving to QA as there's nothing to review here.

Note that frwikisource has two legacy URLs for their OPDS feed, both of which redirect to the new location:

ifried added a subscriber: ifried.

I have been able to access the OPDS catalog for English Wikisource in my tests. I'm marking this work as Done.