cc @Tpt , @Samwilson

cc @Tpt , @Samwilson hello.
On WikiSource and with ws-export, how to you do to get the full text for a root page such as https://fr.wikisource.org/wiki/Élégies,_Marie_et_romances/Romances which only contain a hard to read template ?
Even on a page itself I can't get the plain text

• https://fr.wikisource.org/w/api.php?action=query&prop=extracts&explaintext&format=json&utf8=1&origin=*&titles=Page:Desbordes-Valmore_-_%C3%89l%C3%A9gies,_Marie_et_romances.pdf/185

Any code to point us to and to share ?

Do you actually want plain text (i.e. without any formatting), or do you mean the full HTML of the page?

You could use ws-export to get the HTML, e.g. https://ws-export.wmcloud.org/?lang=fr&title=Élégies%2C_Marie_et_romances%2FRomances&format=htmlz

Does using WS Export not work for your use case? Is it the rate limiting? If you log in, you can bypass that.

Hello @Samwilson ,

I actually didn't noticed your previous message :

In T279816#10005450, @Samwilson wrote:

Do you actually want plain text (i.e. without any formatting), or do you mean the full HTML of the page?

You could use ws-export to get the HTML, e.g. :

• https://ws-export.wmcloud.org/?lang=fr&title=Élégies%2C_Marie_et_romances%2FRomances&format=htmlz

Maybe the following variant programmatic URL could work for our case :

• https://ws-export.wmcloud.org/?lang=fr&page=Élégies,_Marie_et_romances/Romances&format=txt&fonts=&credits=false&images=false

I note from some reading about a query limit of 1 per minute (?), this would problematic.

It would required either :

• to whitelist queries from https://*.lingualibre.org

or

• Given our users will already be Oauth logged onto wiki(m|p)edia.org network, should we send some Oauth key to ws-export.wmcloud.org ?

I note from some reading about a query limit of 1 per minute (?), this would problematic.

Yep, the request limit is currently 2 requests per minute. Have you been hitting the limit, or are you just being cautious? We can definitely figure out a way for lingualibre to bypass it (or you can have the lingualibre tool look in and store it's oauth access token), but how many are you expecting to use it? It seems like you'd probably want to be caching the exported works, and not re-requesting repeatedly.

I'm being cautious.
We just have about 4~6 Lingua Libre users per day, less than one per day will use this Wikisource tool.
It wont be a burden on your service but when our user fetch a text from Wikisource they typically redo to adjusts it a bit.
So being cautious since I expect such hesitations and redo reaching up to 2~4 times per minutes once a week.

No worries.

What is the "redo to adjust" process? Do you mean they'll get e.g. a chapter to read, notice some errors with it, go and fix those, and then get lingualibre to re-download that chapter? Because there's also caching within the tool, so that mightn't work that well.