Page MenuHomePhabricator

Special:IndexPages not shown total Index Pages in Bengali Wikisource ( https://bn.wikisource.org/)
Open, MediumPublic

Description

Special:IndexPages ( https://bn.wikisource.org/w/index.php?title=Special:IndexPages) not show total Index Pages in Bengali Wikisource ( https://bn.wikisource.org/) . Presently there are 80+ Index pages, But Special:IndexPages is showing only 61 pages.

Event Timeline

jayantanth raised the priority of this task from to Medium.
jayantanth updated the task description. (Show Details)
jayantanth added a subscriber: jayantanth.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 2 2015, 7:10 PM
jayantanth renamed this task from Special:IndexPages not show total Index Pages in Bengali Wikisource ( https://bn.wikisource.org/) to Special:IndexPages not shown total Index Pages in Bengali Wikisource ( https://bn.wikisource.org/).Sep 2 2015, 7:10 PM
jayantanth set Security to None.
Krenair added a subscriber: Krenair.
jayantanth updated the task description. (Show Details)Sep 2 2015, 7:21 PM
Tpt added a subscriber: Tpt.EditedSep 3 2015, 7:49 AM

Thank you for this bug report. Could you give me link of an Index: page not shown in Special:IndexPages? It would help me to find the cause of this bug.

jayantanth added a comment.EditedSep 3 2015, 3:58 PM

I am adding the following index pages which is not shown in Special:IndexPages

1.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:%E0%A6%B6%E0%A7%87%E0%A6%B7_%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%B6%E0%A7%8D%E0%A6%A8.djvu

2.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:%E0%A6%B6%E0%A6%95%E0%A7%81%E0%A6%A8%E0%A7%8D%E0%A6%A4%E0%A6%B2%E0%A6%BE_(%E0%A6%B8%E0%A6%BF%E0%A6%97%E0%A6%A8%E0%A7%87%E0%A6%9F_%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A7%87%E0%A6%B8_%E0%A6%B8%E0%A6%82%E0%A6%B8%E0%A7%8D%E0%A6%95%E0%A6%B0%E0%A6%A3).djvu

3.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:%E0%A6%B6%E0%A6%95%E0%A7%81%E0%A6%A8%E0%A7%8D%E0%A6%A4%E0%A6%B2%E0%A6%BE_(%E0%A6%86%E0%A6%A6%E0%A6%BF_%E0%A6%AC%E0%A7%8D%E0%A6%B0%E0%A6%BE%E0%A6%B9%E0%A7%8D%E0%A6%AE%E0%A6%B8%E0%A6%AE%E0%A6%BE%E0%A6%9C_%E0%A6%B8%E0%A6%82%E0%A6%B8%E0%A7%8D%E0%A6%95%E0%A6%B0%E0%A6%A3).djvu

4.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:%E0%A6%B0%E0%A6%99%E0%A7%8D%E0%A6%97%E0%A6%B2%E0%A6%BE%E0%A6%B2-%E0%A6%97%E0%A7%8D%E0%A6%B0%E0%A6%A8%E0%A7%8D%E0%A6%A5%E0%A6%BE%E0%A6%AC%E0%A6%B2%E0%A7%80.djvu

5.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:%E0%A6%AC%E0%A6%BE%E0%A6%97%E0%A7%87%E0%A6%B6%E0%A7%8D%E0%A6%AC%E0%A6%B0%E0%A7%80_%E0%A6%B6%E0%A6%BF%E0%A6%B2%E0%A7%8D%E0%A6%AA-%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%AC%E0%A6%A8%E0%A7%8D%E0%A6%A7%E0%A6%BE%E0%A6%AC%E0%A6%B2%E0%A7%80.djvu

6.https://bn.wikisource.org/wiki/%E0%A6%A8%E0%A6%BF%E0%A6%B0%E0%A7%8D%E0%A6%98%E0%A6%A3%E0%A7%8D%E0%A6%9F:Vanga_Sahitya_Parichaya_Part_1.djvu

and many more....

@Tpt, this problem has been described in detail in T93397.
modifying an Index page updates Special:IndexPages with the previous revision of the Index page, so... if someone creates new Index page, the Special:IndexPages is updated only after a purge or next edit of the Index page (before that the new Index is "invisible" on Special:IndexPages).
I have refreshed page # 1 from the list and... already Index is on Special:IndexPages.

On pl ws, we use the bot to purge Index pages that were created or modified (every few hours).

Z.

Billinghurst added a subscriber: Billinghurst.

I will run Wikisource-bot through to purge all the index files, and see if that makes a difference.

@jayantanth I have purged all the Index: ns pages. Please check whether that has resolved your issue.

 $ python /shared/pywikipedia/core/scripts/touch.py -lang:bn -family:wikisource -namespace:102 -start:\! -purge -pt:1



162 pages read
0 pages written
Execution time: 163 seconds
Read operation time: 1 seconds
Script terminated successfully.
Billinghurst moved this task from Backlog to Next-up on the Wikisource board.

Excellent! Thanks Billinghurst. Now its shown all index pages. Is it possible to run this bot frequently for us?

We shouldn't have to do so, though we can if there is value.

Let me do some x-language checks to see if we have other wikis with the
issue. Plus some thinking about some monitoring. I would welcome any
comment about what people see as the cause.

Zdzislaw added a comment.EditedSep 26 2015, 4:03 PM

@Billinghurst as I mentioned in T93397, if we want to make Special:IndexPages showing the current state of quality-bars, Index: page shall be always refresh after a change of status (none ->Not proofread; Not proofread -> Proofraed...) of any Page: from the Index (as we do by bot on pl ws).
The problem is (it is not only problem with the "new" Index: ns pages) that changing status of any Page:, the Index: page is updated but Special:IndexPages is update only after a next purge or edit of the Index:, see: Special:IndexPages of The story of geographical discovery.djvu -> qualitybar (0 validated pages, 5 only proofread pages and 5 not proofread pages) while nearly all of the Pages are validated (yellow) on Index:The_story_of_geographical_discovery.djvu. The same on Special:IndexPages নির্ঘণ্ট:গীতরত্ন গ্রন্থঃ (১৮৭০)- রামনিধি গুপ্ত.djvu (28 pages->red) and Index:গীতরত্ন গ্রন্থঃ (১৮৭০)- রামনিধি গুপ্ত (40 pages -> red).
So, if you want to fix this problem by using a bot, and no change the way of Special:IndexPages refresh, bot would purge the Index: ns pages on all ws every few hours :(

Z.

jayvdb added a subscriber: jayvdb.Oct 3 2015, 8:41 AM

P2148 is a (unix only) Pywikibot script that purges indexes pages for page: modified in the last five hours. The first time it is run on all wikis, increasing the period greater than five hours is probably a good idea.

Pages like this are usually updated once a week, try to wait until next update and see, if the missing pages were added

Dvorapa removed a subscriber: Dvorapa.

the Special page is solely part of ProofreadPage extension, not one of the standard specials.

@Billinghurst: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Do you still plan to work on fixing this task? Thanks.

@Aklapper I did the task and had it fixed at the time, left as others were continuing part of the conversation re the consequences.

@jayantanth @Bodhisattwa is there a need for a recurring cronjob for Wikisource-bot to touch index files? Guessing if there is for bnWS, then we may have that requirement for all wikisources.

@Zdzislaw made comment about the more general issue, and I am not certain whether @Tpt has a comment in that respect to the special page. I am not certain of the best result to manage this within proofread page, though I can set the bot to run cronjobs if that is a requirement, as it should all be zero edit touches.

@jayantanth @Bodhisattwa: Ping. Could you answer the last comment, please?

Billinghurst added a comment.EditedApr 22 2020, 3:50 PM

Issue has occurred at enWP again, so will be running the bot through. Just amending the script line to be

this equivalent though poked to the grid

python pwb.py touch.py -lang:en -family:wikisource  -namespace:106  -start:\! -purge -pt:2 -botflag

Whacking the server indiscriminately to purge cycle 17k Index: pages for enWS is not a good way to go IMNSHO.

Let us see how long it takes, and we can work out how and what needs doing holistically.

@Billinghurst Sorry for the late reply, If possible please continue run this python script automatically every.

Aklapper removed Billinghurst as the assignee of this task.Jun 19 2020, 4:24 PM

This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!

For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)