I was gonna file an issue but it seems like this in the intended behavior and I could have handled it by using a try/catch: http://addwiki.readthedocs.io/projects/mediawiki-api-base/en/latest/quickstart.html#sending-requests
Okay, this patch fixes the issue.
Okay, a kind soul on IRC helped! https://en.wikipedia.org/w/api.php?action=parse&page=Wikipedia:WikiProject%20Medicine/Popular%20pages&prop=sections
Okay, even if I do a 'query' it still gives a Fatal. https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=content&rvsection=7&format=jsonfm&titles=Wikipedia:WikiProject%20Adelaide/Popular%20pages
Okay. I see now. We're trying to 'parse' a non-existent section. In case of page we 'query' it. There's gotta be a workaround...
Wed, Mar 22
I think right now it is pretty obscure. It's not very intuitive that clicking the total pageviews count would pop up the graph.
Tue, Mar 21
@Pastakhov Some screenshots of the current styles:
Why is the graph not embedded directly into the page? Maybe like a thumbnail sized-version and clicking it can open a larger view?
Mon, Mar 20
Done. I'm limiting redirects to 500 for now. Should be okay.
@Framawiki could you elaborate the use case a bit here? The tool works quite well as it does now and for the English version there are less than 100 open records.
Fri, Mar 17
The limit without authentication is 500 results, but I doubt there are many pages with more than 500 redirects.
Since we're using the Community Tech bot account, this shouldn't be a problem.
Surprisingly not that many:
- Wikipedia:Version 0.5 --- Deleted by me, it doesn't look like it's something that should have a popular pages report.
- Wikipedia:Version 1.0 Editorial Team -- Same as above.
- Wikipedia:Wiki Loves Monuments 2011 -- Pretty much defunct
- Wikipedia:WikiProject Animation/Animated films work group - Added
- Wikipedia:WikiProject Animation/Animated television work group - No data in page assessment
- Wikipedia:WikiProject Animation/Computer animation work group - No data in page assessment
- Wikipedia:WikiProject Animation/Looney Tunes work group - No data in page assessment
- Wikipedia:WikiProject Animation/World animation work group - No data in page assessment
- Wikipedia:WikiProject BeyoncÃ© - Project no longer exists
- Wikipedia:WikiProject BjÃ¶rk - Project no longer exists
- Wikipedia:WikiProject California State Highways - No data in page assessment
- Wikipedia:WikiProject Food and Drink - Already in config with a slightly changed name (Food and drink)
- Wikipedia:WikiProject Michigan State Highways - No data in page assessment
- Wikipedia:WikiProject Molecular and Cellular Biology - Changed to Molecular and Cell Biology
- Wikipedia:WikiProject Oklahoma State Highways - No data in page assessment
- Wikipedia:WikiProject SUNY - No data in page assessment
- Wikipedia:WikiProject Space - Defunct
- Wikipedia:WikiProject Television/Episode coverage task force - Seems pretty defunct at this point
- Wikipedia:WikiProject The Who - No data in page assessment
- Wikipedia:WikiProject Utah State Highways - No data in page assessment
- Wikipedia:WikiProject Washington State Highways - No data in page assessment
- Wikipedia:WikiProject intelligent design - Moved to WikiProject Creationism
I'm waiting on this for a bit. I've reviewed all of the current popular pages and removed some of the projects from the config because they were defunct or there wasn't any data in page assessments about the project. I have tagged the reports for those projects for deletion. Let's wait till that happens.
Okay, I've cleaned up the config to match with data in Page Assessments. The pages should be updated in the next bot run, which I've submitted a job for.
Thu, Mar 16
@kaldari How else could this be done? Update the first "section" on the page?
Not yet. There's a ticket as you noted.
- The bot just ran on many pages, but WP Louisville just got an empty report. I assume others might have gotten empty reports as well.
Yes, that's also a known bug T159774: Popularpages bot not generating reports for some projects and task forces
- The previous bot didn't write over page headers (e.g. project nav banner) but the new version does. This will have to be fixed.
Doc James brought this up on the bot talk page. One possible solution is that the bot only update a specific section of the page. Say "List" or "Report". Can you file a ticket?
Wed, Mar 15
Yeah, I'm working off the list in https://en.wikipedia.org/wiki/User:Community_Tech_bot/Popular_pages.
Tue, Mar 14
Added to crontab.
The bot should now log pages it didn't find pageview data for. Hopefully this marks the end of this ticket.
The bot progress can be seen on https://en.wikipedia.org/wiki/User:Community_Tech_bot/Popular_pages
Down to 171. Unfortunately my edit over-wrote your comments. :(
Fri, Mar 10
@DannyH: What most websites do is to allow you to log yourself out of all active sessions. So the invader would immediately find himself logged out and unable to do anything. This should probably already exist in MediaWiki and if not, we should consider adding it to the extension because without it, it seems quite unhelpful.
@DannyH, but they are already in. So they can mess with pretty much everything.
@DannyH, Hmm, if someone logged in to your account from some other IP/computer then changing the password won't help much, I suppose.
Thu, Mar 9
Notifications for login attempts (successful or unsuccessful) from unrecognized computers/IPs is pretty standard. I know that Twitter, Facebook and Gmail definitely do this.
Went from 237->196
Wed, Mar 8
Tue, Mar 7
We can add the column for project once we get to that stage. Won't be for a while, I'm guessing.
Yep, that makes sense. The full job takes > 24 hours but if we add the -once flag and ensure that we only update projects not already updated, it should be fine.
Proposed table schema:
|Spiders||1||3 March 2017|
|Albums||3||1 February 2017|
|Australia||2||3 March 2017|
Mon, Mar 6
This task is blocked on the IRC reconnection problem - T157120: Refactor Eranbot to adjust wait time before IRC reconnection per-wiki.
Yep, it was local. Thanks, that looks like the problem. I'll re-run with promises and hope it all works this time.
- The bot is working fine for *most* projects but inexplicably, it fails for some very large projects. The exact point of failure lies in the setText function. I'm investigating this and will hopefully push a patch soon.
- I tried using Promises and while they also work fine for small projects, I got a whole new bunch of errors with larger projects:
PHP Warning: include(/Library/WebServer/Documents/popularpages/vendor/guzzlehttp/guzzle/src/Exception/ConnectException.php): failed to open stream: Too many open files in /Library/WebServer/Documents/popularpages/vendor/composer/ClassLoader.php on line 412
Maybe remove formatting from the table/use a normal wikitable. It won't cram up the Expires column then and span the whole page width.
Sat, Mar 4
Fri, Mar 3
Considering this done. The bot is working off of the JSON config on wiki.
Mon, Feb 27
Thoughts on this?
Wed, Feb 22
I'm not clear how the Template version would work. Would the report be stored in a database and if the template is used on a page, it loads the report on it somehow?
Feb 22 2017
In T156856#3044777, @MusikAnimal wrote:
The "description" Niharika put in at Popular pages config.json looks good, except we didn't point out the issue with commas, which is my biggest concern. The last entry can't have a comma at the end or it is invalid JSON that I assume will break the bot.
I've updated the page to hopefully make it clearer. Feel free to edit! :)
In a startling revelation, it seems to be impossible to save the page if the JSON is invalid. And the more interesting thing - if we add a trailing comma at the end, of the last field or the last wikiproject block, it saves the JSON and removes the comma on its own. Feel free to test it out on this shorter version of that page/Sandbox.json).
The one last thing I wanted to say was you should probably wait until the 2nd or 3rd of the month to run the reports. Data is not guaranteed to be available until 24 hours after the fact, and even then sometimes the jobs run a little late, from my experience. For Topviews I don't allow querying of the previous month until the 2nd, but since the popular pages bot is a monthly job, we might wait until the 3rd just in case things are a little behind.
The wikitext table I intend to keep updated would look like:
|Wikiproject title||Report link||Limit||Updated on|
|Wikipedia:WikiProject_Spiders||Wikipedia:WikiProject_Spiders/Popular_pages||500||21 Feb 2017|
|Wikipedia:WikiProject_Biology||Wikipedia:WikiProject_Biology/Popular_pages||1000||21 Feb 2017|
And I'd make it clear in the table header that the bot will update the page after every run cycle, so don't edit it. Does that still seem like a bad idea? I thought it'd serve as an index of sorts.
Feb 21 2017
The bot has successfully completed its trial run and updated 50 project reports. They are the first 50 in the "Report" column in https://en.wikipedia.org/wiki/User:Community_Tech_bot/Popular_pages_config.
Since nobody raised any objections, I'll shortly be moving the config page to JSON instead of the current wikitext table.
Feb 20 2017
Feb 16 2017
Feb 15 2017
Feb 14 2017
Per T128806: Switch German Wikipedia to uca-de category collation, this would need some consensus. I'm going to merge this task into that one.
Feb 13 2017
Thank you, @Stevietheman. Would it simplify things if we defaulted to 1000 for every project without there being a config to change it? It isn't computationally expensive as it was with the prior bot.
I have a question about the current config. What's the purpose of "Number of pages"?