Page MenuHomePhabricator

Start a new instance of the popular pages bot lower down on the list
Closed, DeclinedPublic2 Estimated Story Points

Description

It looks like there's a chance the popular pages bot isn't going to finish all the reports before the end of the month. Let's start up a new instance of the bot somewhere lower in the list so that we can make sure all the reports get generated before the month ends.

One way to accomplish this would be to add an optional parameter to the checkReports.php script that let's the user specify a project to start at. (See generateReport.php for an example of using a command line param.)

This month was kind of chaotic since we were still getting things set up and modifying the code. Next month, let's let it run normally and see how far it gets by the Hackathon. If it hasn't made good progress, we can hack on it some more at the Hackathon.

Don't run this from the same Tool Labs tool or you'll hit the IP address throttle. You might want to just run this locally to be safe.

Event Timeline

kaldari set the point value for this task to 2.
kaldari moved this task from Needs Discussion to Up Next on the Community-Tech board.

I believe the reason behind the bot slow-ness is that we included the feature for adding up redirect pageviews after last month. So even if the total number of project pages in a wikiproject is 10k, it's easily fetching pageviews for 30k articles or more. I think it's sane to spawn a second bot-job starting further down the list for future. Will need a little bit of code tweaks including a separate logging file for the second bot.

Since April has already passed, I'm going to close this bug in favor of T164178.