wooster wrote:

On Sat Jan 28 00:20:39 2012, robla wrote:

See https://bugzilla.wikimedia.org/show_bug.cgi?id=16112

Periodic runs = daily. weekly, fortnightly?

Status changed from 'new' to 'open' by RT_System

On Wed Feb 01 17:57:51 2012, wooster wrote:

On Sat Jan 28 00:20:39 2012, robla wrote:

See https://bugzilla.wikimedia.org/show_bug.cgi?id=16112

Periodic runs = daily. weekly, fortnightly?

Your call. This should be resolved with the original requestor in Bugzilla.

wooster wrote:

On Thu Feb 02 02:04:50 2012, robla wrote:

On Wed Feb 01 17:57:51 2012, wooster wrote:

On Sat Jan 28 00:20:39 2012, robla wrote:

See https://bugzilla.wikimedia.org/show_bug.cgi?id=16112

Periodic runs = daily. weekly, fortnightly?

Your call. This should be resolved with the original requestor in Bugzilla.

Discussed to RobLa. Set it to run as a monthly.

wooster wrote:

On Fri Feb 10 18:59:24 2012, wooster wrote:

On Thu Feb 02 02:04:50 2012, robla wrote:

On Wed Feb 01 17:57:51 2012, wooster wrote:

On Sat Jan 28 00:20:39 2012, robla wrote:

See https://bugzilla.wikimedia.org/show_bug.cgi?id=16112

Periodic runs = daily. weekly, fortnightly?

Your call. This should be resolved with the original requestor in

Bugzilla.

Discussed with RobLa. Set it to run as a monthly.

so, looks like there are these options for a (cron) command on ALL wikis:
1) foreachwiki
cat /usr/local/bin/foreachwiki .... and/ or a subset using a dblist:
/usr/local/bin/foreachwikiindblist ..
2) mwscriptwikiset
since mwscriptwikiset appears on the Het-deploy page, i guess that would be
preferred, even though "foreachwiki" also seems to use that.
the command to run refreshLinks --dfn-only an ALL Wikipedias _should_ look like
this then:
mwscriptwikiset refreshLinks.php --dfn-only wikipedia.dblist
http://wikitech.wikimedia.org/view/Heterogeneous_deployment#Run_a_maintenance_script_on_a_wiki
But there is rumour that running it on all wikis on a regular basis might cause
performance issues.?!

Queue changed from ops-requests to core-ops by dzahn

Issue taken by dzahn

wooster wrote:

Suggested to Daniel that he runs it once with RobLa's team on standby. If
everything is fine, add job to cron and staggered the run so that the
performance impact is minimal.
On Sat Feb 11 10:44:48 2012, dzahn wrote:

so, looks like there are these options for a (cron) command on ALL
wikis:

1. foreachwiki

cat /usr/local/bin/foreachwiki .... and/ or a subset using a dblist:
/usr/local/bin/foreachwikiindblist ..

2. mwscriptwikiset

since mwscriptwikiset appears on the Het-deploy page, i guess that
would be
preferred, even though "foreachwiki" also seems to use that.

the command to run refreshLinks --dfn-only an ALL Wikipedias _should_
look like
this then:

mwscriptwikiset refreshLinks.php --dfn-only wikipedia.dblist

http://wikitech.wikimedia.org/view/Heterogeneous_deployment#Run_a_maintenance_script_on_a_wiki

But there is rumour that running it on all wikis on a regular basis
might cause
performance issues.?!

Cc reedy added by robla

Sure. Sam, can you be available to help out with the refreshLinks.php test run?
Daniel, can you pick a time that works for you and Reedy?
Thanks
Rob
On Mon Feb 13 17:57:16 2012, wooster wrote:

Suggested to Daniel that he runs it once with RobLa's team on standby.
If
everything is fine, add job to cron and staggered the run so that the
performance impact is minimal.

On Sat Feb 11 10:44:48 2012, dzahn wrote:

so, looks like there are these options for a (cron) command on ALL
wikis:

1. foreachwiki

cat /usr/local/bin/foreachwiki .... and/ or a subset using a dblist:
/usr/local/bin/foreachwikiindblist ..

2. mwscriptwikiset

since mwscriptwikiset appears on the Het-deploy page, i guess that
would be
preferred, even though "foreachwiki" also seems to use that.

the command to run refreshLinks --dfn-only an ALL Wikipedias

_should_

look like
this then:

mwscriptwikiset refreshLinks.php --dfn-only wikipedia.dblist

http://wikitech.wikimedia.org/view/Heterogeneous_deployment#Run_a_maintenance_script_on_a_wiki

But there is rumour that running it on all wikis on a regular basis
might cause
performance issues.?!

RFC:
https://gerrit.wikimedia.org/r/#patch,sidebyside,5104,3,manifests/mediawiki.pp

https://gerrit.wikimedia.org/r/#change,5104
https://gerrit.wikimedia.org/r/#change,5322

notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s6@6]/Cron[cron-refreshlinks-s6@6]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s5@5]/Cron[cron-refreshlinks-s5@5]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s1@0]/Cron[cron-refreshlinks-s1@0]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s7@7]/Cron[cron-refreshlinks-s7@7]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s3@3]/Cron[cron-refreshlinks-s3@3]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s2@2]/Cron[cron-refreshlinks-s2@2]/ensure:
created
notice:
/Stage[main]/Mediawiki::Refreshlinks/Mediawiki::Refreshlinks::Refreshlinks::Cronjob[s4@<4]/Cron[cron-refreshlinks-s4 at 4]/ensure:
created
notice: Finished catalog run in 41> seconds
---
root@hume:~# crontab -u mwdeploy -l
# HEADER: This file was autogenerated at Thu Apr 19 13:50:36 +0000 2012 by
puppet.
# HEADER: While it can still be managed manually, it is definitely not
recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name'
should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: <cron-refreshlinks-s6 at 6
0 6 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s5 at 5
0 5 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s1 at 0
0 0 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s7 at 7
0 7 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s3 at 3
0 3 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s2 at 2
0 2 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only
# Puppet Name: <cron-refreshlinks-s4 at 4
0 4 * * * /usr/local/bin/mwscriptwikiset refreshLinks> --dfn-only

15:56 < mutante> !log adding refreshLinks cron jobs to hume per RT-2355 (via
puppet). if there should be any performance issues, schedule can be changed
like
<cluster> at <hour> in mediawiki> (and/or remove mediawiki::refreshlinks from
hume and clear out the jobs of user mwdeploy)

for completeness: http://www.mediawiki.org/wiki/Manual:RefreshLinks.php
http://www.mediawiki.org/wiki/Manual_talk:RefreshLinks.php "I've added a
troubleshooting section as the known memory issues have not been dealt with
here. See also Help:Download @de-Wiki. --DuyTrinh 08:45, 2 November 2011 (UTC)"

added in puppet: write logs to /home/mwdeploy/refreshLinks/
(at least for a while to debug)

it ran fine, afaict, on s2-s7, example log:
arwiki
-------------------------------------
Retrieving illegal entries from pagelinks... 0..0
Retrieving illegal entries from imagelinks... 0..0
Retrieving illegal entries from categorylinks... 0..0
Retrieving illegal entries from templatelinks... 0..0
Retrieving illegal entries from externallinks... 0..0
Retrieving illegal entries from iwlinks... 0..0
Retrieving illegal entries from langlinks... 0..0
Retrieving illegal entries from redirect... 0..0
Retrieving illegal entries from page_props... 0..0
-------------------------------------
---
It went through all wikis on clusters s2 to s7 like that, always getting
"0..0".. This is good, right?
But on s1, it lost connection to MySQL in between:
----
-------------------------------------
enwiki
-------------------------------------
Retrieving illegal entries from pagelinks...
0..100..200..300..400..500..600..700..800..900..1000..1100..1200..1300..1400..1500..1600..1700..1800..1900..2000..2100..2200..2300..2400..2500..2600..2700..2800..2900..3000..3100..3200..3300..3400..3500..3600..3700..3800..3900..4000..4100..4200..4300..4400..4500..4600..4700..4800..4900..5000..5100..5200..5300..5400..5500..5600..5700..5800..5900..6000..6100..6200..6300..6400..6500..6600..6700..6800..6900..7000..7100..7200..7300..7400..7500..7600..7700..7800..7900..8000..8100..8200..8300..8400..8500..8600..8700..8800..8900..9000..9100..9200..9300..9400..9500..9600..9700..9800..9900..10000..10100..10200..10300..10400..10500..10600..10700..10800..10900..11000..11100..11200..11300..11400..11500..11600..11700..11800..11900..12000..12100..12200..12300..12400..12500..12600..12700..Error
in fetchObject(): Lost connection to MySQL server during query (10.0.6.46)
-------------------------------------
---
log files are in:
@hume:/home/mwdeploy/refreshLinks/

for some reason the latest cronjob still setup to run on s1 (which until now we
never saw actually finish, even though it should be able to resume), now failed
with: "mwscript command not found" (uh?) instead of the mysql connection
timeout.
running it again in screen on hume.
any other ideas if it should keep timing out ? Changing the other unproblematic
clusters to regular cronjobs, but just once monthly, as in original request and
leaving the s1 question open.

mwscriptwikiset relied on mwscript being in path. fixing...
but running refreshLinks on s1 still always fails..so kind of stuck

Status changed from 'open' to 'stalled' by dzahn

dzahn added tag hackaton

stalling and adding hackathon tag because s2-s7 work but s1 remains a problem
because of the db connection timeout.

AdminCc hashar added by dvanliere

AdminCc springle added by dvanliere

AdminCc reedy added by dvanliere

dvanliere wrote:

@Springle, @Reedy, @Hashar: can you guys talk to each other and figure out
what's left to be done on this ticket?

Status changed from 'stalled' to 'open' by RT_System

Untaken by dzahn

On Wed Feb 05 20:46:21 2014, dvanliere wrote:

@Springle, @Reedy, @hashar: can you guys talk to each other and figure out
what's left to be done on this ticket?

the thing is that running this on S1 was just always too slow, and timed out,
while we could/can do that on the others and the question is if anything has
changed there meanwhile

"s2-s7 work but s1 remains a problem because of the db connection timeout."
<- this was why we did not run this on s1, so question is what to do against
that db connection timeout