Page MenuHomePhabricator

cronspam cleanup: Cron <www-data@terbium> /usr/local/bin/foreachwiki maintenance/cleanupUploadStash.php > /dev/null
Closed, ResolvedPublic

Description

This task has initially been created in the restricted nda security settings due to its listing of cron output, including failed deletion output. Since some of these may have been deleted for cause, it seems rash to push them immediately into a public task.

It seems that there is cronspam every single day for the result of this script, even when it contains nothing but

Set $wgShowExceptionDetails = true; in LocalSettings.php to show detailed debugging information.

Other times, it has some detailed information:

Array
(

[0] => Array
    (
        [0] => backend-fail-delete
        [1] => mwstore://local-swift-eqiad/local-temp/7/77/20161106204334!chunkedupload_3190ca6609a4.jpg
    )

)

Array
(

[0] => Array
    (
        [0] => backend-fail-delete
        [1] => mwstore://local-swift-eqiad/local-temp/a/a7/20161106121632!56T95C.jpg
    )

)

Array
(

[0] => Array
    (
        [0] => backend-fail-delete
        [1] => mwstore://local-swift-eqiad/local-temp/b/b3/20161106132354!fzbSTf.jpg
    )

)

Array
(

[0] => Array
    (
        [0] => backend-fail-delete
        [1] => mwstore://local-swift-eqiad/local-temp/d/df/20161106105330!chunkedupload_c838e77da4ec.jpg
    )

)

Array
(

[0] => Array
    (
        [0] => backend-fail-delete
        [1] => mwstore://local-swift-eqiad/local-temp/f/fe/20161106210557!6h6s6Z.jpg
    )

)

Set $wgShowExceptionDetails = true; in LocalSettings.php to show detailed debugging information.

Ideally, rather than output daily, this should ONLY output when there are errors.

Event Timeline

RobH created this task.Nov 9 2016, 9:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 9 2016, 9:38 PM
fgiunchedi triaged this task as Medium priority.Nov 30 2016, 2:07 AM

Can someone give some background of what is these emails are about? Not asking for a fix, just for context. Is it some kind of swift deletion log?

Joe added a subscriber: Joe.Jun 18 2018, 6:08 AM

I actually think the best way to handle this is to add a redirection of stderr and stdout to files, and to properly logrotate them.

Joe changed the visibility from "Custom Policy" to "Public (No Login Required)".Jun 18 2018, 6:14 AM
Restricted Application added projects: Commons, Multimedia. · View Herald TranscriptJun 18 2018, 6:25 PM
jijiki moved this task from Inbox 🐅 to Radar 📻 on the User-jijiki board.Oct 29 2018, 5:16 PM

Change 470877 had a related patch set uploaded (by Thifranc; owner: Thifranc):
[operations/puppet@production] puppet:Reduce cronspam from modules/mediawiki/

https://gerrit.wikimedia.org/r/470877

@Joe, we are trying to fix this with @thifranc. Output of most cronjobs have no logrotation i.e. we just keep the output of the last run. Adding proper logging and logrotation to this job, will make it special :). Secondly, if we redir stderr to logfiles as well, we will miss it when something won't go as expected. What do you think?

@Joe: Could you please reply to the last comment when you find time? Thanks!

jijiki added a comment.Dec 6 2018, 2:28 PM

@Aklapper We have resolved it on IRC and commented on gerrit about what is left about this task

Dzahn added a comment.Dec 8 2018, 12:13 AM

@Aklapper We have resolved it on IRC and commented on gerrit about what is left about this task

@thifranc amended the change as we suggested, looks good to me. can we merge, Effie?

Change 470877 merged by Effie Mouzeli:
[operations/puppet@production] puppet:Reduce cronspam from modules/mediawiki/

https://gerrit.wikimedia.org/r/470877

jijiki closed this task as Resolved.Jan 8 2019, 12:08 PM
jijiki claimed this task.

I will reopen if something goes wrong, thank you @thifranc !

Dzahn awarded a token.Jan 8 2019, 4:28 PM