Page MenuHomePhabricator

Output of wmf-beta-update-databases.py is not clear on errors
Closed, ResolvedPublic

Description

For example, I got:

Traceback (most recent call last):
  File "/usr/local/bin/wmf-beta-update-databases.py", line 98, in <module>
    sys.exit(main())
  File "/usr/local/bin/wmf-beta-update-databases.py", line 95, in main
    run_updates(dblist, args.batch)
  File "/usr/local/bin/wmf-beta-update-databases.py", line 51, in run_updates
    do_wait(procs)
  File "/usr/local/bin/wmf-beta-update-databases.py", line 32, in do_wait
    raise Exception("command: ", cmd, "output: ", f.read())
Exception: ('command: ', '/usr/local/bin/mwscript update.php --wiki=hewiki --quick', 'output: ', '')

There is no other output for that wiki, so I have no idea what happened (the last output is for fawiki).

See also T111267: update.php broken in Beta where it's again unclear.

Related Objects

Event Timeline

Mattflaschen-WMF updated the task description. (Show Details)
Mattflaschen-WMF raised the priority of this task from to Needs Triage.
Restricted Application added subscribers: Luke081515, Aklapper. · View Herald TranscriptAug 26 2015, 10:02 PM
Mattflaschen-WMF set Security to None.
mmodell triaged this task as Low priority.
mmodell added a subscriber: mmodell.
hashar added a subscriber: hashar.Feb 16 2016, 9:15 AM
if p.wait() > 0:
    raise Exception("command: ", cmd, "output: ", f.read())
f.seek(0)
print f.read().strip()
f.close()

The f.read() call yields nothing since the file pointer is at the end ... Should seek to beginning as seen at the line below.

Anyway, I don't think we need a tmp file.

Change 270902 had a related patch set uploaded (by Hashar):
beta: update db script strip output on error

https://gerrit.wikimedia.org/r/270902

hashar claimed this task.Feb 16 2016, 9:30 AM
hashar moved this task from Backlog to In-progress on the Beta-Cluster-Infrastructure board.
greg added a subscriber: greg.Mar 25 2016, 8:10 PM

This is happening again...

19:20:41 Traceback (most recent call last):
19:20:41   File "/usr/local/bin/wmf-beta-update-databases.py", line 98, in <module>
19:20:41     sys.exit(main())
19:20:41   File "/usr/local/bin/wmf-beta-update-databases.py", line 95, in main
19:20:41     run_updates(dblist, args.batch)
19:20:41   File "/usr/local/bin/wmf-beta-update-databases.py", line 51, in run_updates
19:20:41     do_wait(procs)
19:20:41   File "/usr/local/bin/wmf-beta-update-databases.py", line 32, in do_wait
19:20:41     raise Exception("command: ", cmd, "output: ", f.read())
19:20:41 Exception: ('command: ', '/usr/local/bin/mwscript update.php --wiki=enwikibooks --quick', 'output: ', '')

Just to be clear, this seems like just an error in how we handle the log file, not the actual update, so we *should* be OK, right?

The beta-update-databases job has been failing (with this error) since 25 March at 18:20: https://integration.wikimedia.org/ci/view/Beta/job/beta-update-databases-eqiad/

Just to be clear, this seems like just an error in how we handle the log file, not the actual update, so we *should* be OK, right?

No, the update is also failing. See T111267: update.php broken in Beta for an example of a real problem ("This is because MWNamespace::exists( 100 ) is false (there is no such namespace on enwiki Beta currently).") that triggered this.

Change 282441 had a related patch set uploaded (by Mattflaschen):
Fix Beta update.php error handling

https://gerrit.wikimedia.org/r/282441

Anyway, I don't think we need a tmp file.

If it runs in parallel without a tmp file, are you sure it won't interleave the output (mixing output from enwiki and hewiki, etc.)?

Change 270902 abandoned by Hashar:
beta: update db script strip output on error

Reason:
Looks like https://gerrit.wikimedia.org/r/#/c/282441/ is easier :-}

https://gerrit.wikimedia.org/r/270902

Change 282441 merged by Filippo Giunchedi:
Fix Beta update.php error handling

https://gerrit.wikimedia.org/r/282441

Mattflaschen-WMF closed this task as Resolved.May 12 2016, 11:16 PM