Context

Upon running the mediamoderation maintenance script
mwscript extensions/MediaModeration/maintenance/ModerateExistingFiles.php [...]
to scan images on commons T258603: Scan all images on Wikimedia Commons
it is difficult to understand the status of the the script and related jobs that it intiates. @mepps and @eigyan reported that the script was not working, but that it did not provide useful progress or error output at the shell running the script. All the operator can tell from the cli is that the script is running in the foreground, but not if it is doing useful work or if it is stalled. Also, there are a number of 'post-script run' steps that the person executing the script must perform to try to ascertain the status of the process. It would be great to find ways to provide some of that manually-gathered information directly from the script output as well.

Questions to answer

1. What are the the organizational units of processing within the code that should have status output?
   • examples:
     • batches for the job runner
     • buckets of images to check in the external apis
     • individual files that need checked
2. For each item identified in question 1, what progress and/or status should be surfaced via the cli when running the script?
   • examples:
     • progress: number of batch jobs run/number of batch jobs requested; timestamp of last first file checked / timestamp of last file checked; number of files checked vs number requested, etc
     • task success/failure: job failed, file scan failed, etc
     • content moderation results: number of files that passed moderation, number of files that failed moderation due to matching known bad content
3. For each item identified in question 1, what identifiers/fields/additional information can and should be included directly in the script output or used to look up information in other systems such as logstash or mwlog1002?
   • examples (to be clear, I don't know if these are available to the host running the script or not, so any items here should be checked to make sure they are actually available -- jason):
     • jobs: job id, job status, tracking id, exception
     • files: filenames, id, etc (I don't actually know what file info is available --jason)
     • api requests: request ids, response status, response message
4. What would the steps be for proposed new Running the MediaModeration maintenance script instructions that would incorporate the new script output to guide the operator through running the script and understanding its status?

Answers

What are the the organizational units of processing within the code that should have status output?

• Batches

For each item identified in question 1, what progress and/or status should be surfaced via the cli when running the script?

• Indication for each batch that has started, for example, by outputting a "dot" (.) at the initiation of each batch, or perhaps an updated count of batches started depending on if any enhancements are implemented.
• Optional enhancement: batch exception output

For each item identified in question 1, what identifiers/fields/additional information can and should be included directly in the script output or used to look up information in other systems such as logstash or mwlog1002?

• None, we only need to indicate that a batch started.
• Optional enhancement: timestamp for each batch start

What would the steps be for proposed new Running the MediaModeration maintenance script instructions that would incorporate the new script output to guide the operator through running the script and understanding its status?

• No change.

Original Description

After starting the job, it's unclear what's happening.

Two things that would help:

• Allow for console version for single file
• Output job creation and give more feedback