Message "Script terminated successfully", but upload failed
Open, Needs TriagePublicFeature
Actions

Assigned To

None

Authored By

	Yann
	Jun 13 2023, 3:54 PM

Description

After T338603, I also got

Several pages of gibberish, and then

xff\xf9\xf8\xb2o\x87\xaf\x97\xcb~\xba\x1e\x9f\x0f\xd7\xed\xe1{\xdd\x8e\xfb\xd3\xf3i\xbf\xeco\xaf_/\x87};\xbf\x9e\x7f\xcc\xfd\xf8\xf8\xcb\x7f\x04\x18\x00~\xd8\xef\xb9\nendstream\nendobj\nxref\n0 188 \n0000000000 65535 f \n0000000015 00000 n \n0000003555 00000 n \n0000003645 00000 n \n0000003733 00000 n \n0000017739 00000 n \n0000017832 00000 n \n0000017962 00000 n \n0000018474 00000 n \n0000018570 00000 n \n0000019919 00000 n \n0000029389 00000 n \n0000029490 00000 n \n0000029584 00000 n \n0000029676 00000 n \n0000029807 00000 n \n0000170158 00000 n \n0000170671 00000 n \n0000171184 00000 n \n0000171276 00000 n \n0000172540 00000 n \n0000172636 00000 n \n0000174495 00000 n \n0000174625 00000 n \n0000174716 00000 n \n0000174806 00000 n \n0000174896 00000 n \n0000174986 00000 n \n0000187586 00000 n \n0000187677 00000 n \n0000187766 00000 n \n0000188279 00000 n \n0000188938 00000 n \n0000189043 00000 n \n0000310179 00000 n \n0000312988 00000 n \n0000313079 00000 n \n0000313168 00000 n \n0000313260 00000 n \n0000313527 00000 n \n0000313841 00000 n \n0000313938 00000 n \n0000314015 00000 n \n0000314104 00000 n \n0000315963 00000 n \n0000316228 00000 n \n0000316319 00000 n \n0000316411 00000 n \n0000316503 00000 n \n0000316599 00000 n \n0000316690 00000 n \n0000317836 00000 n \n0000319695 00000 n \n0000321106 00000 n \n0000727593 00000 n \n0000727723 00000 n \n0000727820 00000 n \n0000733911 00000 n \n0000853667 00000 n \n0000855526 00000 n \n0000855618 00000 n \n0000855883 00000 n \n0000856396 00000 n \n0000858255 00000 n \n0000988632 00000 n \n0000988763 00000 n \n0000988852 00000 n \n0000988943 00000 n \n0001097857 00000 n \n0001099716 00000 n \n0001099808 00000 n \n0001099899 00000 n \n0001099996 00000 n \n0001100509 00000 n \n0001101022 00000 n \n0001101119 00000 n \n0001101211 00000 n \n0001101310 00000 n \n0001101399 00000 n \n0001101530 00000 n \n0001102874 00000 n \n0001104234 00000 n \n0001104325 00000 n \n0001104456 00000 n \n0001106315 00000 n \n0001106581 00000 n \n0001106672 00000 n \n0001115242 00000 n \n0001115372 00000 n \n0001115593 00000 n \n0001124029 00000 n \n0001221312 00000 n \n0001396912 00000 n \n0001397425 00000 n \n0001397515 00000 n \n0001402903 00000 n \n0001403009 00000 n \n0001404868 00000 n \n0001541295 00000 n \n0001543154 00000 n \n0001543667 00000 n \n0001948300 00000 n \n0001948322 00000 n \n0001950424 00000 n \n0001950555 00000 n \n0001950860 00000 n \n0001954379 00000 n \n0001954510 00000 n \n0002085878 00000 n \n0002085980 00000 n \n0002205528 00000 n \n0002205700 00000 n \n0002206980 00000 n \n0002207072 00000 n \n0002207170 00000 n \n0002207263 00000 n \n0002208721 00000 n \n0002208852 00000 n \n0002210712 00000 n \n0002220804 00000 n \n0002220901 00000 n \n0002228755 00000 n \n0002228853 00000 n \n0002228951 00000 n \n0002229082 00000 n \n0002230264 00000 n \n0002230531 00000 n \n0002230624 00000 n \n0002237995 00000 n \n0002238509 00000 n \n0002238601 00000 n \n0002238692 00000 n \n0002238998 00000 n \n0002241444 00000 n \n0002241542 00000 n \n0002241640 00000 n \n0002243097 00000 n \n0002243376 00000 n \n0002243466 00000 n \n0002243564 00000 n \n0002245035 00000 n \n0002247332 00000 n \n0002247425 00000 n \n0002247695 00000 n \n0002248209 00000 n \n0002248340 00000 n \n0002248430 00000 n \n0002248525 00000 n \n0002248615 00000 n \n0002375395 00000 n \n0002375485 00000 n \n0002375748 00000 n \n0002375838 00000 n \n0002376352 00000 n \n0002528536 00000 n \n0002528629 00000 n \n0002528805 00000 n \n0002528903 00000 n \n0002528995 00000 n \n0002530594 00000 n \n0002530860 00000 n \n0002531374 00000 n \n0002531471 00000 n \n0002532803 00000 n \n0002533317 00000 n \n0002533515 00000 n \n0002535375 00000 n \n0002537235 00000 n \n0002537328 00000 n \n0002546029 00000 n \n0002546119 00000 n \n0002546209 00000 n \n0002547419 00000 n \n0002547615 00000 n \n0002547930 00000 n \n0002548032 00000 n \n0002548127 00000 n \n0002548219 00000 n \n0002548317 00000 n \n0002548415 00000 n \n0002703059 00000 n \n0002703190 00000 n \n0002703283 00000 n \n0002703548 00000 n \n0002703638 00000 n \n0002703742 00000 n \n0002705602 00000 n \n0002877296 00000 n \ntrailer\n<</Size 188/Length 597/Root 41 0 R/Info 155 0 R/ID[(\xe4\xb9\x1d`p\xcdf\xe8\xcf\xec.\x15@\xbf\x91\xde\x07\xdfT\xb4\x8cD\xf2c\xa2\x0e\xdf\xe0W\xbbt\xeb)(|z\x8b\xd7j\xeb*M\xb7\xa6\\r\xd9\xc4&\x82\xcb)]>>\nstartxref\n2879156\n%%EOF\n\n--===============3482324537353214119==--\n';
 stasherrors: [{'message': 'uploadstash-exception', 'params': ['UploadStashBadPathException', "Path doesn't exist."], 'code': 'uploadstash-exception', 'type': 'error'}];
 servedby: mw1483;
 help: See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/&gt; for notice of API deprecations and breaking changes.]
1 read operation
Script terminated successfully.

The issue here is mainly that the file DIDN'T upload, although the bot produces a success message.

The script command was
$PYTHON ~/core_stable/pwb.py upload "$filename" $sim $chunk -noverify -filename:"$uploadname" -keep -abortonwarn -descfile:"$desc"

Related Objects

Mentioned In: T338603: "The write operation timed out" with Pywikibot
Mentioned Here: T338603: "The write operation timed out" with Pywikibot

Event Timeline

Yann created this task.Jun 13 2023, 3:54 PM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJun 13 2023, 3:54 PM

The beginning of this message would be important, looks like an mw API warning. The "Script terminated successfully" only indicated whether the script does not terminated with an exception. It is not a summary of processed threads. Processed threads are shown by Counter values; here only one read operation but no write or upload was made.

Xqt added a comment.Jun 13 2023, 11:00 PM

This comment was removed by Xqt.

@Yann: what was your script command?

Xqt added a project: Upstream.Jun 13 2023, 11:09 PM

See https://www.mediawiki.org/wiki/Topic:Xka2vta4siuk04nj
It seems Pywikibot was coded this way, which seems quite weird to me. How to change this to a feature request?

Xqt updated the task description. (Show Details)Jun 26 2023, 10:01 AM

As described above the message "Script terminated successfully" only indicated whether the script does not terminated with an exception. I would not like to change this behaviour. Consider for example this case: pwb add_text test -page:Foo -page:Bar which tries to add "test" to the both pages "Foo" and "Bar" and assume that "Bar" has a {{nobots}} template in it. The script would exit with something like this:

2 read operations
1 write operation

Execution time: 12 seconds
Read operation time 6 seconds
Write operation time 5 seconds
Script terminated successfully.

This script is successfully even if one write operation failed. The failing operation is indicated by a warning. (I aggree that the warning above is ugly!) The only way I can suggest is to use ignore_save_related_errors set to False which would raise a PageSaveRelatedError and terminate the script immediately by: Script terminated by exception: PageSaveRelatedError. Maybe this ignore_save_related_errors setting could be a config setting and can be overridden by command line.

Xqt changed the subtype of this task from "Task" to "Feature Request".Jun 26 2023, 10:29 AM

I managed to do that

retry_count = 3  # Number of upload retries
success = False

while retry_count > 0 and not success:
    try:
        filepage = pywikibot.FilePage(site, key)
        filepage.upload(
            source=filename,
            comment=desc_content,
            text=desc_content,
            chunk_size = "5M",
            ignore_warnings=False,
        )
        success = True
    except Exception as e:
        print("Upload failed. Retrying...")
        retry_count -= 1

if not success:
    print("Upload failed after retries.")

In T338969#8964613, @Yann wrote:

I managed to do that

...

Retrying uploads is a good idea. I'm just wondering why this hasn't been implemented before.

Message "Script terminated successfully", but upload failedOpen, Needs TriagePublicFeatureActions

Description

Related Objects

Event Timeline

Message "Script terminated successfully", but upload failed
Open, Needs TriagePublicFeature
Actions