Page MenuHomePhabricator

Occasional Unexpected offset warnings with one byte off in Pywikibot
Closed, DuplicatePublic

Description

I seem to get occasional warnings that the offset is wrong with chunked uploading:

$ less pwb-bot.log | grep offset
2016-04-23 09:15:30 site.py, 5896 in upload: VERBOSE Old offset: 47000000; Returned offset: 47999999; Chunk size: 1000000
2016-04-23 09:15:30 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 09:28:52 site.py, 5896 in upload: VERBOSE Old offset: 18000000; Returned offset: 18999999; Chunk size: 1000000
2016-04-23 09:28:52 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:08:41 site.py, 5896 in upload: VERBOSE Old offset: 5000000; Returned offset: 5999999; Chunk size: 1000000
2016-04-23 10:08:41 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:16:18 site.py, 5896 in upload: VERBOSE Old offset: 30000000; Returned offset: 30999999; Chunk size: 1000000
2016-04-23 10:16:18 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:24:31 site.py, 5896 in upload: VERBOSE Old offset: 37000000; Returned offset: 37999999; Chunk size: 1000000
2016-04-23 10:24:31 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:45:01 site.py, 5896 in upload: VERBOSE Old offset: 24000000; Returned offset: 24999999; Chunk size: 1000000
2016-04-23 10:45:01 site.py, 5897 in upload: WARNING Unexpected offset.

It's always expected-1

Example upload:
2016-04-23 10:15:47 getty_import.py, 297 in run: INFO File:Jacob van Hulsdonck - Still Life with Lemons, Oranges and a Pomegran
ate - 86.PB.538 - J. Paul Getty Museum.jpg
2016-04-23 10:15:47 getty_import.py, 299 in run: INFO {{User:Multichill/J. Paul Getty Museum

creator=Jacob van Hulsdonck
date=about 1620 - 1640
id=86.PB.538
imageurl=http://www.getty.edu/museum/media/images/web/download/00091101.jpg
location=Q29247
medium=Oil on panel
title=Still Life with Lemons, Oranges and a Pomegranate
url=http://www.getty.edu/art/collection/objects/791/jacob-van-hulsdonck-still-life-with-lemons-oranges-and-a-pomegranate-flemish-about-1620-1640/
wikidata=Q20180102

}}
2016-04-23 10:15:48 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
.....
2016-04-23 10:16:15 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:16 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:17 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:18 site.py, 5896 in upload: VERBOSE Old offset: 30000000; Returned offset: 30999999; Chunk size: 1000000
2016-04-23 10:16:18 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:16:18 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:19 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:20 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
......
2016-04-23 10:16:34 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:44 site.py, 5900 in upload: WARNING Offset was not supplied. <- this is a different bug, see T133443
2016-04-23 10:16:44 site.py, 5903 in upload: VERBOSE Finished uploading last chunk.
2016-04-23 10:16:54 site.py, 5978 in upload: INFO Upload successful.

This could be a pywikibot or a MediaWiki problem. Pywikibot version: Pywikibot: [https] r-pywikibot-core.git (75134ba, g7223, 2016/04/21, 22:05:31, ok)

Event Timeline

I bet this is the same issue as T132676. Check and you'll see that the problematic chunks you're sending end with the '\r' (CR) byte.

MarkTraceur triaged this task as Medium priority.Dec 5 2016, 5:05 PM
MarkTraceur moved this task from Untriaged to Triaged on the Multimedia board.
MarkTraceur subscribed.

I don't think this is the Multimedia team's purview, removing tag.