Page MenuHomePhabricator

Occasional Unexpected offset warnings with one byte off in Pywikibot
Closed, DuplicatePublic

Description

I seem to get occasional warnings that the offset is wrong with chunked uploading:

$ less pwb-bot.log | grep offset
2016-04-23 09:15:30 site.py, 5896 in upload: VERBOSE Old offset: 47000000; Returned offset: 47999999; Chunk size: 1000000
2016-04-23 09:15:30 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 09:28:52 site.py, 5896 in upload: VERBOSE Old offset: 18000000; Returned offset: 18999999; Chunk size: 1000000
2016-04-23 09:28:52 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:08:41 site.py, 5896 in upload: VERBOSE Old offset: 5000000; Returned offset: 5999999; Chunk size: 1000000
2016-04-23 10:08:41 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:16:18 site.py, 5896 in upload: VERBOSE Old offset: 30000000; Returned offset: 30999999; Chunk size: 1000000
2016-04-23 10:16:18 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:24:31 site.py, 5896 in upload: VERBOSE Old offset: 37000000; Returned offset: 37999999; Chunk size: 1000000
2016-04-23 10:24:31 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:45:01 site.py, 5896 in upload: VERBOSE Old offset: 24000000; Returned offset: 24999999; Chunk size: 1000000
2016-04-23 10:45:01 site.py, 5897 in upload: WARNING Unexpected offset.

It's always expected-1

Example upload:
2016-04-23 10:15:47 getty_import.py, 297 in run: INFO File:Jacob van Hulsdonck - Still Life with Lemons, Oranges and a Pomegran
ate - 86.PB.538 - J. Paul Getty Museum.jpg
2016-04-23 10:15:47 getty_import.py, 299 in run: INFO {{User:Multichill/J. Paul Getty Museum

creator=Jacob van Hulsdonck
date=about 1620 - 1640
id=86.PB.538
imageurl=http://www.getty.edu/museum/media/images/web/download/00091101.jpg
location=Q29247
medium=Oil on panel
title=Still Life with Lemons, Oranges and a Pomegranate
url=http://www.getty.edu/art/collection/objects/791/jacob-van-hulsdonck-still-life-with-lemons-oranges-and-a-pomegranate-flemish-about-1620-1640/
wikidata=Q20180102

}}
2016-04-23 10:15:48 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
.....
2016-04-23 10:16:15 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:16 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:17 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:18 site.py, 5896 in upload: VERBOSE Old offset: 30000000; Returned offset: 30999999; Chunk size: 1000000
2016-04-23 10:16:18 site.py, 5897 in upload: WARNING Unexpected offset.
2016-04-23 10:16:18 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:19 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:20 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
......
2016-04-23 10:16:34 api.py, 1925 in submit: VERBOSE Submitting unthrottled action 'upload'.
2016-04-23 10:16:44 site.py, 5900 in upload: WARNING Offset was not supplied. <- this is a different bug, see T133443
2016-04-23 10:16:44 site.py, 5903 in upload: VERBOSE Finished uploading last chunk.
2016-04-23 10:16:54 site.py, 5978 in upload: INFO Upload successful.

This could be a pywikibot or a MediaWiki problem. Pywikibot version: Pywikibot: [https] r-pywikibot-core.git (75134ba, g7223, 2016/04/21, 22:05:31, ok)

Event Timeline

Restricted Application added a project: Multimedia. · View Herald TranscriptApr 23 2016, 10:52 AM
Restricted Application added subscribers: pywikibot-bugs-list, Avicennasis, Steinsplitter, Aklapper. · View Herald Transcript

I bet this is the same issue as T132676. Check and you'll see that the problematic chunks you're sending end with the '\r' (CR) byte.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptApr 23 2016, 11:36 AM
MarkTraceur triaged this task as Normal priority.Dec 5 2016, 5:05 PM
MarkTraceur moved this task from Untriaged to Triaged on the Multimedia board.
MarkTraceur added a subscriber: MarkTraceur.

I don't think this is the Multimedia team's purview, removing tag.

Restricted Application added a project: Multimedia. · View Herald TranscriptJun 5 2017, 3:21 PM