Page MenuHomePhabricator

Bot does not recognize EditConflict
Open, HighPublicBUG REPORT

Description

Bug is reported on my talk page [1]
The edit difference shows ignoring older edits [2] and saving without getting an edit confict:

New timestamp: 20150320115258
blocked user: 91.2.27.113 blocked by Graphikus,
time: 20150320115257 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

blocked user: Huan Schmock blocked by Regi51,
time: 20150320115059 length: unbeschränkt,
reason: Kein Wille zur enzyklopädischen Mitarbeit erkennbar

blocked user: 91.19.70.54 blocked by Doc.Heintz,
time: 20150320115005 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

blocked user: 84.143.218.173 blocked by Graphikus,
time: 20150320114947 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

blocked user: 87.163.31.137 blocked by Logograph,
time: 20150320114449 length: 3 Stunden,
reason: geh bitte draußen spielen

blocked user: 93.220.155.168 blocked by Ukko,
time: 20150320114411 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

blocked user: 91.4.144.201 blocked by Jivee Blau,
time: 20150320113845 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

blocked user: 84.139.213.191 blocked by Jivee Blau,
time: 20150320113807 length: 6 Stunden,
reason: [[WP:Vd|Unsinnige Bearbeitungen]]

@@ -216,2 +216,3 @@
- == [[Benutzer:91.2.27.113]] ==
+ == [[Benutzer:91.2.27.113]] (erl.) ==
- {{Benutzer|1=91.2.27.113}} Unsinnige Bearbeitungen:  [[Special:Diff/14010
8986|1]] [[Special:Diff/140108923|2]] --[[Benutzer:Fish-guts|Fish-guts]]  [[Benu
tzer Diskussion:Fish-guts|Disk]]  [[Benutzer:Fish-guts/Bewertung|+/-]]  [[WP:MP|
MP]] 12:52, 20. Mär. 2015 (CET)
+ {{Benutzer|1=91.2.27.113}} Unsinnige Bearbeitungen:  [[Special:Diff/14010
8986|1]] [[Special:Diff/140108923|2]] --[[Benutzer:Fish-guts|Fish-guts]]  [[Benu
tzer Diskussion:Fish-guts|Disk]]  [[Benutzer:Fish-guts/Bewertung|+/-]]  [[WP:MP|
MP]] 12:52, 20. Mär. 2015 (CET)
+ {{subst:Benutzer:Euku/Vorlage:VM-erl|Gemeldeter=91.2.27.113|Admin=Graphikus|Ze
it=6 Stunden|Begründung=[[WP:Vd{{subst:!}}Unsinnige Bearbeitungen]]|subst=subst:
}}

markiere: [[Spezial:Beiträge/91.2.27.113|91.2.27.113]]
Page [[Wikipedia:Vandalismusmeldung]] saved
defendant: Stauffen, accuser: Anidaat, time: 2015 Mär 20 12:16
waiting 16 seconds.
>> 12:53:31:

page was read and it was saved after another user wrote to the page. Ther was no EditConflict exception or EditConflict Error which would cause the script restart reading the page and try again.

[1] https://de.wikipedia.org/wiki/Benutzer_Diskussion:Xqt#Fehler_im_Bot.3F
[2] https://de.wikipedia.org/w/index.php?title=Wikipedia:Vandalismusmeldung&diff=next&oldid=140109023

Event Timeline

Xqt raised the priority of this task from to High.
Xqt updated the task description. (Show Details)
Xqt added a project: Pywikibot.
Xqt subscribed.
Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptMar 20 2015, 2:50 PM

Was this resolved? Or, have you seen it re-occur?

A doubt.
Before submitting the API request in site.py:

if lastrev is not None:
            params['basetimestamp'] = lastrev.timestamp

The timestamp has been updated to the value of the other edit, so no edit conflict appeared.
Shouldn't basetimestamp be the timestamp of our current page instead the one of the latest revision?

The problem here is that we should timestamp (using touched timestamp when retrieving it) the text that we are going to modify and use that timestamp as basestamp in editpage.
The tricky part to me is to understand when the text that is going to be modified is loaded. First access? When page._text is first created?
Updating timestamp every time page._text is updated is not reliable, as a call to page.latest_revision updates page._text as well, and this might happen also after the bot has modified the intended text.

I did this hack:

oldRawVMText = vmPage.text
rev_id = vmPage.latest_revision_id

for reading and a sanity check before writing:

# sanity check
if vmPage.latest_revision.revid != rev_id:
    print('Revision ID changed')
    raise pywikibot.EditConflict
vmPage.put(newRawText,
           u"Bot: Abschnitt%s erledigt: %s" % (('', 'e')[bool(userOnVMpageFound-1)],
                                               editSummary + openSections +
                                               oldestHeadlineWithOpenStatus),
           False, minorEdit=True, force=True)

With this change, I think this should not happen any more:
https://gerrit.wikimedia.org/r/208574

@Xqt, if you keep your patch and you see a conflict also after pulling this change, please notify.

I've been working on a test case and it shows that there still seems to be a possibility for an EditConflict (or I misunderstood how it works): https://gerrit.wikimedia.org/r/215338

Strange. I tried on en.wikisource but I got an edit conflict.
Might be a wiki-specific setting?

>>> site.login()
Logging in to wikisource:en as MpaaBot
>>> page=pywikibot.Page(site, 'User:Mpaa/x')
>>> page.latest_revision
{'comment': u'test', 'text': u'[[Category:Male authors|test]]\n[[Category:1 births|sortkey]]\n\n{{test|test1}}\n{{test|test1}\n{{test|test1}}', 'revid': 5432692, 'anon': False, 'user': u'Mpaa', 'rollbacktoken': None, 'timestamp': Timestamp(2015, 5, 23, 15, 42, 13), 'minor': False}
>>> site.user()
u'MpaaBot'
>>> page.text = 'reset'

... In the browser I saved an edit as Mpaa ...
https://en.wikisource.org/w/index.php?title=User:Mpaa/x&diff=next&oldid=5432692

>>> page.save(summary='test again')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/squiddy/python/core/pywikibot/tools/__init__.py", line 1102, in wrapper
    return obj(*__args, **__kw)
  File "/home/squiddy/python/core/pywikibot/page.py", line 1061, in save
    cc=apply_cosmetic_changes, **kwargs)
  File "/home/squiddy/python/core/pywikibot/page.py", line 1085, in _save
    raise err
pywikibot.exceptions.EditConflict: Page [[en:User:Mpaa/x]] could not be saved due to an edit conflict

Hmm weird. It now works for me too and it raises the EditConflict exception. Of course it's unusable without two accounts working at the same time though.

This bug is still valid. The following occurred:

  1. page has been written by a user [1]
  2. bot reads that page (via preloading)
  3. page has been changed by same user [2]
  4. bot checks whether page has been changed and saves it [3] but it was the old text

[1] https://de.wikipedia.org/w/index.php?title=Wikipedia%3AAdminwiederwahl%2FHolmium&type=revision&diff=151344424&oldid=151254490
[2] https://de.wikipedia.org/w/index.php?title=Wikipedia:Adminwiederwahl/Holmium&diff=next&oldid=151344424
[3] https://de.wikipedia.org/w/index.php?title=Wikipedia:Adminwiederwahl/Holmium&diff=next&oldid=151431668

the code looks like

text = page.get()
restrictions = page.getRestrictions()
head = ''
text = head + text
old = page.get()
if text != old
pywikibot.showDiff(old, text)
page.put(text)

The second page.get() wasn't forced; it's unclear why it was reloading from live wiki.

Another sample to investigate into this matter:

>>> import pwb, pywikibot as py
>>> s = py.Site()
>>> p = py.Page(s, 'user:xqt/Test')
>>> t = p.text
>>> p.latest_revision_id
151438066

# now change the page online!

>>> r = p.getRestrictions()
>>> tt = p.text
>>> p.latest_revision_id
151438784
>>> t == tt
False

Change 270598 had a related patch set uploaded (by Xqt):
Workarround for overwrite conflict.

https://gerrit.wikimedia.org/r/270598

Change 270598 merged by Xqt:
Workarround for overwrite conflict.

https://gerrit.wikimedia.org/r/270598

Xqt changed the subtype of this task from "Task" to "Bug Report".Apr 12 2021, 8:35 AM

There is another case reported with https://en.wikipedia.org/w/index.php?title=User_talk:Xqt&oldid=1017350521#Bot_bug?

One bot task of Redirect.py edited the page correctly: https://en.wikipedia.org/w/index.php?title=User:1234qwer1234qwer4/Xqbot_bug&diff=prev&oldid=1017183095 but the second task made this malformed change at the same time.

Both bot tasks have the same user and IP.