Page MenuHomePhabricator

[Regression] ‘Empty server response’ on saving through VE when CAPTCHA issued (API response terminated?)
Closed, ResolvedPublic1 Estimated Story Points

Description

A user at Russian Wikipedia gets ‘Empty server response’ error when editing the following article:
https://ru.wikipedia.org/wiki/Проект_«Раковый_геном»

Useragent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/57.0.2987.133 Safari/537.36
Nickname: Элен Теванян
Configuration: Desktop, Visual Editor.

Judging by IRC, the same problem appeared temporarily with user named @Lisawinegar at this page:
https://en.wikipedia.org/wiki/Draft:Kathy_Brodsky

Might be related to T72010.

^7EAAF3F3E33F041F2013B2346660A9776C675C06149B53799E^pimgpsh_fullsize_distr.png (367×539 px, 68 KB)

Event Timeline

I was trying to track this issue down together with @Lisawinegar when she came up with it on IRC yesterday. To quote her, she used "OS X El Capitan and i'm in safari v10,0" and experienced the issue from MobileFrontend wikitext (not visual) edit page¹. When we tried to reproduce this, we were unable to do so - tested all four combinations of wikitext/VE and MobileFrontend/Desktop, it even worked out for her on the very same page (MobileFrontend wikitext) she had tried in the first place, as proven by a test edit with Tag: Mobile web edit on that page. I couldn't reproduce this in any way either, so we agreed it must have been transient and didn't file a bug, until this came up again at some other place today as described in the task.

¹ Don't get confused by that browser information and the usage of MobileFrontend - she used a link that was sent to her, resulting in using the mobile page on desktop.

#wikimedia-tech
[14:16:05] <Waggie> Several people have reported getting "empty server response" errors in -help. Is there something going on we should be aware of?
[14:28:38] <p858snake|_> ^ TimStarling, since I don't know who from the ops team is usually awake at this hour
[14:31:36] <TimStarling> any more details? 5xx rate looks fine
[14:32:19] <TimStarling> what does "empty server response" mean?
[15:22:54] <p858snake|_> TimStarling: I just remembered seeing https://phabricator.wikimedia.org/T164157 yesterday, looks like visual editor possibly
EddieGP triaged this task as High priority.May 1 2017, 12:48 PM

^ As more and more reports are coming in by now, setting priority accordingly. We need to investigate why and how this happens.

I'm having the same problem with a user in -help today.

The user reports trying to create the page "Paula Klien" using a manual translation of the text from the pt.wiki results in the "empty server response" error. Their page creation trips numerous edit filters (https://en.wikipedia.org/w/index.php?title=Special:AbuseLog&wpSearchUser=Pkartwoman) but examining each edit filter reveals that they're all empty (new size, old size and edit size change are all 0 bytes).

They also report disabling VisualEditor does not resolve the situation.

This was also reported in OTRS ticket #2017050110007352; I've requested further details.

See also the reports here: https://en.wikipedia.org/wiki/Wikipedia:VisualEditor/Feedback#Error_saving_data_to_server:_Empty_server_response

Wiki Ed has been getting a lot of messages from instructors and student editors over the weekend, continuing through today. Seems to be hitting en-wiki pretty hard. @Jdforrester-WMF is this VE-specific?

is this VE-specific?

Given that we have no clue were this comes from, the best thing to look for is probably the error message. In a nearly pure vagrant mw folder, role visualeditor enabled:

eddie@eddie-pc:~/develop/mediawiki/mediawiki$ grep -nre "Empty server response"
extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.ArticleTarget.js:731:     this.showSaveError( ve.msg( 'visualeditor-saveerror', 'Empty server response' ), false /* prevents reapply */ );

So as this message is not to be found anywhere in core but only in extensions/VisualEditor and given that all screenshots I've seen so far show the VE, all reports either mention the VE or don't mention anything about which editor was used, chances are that it is VE-specific.

As I'm counting six reports plus a unknown number of

In T164157#3225789, @Ragesoss wrote: Wiki Ed has been getting a lot of messages

and given that even if we find a fix, we won't be able to deploy it until Monday, we should send out a notice about this IMHO.

I've recieved this error as well, using 2 different computers, both windows. What is interesting is that I also get a similar error when previewing source editing: Error loading data from server: Failed to connect. I have a perfect connection and other websites load fine which would give me reason to believe that this is a server-sided error

One instructor reported that "many of them are receiving" the empty server response error message, there are at least two other reports of the same error from other users that Wiki Ed received.

From what I can tell, it seems to be intermittent; I think at least some users were able to edit successfully later, after failing initially. I just made a few edits to pages that I think one user was having trouble with.

They also report disabling VisualEditor does not resolve the situation.

What is interesting is that I also get a similar error when previewing source editing: Error loading data from server: Failed to connect.

Both of you, can you tell what "source editor" was actually used? We do have the plain old wikitext editor in core but also a newer version that is part of the VisualEditor extension. @Rowanharley Your report and error message suggest that you've experienced the issue with the former. If that is really the case, we indeed can drop the idea of searching the issue in VisualEditor and have to look on the API used for previewing & saving edits in core, that both "plain old Core-Editor" and VE use. I'm going to have a look for that other error message.

@EddieGP

I've not been able to speak to the user who was encountering issues, I would treat their assertion that they disabled VisualEditor fully with a pinch of salt. I did have a report this morning of the same issue (so can confirm it's still occurring, as I write this).

I switched to my non-admin account, enabled VisualEditor, and was able to recreate the error. I enabled the Beta 2017 Wikitext editor and was also able to recreate the error. Checking in the Console, I'm getting just the one error.

Failed to set referrer policy: The value 'origin-when-cross-origin' is not one of 'no-referrer', 'origin', 'no-referrer-when-downgrade', or 'unsafe-url'. Defaulting to 'no-referrer'.

Disabling VisualEditor and using the 'proper' source editor (rather than the VisualEditor extension) to make the exact same edit resulted in no such error, and the edit saved correctly (https://en.wikipedia.org/w/index.php?title=Draft:Impossible_(magic_show)&diff=prev&oldid=778310034)

I switched to my non-admin account, enabled VisualEditor, and was able to recreate the error. I enabled the Beta 2017 Wikitext editor and was also able to recreate the error.
Disabling VisualEditor and using the 'proper' source editor (rather than the VisualEditor extension) to make the exact same edit resulted in no such error, and the edit saved correctly (https://en.wikipedia.org/w/index.php?title=Draft:Impossible_(magic_show)&diff=prev&oldid=778310034)

That, in addition with the error message @Rowanharley reported ("Error loading data from server:") which is only to be find within the translation files of VisualEditor (but not core) suggests that this is about VE (including modern wikitext editor). I wasn't able to reproduce this on testwiki, but I'll try again on enwiki

Checking in the Console, I'm getting just the one error.
Failed to set referrer policy: The value 'origin-when-cross-origin' is not one of 'no-referrer', 'origin', 'no-referrer-when-downgrade', or 'unsafe-url'. Defaulting to 'no-referrer'.

Thanks for looking that up. Actually I don't know what that means or if it is helpful - but still good to have that information. I'll have to look up what that is meant to tell me.

As I'm actively trying to track this down (and if I can solve it), I'm grabbing this task now, but please anybody feel free to work on it, having this resolved sooner rather than later will be a good thing, and I honestly don't care if somebody else does so before me.

I think this might be CAPTCHA related. When I try making an edit on an unconfirmed account that adds links (that in the normal text editor would require CAPTCHA), I get this error message. Will test further.

EDIT: Did some further testing and quite confident this is the issue. I only get the error when adding a link that requires CAPTCHA; adding a wikipedia URL doesn't require captcha and doesn't give me the error, but other links do. This is consistent with the posts reported above too ("I can successfully post changes if I don't include a citation to a news article", and Nick's test edit which didn't save from his non-confirmed account).

Also reported on de.wiki and mediawiki.org.

Reported on fr.wp. People managed to avoid this message when they weren't using a specific reference. I'm waiting for clarifications.

This definitely sounds like an issue with displaying CAPTCHA in VE.

Jdforrester-WMF raised the priority of this task from High to Unbreak Now!.
Jdforrester-WMF set the point value for this task to 1.
Jdforrester-WMF moved this task from To Triage to TR0: Interrupt on the VisualEditor board.

Looking into this now.

As I'm actively trying to track this down (and if I can solve it), I'm grabbing this task now,

Please don't ever do this again for urgent tasks. By claiming the task you hid it from my triage process and delayed us fixing it.

Jdforrester-WMF renamed this task from ‘Empty server response’ when editing to [Regression] ‘Empty server response’ on saving through VE when CAPTCHA issued (API response terminated?).May 2 2017, 2:57 PM

All changes in ConfirmEdit that went out in wmf.21:

Neither of those look like they'd have caused this.

All changes to VE's init/integration that went out in wmf.21:

I'm perplexed.

@Jdforrester-WMF

Please don't ever do this again for urgent tasks. By claiming the task you hid it from my triage process

I'm sorry that this doesn't conform with your teams practices and of course won't do so again. That said, your comment doesn't sound friendly to me and I don't really buy the "and delayed us fixing it" part, given that this task is open since Saturday and your comment comes in not even three hours after me grabbing that task. I won't blame you for not noticing this for two and a half days (especially as it was created on the weekend), so please don't blame me in a comment you're posting less than three hours after me grabbing that task. All I've done was trying to help. Thanks.

Change 351324 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/VisualEditor@master] Follow-up 91a75e3fe: Respect the new signature for saveFail() everywhere

https://gerrit.wikimedia.org/r/351324

This was the one; it failed to adjust the code for non-badToken save failures.

Change 351326 had a related patch set uploaded (by Esanders; owner: Esanders):
[mediawiki/extensions/VisualEditor@master] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/351326

Change 351324 abandoned by Jforrester:
Follow-up 91a75e3fe: Respect the new signature for saveFail() everywhere

Reason:
Let's do Ed's then.

https://gerrit.wikimedia.org/r/351324

Change 351326 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/351326

Change 351329 had a related patch set uploaded (by Jforrester; owner: Esanders):
[mediawiki/extensions/VisualEditor@wmf/1.29.0-wmf.21] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/351329

@Jdforrester-WMF: If non-team members working on important bugs creates an issue then your triage process is broken and something you should fix.

Change 351329 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@wmf/1.29.0-wmf.21] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/351329

Mentioned in SAL (#wikimedia-operations) [2017-05-02T17:05:20Z] <catrope@naos> Synchronized php-1.29.0-wmf.21/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.ArticleTarget.js: T164157 (duration: 01m 00s)

Jdforrester-WMF assigned this task to Esanders.
Jdforrester-WMF removed a project: Patch-For-Review.

This is now deployed to production everywhere; very sorry for the inconvenience to all affected users.

Change 388753 had a related patch set uploaded (by MacFan4000; owner: Esanders):
[mediawiki/extensions/VisualEditor@REL1_29] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/388753

Change 388753 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@REL1_29] Follow-up I488fd7e74: Adjust saveFail calls for new function signature

https://gerrit.wikimedia.org/r/388753