What are you suggesting as the metric of an A/B test? I think we can make improvements, but not sure there will be an easily measurable outcome.

It might be worth considering a general approach of not showing a loading bar for the first Xms, to avoid flashing up a loading bar for relatively quick processes. Conversely, if you go too long without showing the user some change they will think their click wasn't registered successfully.

The standard editor could be a lot slower but since it never tells the user explicitly that it's taking time to load, people don't think it's slow.

I think more specifically what is happening is users mentally discount page reload time, even though the browser shows its own loading bar, and may even blank the page.

Most people on wiki however have this "impression" of how 2017 wikitext editor and VE are slow in general. I have a guess that that impression comes from seeing the loading bar. The standard editor could be a lot slower but since it never tells the user explicitly that it's taking time to load, people don't think it's slow.

Agreed. The current behaviour of showing the loading indicator on the toolbar wasn't really intended to be permanent, and since it's a beta feature I decided it was fine to release the performance improvements before that question was resolved. It dropped off my radar after that. Thanks for bringing it up!

I think Ed is right that A/B testing this will be difficult due to user-perceived performance being a subjective issue.

It might be worth considering a general approach of not showing a loading bar for the first Xms, to avoid flashing up a loading bar for relatively quick processes. Conversely, if you go too long without showing the user some change they will think their click wasn't registered successfully.

The median time to interactive is around 0.642 ms. Why don't we try something around that, like 750ms? I don't know whether that'll be too much or too little, and we can see whether it is or not from trying it out.

I agree about A/B testing this being hard. Not showing the loading bar for first 750ms seems like a good idea.

The current behaviour of showing the loading indicator on the toolbar .... The median time to interactive is around 0.642 ms.

The loading indicator on the toolbar shows until the editor is fully loaded (where the median is more like 1.6s), so if we just delay for 750ms, the toolbar loading bar will still show for another second.

Change 434955 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] DesktopArticleTarget.init: delay appearance of loading bar by 750ms

https://gerrit.wikimedia.org/r/434955

There, a patch in the name of having this be something people can test the experience of.

Testing on the beta cluster:

1. Go to a page, such as an Echo test page
2. Click "Edit source"

A loading bar shows, disappears, then shows again on top of the toolbar, which doesn't really look very good. This change probably needs reverting for now, until more testing can be done?

@Deskana it's not merged yet, so I think what you're saying is that the current state of affairs before-this-patch needs changing.

In T193665#4243287, @DLynch wrote:

@Deskana it's not merged yet, so I think what you're saying is that the current state of affairs before-this-patch needs changing.

Absolutely hilarious. What an amazing demonstration of cognitive bias on my part. Or, is that the whole point of this after all, since we're talking about user-perceived performance? ;-)

@Niharika I think I can help clear up the discrepancy between the slide you saw and why "Most people on wiki however have this 'impression' of how 2017 wikitext editor and VE are slow in general".

In short, the community is right and the slide you saw was wrong. Quite some time ago I captured the data VisualEditor was reporting back. Unsurprisingly, I found that VE was reporting back significantly-low false timings. I informed the WMF that the VE data collection was faulty.

However the problem is a lot worse in VE's wikitext mode. While information was being prepared for that slide you saw, I captured the timing data being reported back by VE's wikitext mode. When trying to edit a long page, the timing being reported back was four seconds. However the true time for the editor to reach to a usable state was over one hundred seconds. It was so slow that it caused multiple time out errors in the browser. I reported all of this. I reported it well before that slide was presented to you and other staff.

Why is known-false data being presented around the office?

There is a formal community consensus against deployment. A text editor never should have been shoved inside the Visual engine. It's over-complicated and buggy, slow to the point of unusable on large pages. We wouldn't be in this hole if it hadn't been built de-facto in secret. We need better communication during the concept and design stage on projects, before any code is written.

Thanks for the information, @Alsee.

@Alsee, I am quite disturbed that you have made multiple omissions in your comment here, as well as an unsupported and inflammatory accusation. If you continue to do this, you can expect that I will not engage with you further.

In T193665#4673675, @Alsee wrote:

@Niharika I think I can help clear up the discrepancy between the slide you saw and why "Most people on wiki however have this 'impression' of how 2017 wikitext editor and VE are slow in general".

In short, the community is right and the slide you saw was wrong. Quite some time ago I captured the data VisualEditor was reporting back. Unsurprisingly, I found that VE was reporting back significantly-low false timings. I informed the WMF that the VE data collection was faulty.

There's two separate topics being conflated here. Let's separate them out.

The first topic is the reports of poor performance of the 2017 wikitext editor. You and many other users, on the English Wikipedia and on other sites, reported performance issues with the 2017 wikitext editor. You neglected to mention that those reports were acknowledged and verified by the Editing team, and led to me creating a goal in Q2 2017-18 to analyse and improve the performance of the 2017 wikitext editor. There've been improvements made to it, but the performance is definitely still unsatisfactory in some areas, most notably on large articles.

The second topic is the data collection for the performance of the 2017 wikitext editor. You've omitted a few bits of information here. So far, you're the only person that has reported these issues, and the team has been unable to verify your report. I personally have tried reproducing your report on your reported platform (Chrome, Windows 7) and was unable to do so, for example. If others have tested and verified issues with the accuracy of the data reporting, I'd very much appreciate you showing me these report so that further investigation can be performed.

However the problem is a lot worse in VE's wikitext mode. While information was being prepared for that slide you saw, I captured the timing data being reported back by VE's wikitext mode. When trying to edit a long page, the timing being reported back was four seconds. However the true time for the editor to reach to a usable state was over one hundred seconds. It was so slow that it caused multiple time out errors in the browser. I reported all of this. I reported it well before that slide was presented to you and other staff.

You've also omitted pertinent information here, in particular that most of your tests were performed before significant performance improvements were made, and that the continued existence of performance issues has been acknowledged by the team. I'd appreciate it if you'd rerun your tests, and provide more up-to-date information which can be investigated.

Why is known-false data being presented around the office?

Because, as I mentioned above, so far all I have to suggest the data is wrong is a single unreproduceable report. I welcome more information being provided.

This is also a loaded question, for what it's worth.

There is a formal community consensus against deployment.

Again, you omitted something, that this is a consensus on the English Wikipedia only, and that consequently there are no plans to make the 2017 wikitext editor the default on the English Wikipedia.

A text editor never should have been shoved inside the Visual engine. It's over-complicated and buggy, slow to the point of unusable on large pages. We wouldn't be in this hole if it hadn't been built de-facto in secret. We need better communication during the concept and design stage on projects, before any code is written.

I readily acknowledge that there are ways that the Wikimedia Foundation's product development process could be improved, but it's patently false to claim that the 2017 wikitext editor was "built de-facto in secret" considering that it has a page on mediawiki.org, a Phabricator tag VisualEditor-MediaWiki-2017WikitextEditor, had published goals relating to it, was documented in quarterly check-ins, had newsletters produced about it, and so on. Please do not make inflammatory and unsupported remarks.

That reminds me, a year or so ago you said you'll do a global request for comment on the deployment of the 2017 wikitext editor. How's that going? Is there any data or information I can provide you?

@Deskana I promise that I did not deliberately omit information. I recall being concerned that my post was already too long, and my main focus was whether the graph was accurate.

So far, you're the only person that has reported [data collection] issues

That should not be surprising. It requires technical knowledge to intercept the data being reported back. Of those people, few would know the timing instrumentation exists. And of those people, approximately none would have reason to check their accuracy.

I'd very much appreciate you showing me these report so that further investigation can be performed.

Regarding VisualEditor timings in general:
I just ran a test using webpagetest.org. If you check the film strip you'll see that VE isn't usable until 23 seconds. If you scroll down and click web request #104 Wikipedia.org-stats.sv you'll see that VE reports back 15488ms. VE under-reported the time by 7.5 seconds. (That looks like a significant improvement, a similar VE tests last year on a smaller article took a lot longer, and it under reported by a more severe percentage.)

Hopefully this qualifies as objective verification that there's a general issue with the timing data sent back by VE.

Regarding VE-wikitext mode:
It appears that the 2017Editor can no longer invoked via logged-out URL(?), so I don't think I can test it from online webtest sites.

I just ran some fresh local tests and I'm getting much better results. Assuming that improvement remains consistent, I'd re-classify the 2017Editor-times from "completely unusable" to "usable". However it's still slower than the current wikitext editor, and these kinds of negative tradeoff appear to be very internally driven at the foundation. Two large test pages took ~10 seconds and ~17 seconds to interactive, while reporting back 7.5 and 11.3 seconds. The wikitext is visible for some seconds before it was actually accessible.

In my report half a year ago, the 2017editor sent back a 4 second timing while overloading the system for 100 seconds. I'm not seeing anything like that in my current quick tests, but clearly the time was being sent back before some part of the process had completed.

I readily acknowledge that there are ways that the Wikimedia Foundation's product development process could be improved

ONE key change. One anti-pattern the foundation keeps acknowledging, but can't seem to escape. The easiest, most important, and most effective way to prevent problems from arising is to get community input before the build phase of major products.

Once an initial version is built, it becomes very hard for the foundation to consider undoing all of that work to make a fundamental design change. It's hard to consider undoing that work just because a few random-yahoos show up with some random issue. The longer work rolls forward, the more deeply invested the foundation becomes. The worst case scenario for everyone is when a deploy announcement evokes a formal-consensus wall.

We could avoid so much cost and conflict and stress if we had effective engagement before the build phase.

it's patently false to claim that the 2017 wikitext editor was "built de-facto in secret" considering that it has a page on mediawiki.org

Please check the creation date on that page, as well as the date coding began. Or I can save you the trouble. Community members caught mention that something was being built and showed up asking if there was a project page or other information available. The answer was no, no project page existed. Some partially-working version had already been built. We couldn't get any useful level of information, and we certainly couldn't give meaningful input.

As far as I'm aware there was no meaningful community input before the project moved into build phase, and the project was opaque during build phase. Is "opaque" an acceptable substitute for "built de-facto in secret"?

That reminds me, a year or so ago you said you'll do a global request for comment on the deployment of the 2017 wikitext editor. How's that going? Is there any data or information I can provide you?

I have no plans at the moment. As long as the foundation takes no action on the project, it's a passive stalemate. The foundation should recognize there's sufficient established consensus that project is de-facto non viable for global deployment without successfully engaging the community first. Enwiki represents nearly half of the wiki-universe, consensus was overwhelming, and many editors elsewhere would surely have similar views. What comes next:

1. The foundation could acknowledge the established consensus and open a dialog trying to sort out how all-of-us are going to resolve the matter. Although the foundation's deep investment in the current plan may make this difficult.
2. The foundation could let the project fade away.
3. The foundation could ignore the community response to the product, announce a roll-out to "silent" wikis on the theory that silence equals approval, and assume EnWiki can be sorted out after the product presumptively-succeeds everywhere else.

In case #3, yes, I may actively invite "silent" wikis to speak up. Possibly individual RFC(s) at Commons and/or Meta and/or German, or a Global-Meta discussion. Action, reaction.

The easy options all sailed away at build-phase.

I just ran a test using webpagetest.org. If you check the film strip you'll see that VE isn't usable until 23 seconds. If you scroll down and click web request #104 Wikipedia.org-stats.sv you'll see that VE reports back 15488ms. VE under-reported the time by 7.5 seconds. (That looks like a significant improvement, a similar VE tests last year on a smaller article took a lot longer, and it under reported by a more severe percentage.)

Purely technical comment here: that data doesn't mean what you think it does. That's logging activation time for VE, not the total pageload time which is what you're comparing it to. This is because otherwise we'd be comparing different things in the "loading a veaction=edit URL from scratch" and "clicking edit button on an already-loaded article" cases.

Eyeballing it from the filmstrip, VE activation starts around 4.5 seconds in, so the actual difference is ~3 seconds. I suspect this remaining difference is the time between a background placeholder editing surface activating, and the real one with the article content being ready -- if so, the difference would vanish on a new-article, though enwiki requires you be logged in for that so I can't provide a direct comparison. I do think we should fix this, if that's what it is.

Important to note here is that we actually do collect much more detailed information on this timing... but it's sampled so only 1/16 of the requests to VE submit that data.

It appears that the 2017Editor can no longer invoked via logged-out URL(?), so I don't think I can test it from online webtest sites.

I'd recommend hitting up the beta site for things like this. E.g. https://simple.wikipedia.beta.wmflabs.org/wiki/Barack_Obama?veaction=editsource

(Though, of course, it doesn't report the timings, so it's not a source for the comparisons as above.)

Previous tests I ran on VE timings with online website test sites, over a year ago, were under reporting by upwards of 20 or 30 seconds. Improvements may have improved the times, but it's not merely an issue of not counting some fixed initial page load time. The time being reported was close to half of actual time.

For comparison:
Running same webtest for the same article shows the Wikitext editor ready at 5.5 seconds. If you suggest discount the first 4.5 seconds as some sort of pre-editor page load time, then the Wikitext editor loads in one second.

@Esanders in T193665#4178535, (sorry, a bit off-topic) a long time ago I was involved in a project that had a considerable delay from a button was clicked and until the action was acknowledged. During testing we saw the actions almost instantaneous, but in real use the delay was a problem. We could not find a really good solution, but one of the better attempts was to change the focus color slightly [to emphasize the incomplete state]. The focus color is the small border that shows something is selected. Another was to show the button in a partly pressed mode.

In Phabricator the button becomes slightly darker when focused, and gets a fuzzy border when activated. In Phabricator we would probably have made the fuzzy border and the button itself slightly greyish until the action was acknowledged, but acknowledged in this context would be a reload so the button would not return to its normal coloring.

In general; buttons are usually handled as a two-state item, but when the delay is considerable, or the action itself is a bit random, you must make them tri-state or more. If the action can be abandoned while the button is activated, then it starts to be really fun. A pressed button waiting for an action is not the same as a released button waiting for an action. Also what does a hard reset of an action means when you can't verify the actual state.

And in short; users may accept delay if they know why something is delayed. Waiting for no cause is bad, as it gives the user an impression they have done something wrong. If the users knows they are waiting for a GoodCause™ then they are (usually) willing to wait longer.

@jeblad are you sure you didn't mean esanders?

Ouch… :D

@jeblad: Don't worry, you're closer to being the one who's "on topic" for this actual ticket. :D

@Alsee:

Previous tests I ran on VE timings with online website test sites, over a year ago, were under reporting by upwards of 20 or 30 seconds. Improvements may have improved the times, but it's not merely an issue of not counting some fixed initial page load time. The time being reported was close to half of actual time.

Yes, I covered that in my reply. I might have been a bit too technical and assumed the implications were obvious, though, so I'll elaborate a bit. :D

There's two things the timing metric you're looking at won't catch, assuming my guess at where it's missing something is correct. First is initial page load time, which is deliberate. Second is the full time to load the article's contents (a network request to parsoid), parse them into the VE data model, and then render them. Instead it'd catch the time until the editor is theoretically ready to render something, while that network request and followup is still going on in parallel.

Given that a bunch of optimization work has happened, mostly focused on that bit which isn't being well measured right now with this metric, I'm inclined to say that results from a year ago aren't terribly relevant to our current discussion. If you can find pages with that kind of disparity now, though, I'd of course be interested. ¯\_(ツ)_/¯

If you suggest discount the first 4.5 seconds as some sort of pre-editor page load time, then the Wikitext editor loads in one second.

My stated reasons for discounting it stand. Also, remember that the load times we're talking about here are the less-common-case; the normal VE activation is on an already-loaded article page, where this "activation time" metric is the only one that matters. (Note that it's then worth keeping the load time factored in to WikiEditor's performance for the comparison, since it's always going to be there.)

I use WikitextEditor 2017 a lot, and I don't see these horrible load times anymore. It could be nice to have them reported though, because *someone* sees them. I've been wondering if some of the newer security measures in the browsers might do weird things with the load times. The site isolation in Google Chrome and Firefox for example. Can this give less caching and thus longer load times?