[INVESTIGATION] Evaluate speed of key interactions on Wikibase.dev
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Samantha_Alipio_WMDE
	Dec 20 2021, 12:26 PM

Description

As a user of Wikibase.cloud, I want my wikis to be responsive so that my everyday editing or reading tasks can be completed smoothly and successfully.

Currently, we are encountering certain "slow" interactions on Wikibase.dev (and therefore Wikibase.cloud). Users of WBstack.com have also noted that certain areas of that service could benefit from speed improvements.

To ensure that Wikibase.cloud is adequately performant on initial launch, we should investigate the following while logged into the wiki as a user:

Interaction	Current Speed	Wikidata Speed	Target Speed	Notes
Load the main page of the wiki	1683	1126
Type a QID or item label into search and be suggested a match	687	494
Load an entity page (e.g. Q1) containing statements	1225	853		please also note if there is a delay between page content loading and when the edit icon appears next to a statement
Open the add statement console by clicking +add statement	94	117
Save a statement	3648	602
New item page is displayed after clicking Create on Special:NewItem	8458	1316
Results of a simple query are displayed in the Query Service	2725
Results of a complex query are displayed in the Query Service	N/A

The outcome of this initial investigation will be that the above table is completed with [something like] the average of 3 speed measurements of each interaction (current speed column). Target speed may be defined in collaboration with the UX team. This investigation help to inform a subsequent investigation and/or task focused on improving the speed of Wikibase.cloud.

N.B: for these measurements it's sufficient to use the "developer tools" of your browser.

Related Objects

Mentioned In: T299522: [INVESTIGATION 16hrs] Speed improvements for saving statements and creating new items
T299504: Decide on speed targets for the initial launch of Wikibase.cloud
T299503: [EPIC] Make initial Wikibase.cloud speed improvements before launch

Event Timeline

• Samantha_Alipio_WMDE created this task.Dec 20 2021, 12:26 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 20 2021, 12:26 PM

• Samantha_Alipio_WMDE updated the task description. (Show Details)Dec 20 2021, 1:01 PM

• Samantha_Alipio_WMDE updated the task description. (Show Details)Dec 20 2021, 1:28 PM

• Samantha_Alipio_WMDE updated the task description. (Show Details)Dec 20 2021, 1:35 PM

• BilalShirwani closed this task as a duplicate of T298058: [Impact Analysis] Evaluate Impact of Mobile Reply and New Discussion Tools.Dec 21 2021, 12:03 AM

MusikAnimal reopened this task as Open.Dec 21 2021, 12:17 AM

Moving straight to ready to pick up because this task doesn't require breakdown. It's just collecting some data.

Tarrow moved this task from Ready to Pick Up to Doing on the Wikibase Cloud board.Dec 21 2021, 5:09 PM

Tarrow claimed this task.Dec 22 2021, 9:46 AM

	Current Time for action (ms)
	1	2	3	AVG
Load the main page of the wiki (time to fire on load event)	1814	1552	1684	1683.333333
Type a QID or item label into search and be suggested a match (time between from text string visible to result visible)	830	740	490	686.6666667
Load an entity page (e.g. Q1) containing statements (time to fire onload event)	1135.5	1240	1299	1224.833333
Open the add statement console by clicking +add statement (using "frame"	100	83	100	94.33333333
Save a statement (click save - formatted statement and edit button appearing)	4384	3761	2800	3648.333333
New item page is displayed after clicking Create on Special:NewItem	7707	8600	9067	8458
Results of a simple query are displayed in the Query Service (select all statements from "play button colour change")	3362	2422	2392	2725.333333

I decided that "a complex query" was probably too vague and therefore didn't investigate trying to orchestrate something like this.

All results determined using google chrome performance tools tab of developer tools. On https://coffeebase.wikibase.dev/

Tarrow updated the task description. (Show Details)Dec 22 2021, 11:37 AM

Tarrow moved this task from Doing to Review on the Wikibase Cloud board.Dec 22 2021, 6:16 PM

So, trying to reproduce the above.

For the page load timings:

Turn off caching
Use this button

I get similar numbers:

	Current Time for action (ms)
		1	2	3	AVG
Load the main page of the wiki (time to fire on load event)	1296	1527

So it looks like this would be repeatable.

Not sure how much value there is in the Open the add statement console by clicking +add statement (using "frame" metric.

In the long run we might be better off enabling metric collection from MediaWiki (including Wikibase) etc.

We may also want to dig into why New item page is displayed after clicking Create on Special:NewItem is so slow!

Deniz_WMDE removed Tarrow as the assignee of this task.Jan 4 2022, 9:09 AM

Deniz_WMDE moved this task from Done to Review on the Wikibase Cloud board.

@Tarrow and I tried scaling up some services to see if speed would improve.
https://github.com/wmde/wbaas-deploy/commit/5b148a70bd8c06b326cc5998d79e2b20b81514be
https://github.com/wmde/wbaas-deploy/commit/ea0e17ef87c566bad13d98fa9a0fe191e65e8822
https://github.com/wmde/wbaas-deploy/commit/221111d03ead0c926faf4d7ee106cc54becdbdde
https://github.com/wmde/wbaas-deploy/commit/062f1381df0675f52616511ddf6d34c9cfddd47a
https://github.com/wmde/wbaas-deploy/commit/8c0ccd953ee441a87ce950c1f87a2123e6e0fb74

We came to the conclusion nothing got significantly faster, and speed improvements probably have to come on a case by case basis and via caching enabling etc.

We then scaled back the things

I've been having a chat with @Jan_Dittrich. It is worth us certainly comparing these metrics to similar ones from wikidata.org. Firstly because if wikidata.org has optimised to this level then maybe we can copy the method. Secondly most of our users come with some familiarity with wikidata so we wouldn't want to be "worse" than it.

Generally we should strive for each action to be less than one (1) second in the short term. In the long term we should keep in mind that 1/10 of this (100ms) is closer to what a user would experience as "immediate".

Tarrow claimed this task.Jan 4 2022, 12:22 PM

Tarrow moved this task from Review to Doing on the Wikibase Cloud board.

Here some background on what we talked about:

As general heuristics there are the thresholds of

<0.1s	= immediate reaction
<1s	= "in control", no "waiting" feeling
>1s	= feels slow, waiting
>10s	= what is going on? Lets do something else.

This is based upon this article: https://www.nngroup.com/articles/response-times-3-important-limits/

So what can be done if one is below the threshold needed?

Speed the system up to get under the threshold but if that leads to diminishing returns…
Show a stand-in system reaction. You know these: Boxes instead of content, waiting indicators. Just take care that it does not lead to screen flicker all over the place when the "real" content is there (e.g. fade in)

For Wikibase, it probably matters that:

New pages appear within a second (not necessarily with all interaction) so people experience that they navigate seamlessly.
There should be a reaction to clicks, input fields etc. within 0.1s
Larger tasks that can’t be reduced below or near to 1s should have some stand-in to show that the system is reacting.

For wikibase cloud we need to consider that the distribution of frequency/impact is probably different than on Wikidata since people probably do more writing and less reading operations when experimenting with the system.

Similar work done for Wikidata (and test.wikidata.org). Simple query skipped since it would return faaar more results there than on wikibase.dev

	Current Time for action wikidata (ms)		(test.wikidata.org for new item creation)
	1	2	3	AVG
Load the main page of the wiki (time to fire on load event)	943	1032	1403	1126
Type a QID or item label into search and be suggested a match (time between from text string visible to result visible)	560	380	543	494.3333333
Load an entity page (e.g. Q1/Q4115189) containing statements (time to fire onload event)	812	915	833	853.3333333
Open the add statement console by clicking +add statement (using "frame"	85	146	120	117
Save a statement (click save - formatted statement and edit button appearing)	700	491	615	602
New item page is displayed after clicking Create on Special:NewItem	1481	1208	1260	1316.333333

Tarrow updated the task description. (Show Details)Jan 6 2022, 11:14 AM

Addshore moved this task from Doing to Launch Sprint 1 (2022) on the Wikibase Cloud board.Jan 13 2022, 11:42 AM

Addshore edited projects, added Wikibase Cloud (Launch Sprint 1 (2022)); removed Wikibase Cloud.

Addshore triaged this task as Medium priority.Jan 13 2022, 12:28 PM

Addshore moved this task from Sprint Backlog to Doing on the Wikibase Cloud (Launch Sprint 1 (2022)) board.

Jakob_WMDE subscribed.Jan 14 2022, 9:11 AM

@Addshore and I talked about what caching we might implement. For both the MainCache
A very casual switch to use redis for caching on my local development environment resulted in no noticeable speed up

Finally I saw a ~2x speed improvement (but un-impacted by the absence or presence of redis caching) by increasing the resources allocated to pods on my local minikube cluster.

MediaWiki

resources:
   web:
     requests:
-      cpu: 150m
-      memory: 350Mi
+      cpu: 2
+      memory: 2000Mi
     limits:
-      cpu: 400m
-      memory: 750Mi
+      cpu: 2
+      memory: 2000Mi
   webapi:
     requests:
-      cpu: 200m
-      memory: 250Mi
+      cpu: 2
+      memory: 2000Mi
     limits:
-      cpu: 500m
-      memory: 600Mi
+      cpu: 2
+      memory: 2000Mi

SQL primary:

resources:
     requests:
-      cpu: 40m
-      memory: 700Mi
+      cpu: 2
+      memory: 2000Mi
     limits:
-      cpu: 750m
-      memory: 1000Mi
+      cpu: 2
+      memory: 2000Mi

I saw a similar but slightly slower increase with less excessive resource allocation.

This ticket is probably now "done". We'll have a follow-up with actual targets to hit if @Samantha_Alipio_WMDE thinks that we need to speed up now.

• Samantha_Alipio_WMDE mentioned this in T299503: [EPIC] Make initial Wikibase.cloud speed improvements before launch.Jan 19 2022, 11:58 AM

• Samantha_Alipio_WMDE mentioned this in T299504: Decide on speed targets for the initial launch of Wikibase.cloud.Jan 19 2022, 12:07 PM

Addshore mentioned this in T299522: [INVESTIGATION 16hrs] Speed improvements for saving statements and creating new items.Jan 19 2022, 1:55 PM

	F34905870: image.png
	Jan 3 2022, 2:31 PM

[INVESTIGATION] Evaluate speed of key interactions on Wikibase.devClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

[INVESTIGATION] Evaluate speed of key interactions on Wikibase.dev
Closed, ResolvedPublic
Actions