Page MenuHomePhabricator

Analyze number of newcomers who created account after deployment using each available skin on English Wikipedia
Closed, ResolvedPublic

Description

Background

We would like to measure skin status of new comers after the Vector 2022 skin deployment on enwiki.

Goal

To measure number of new comers who created account after deployment using each available skin on English Wikipedia

Event Timeline

Summary

The Vector-2022 was deployed on English Wikipedia at 15:30 on 2023-01-18 (UTC). Below analysis focused on the new users who have created accounts on English Wikipedia after the deployment.

There was a spike of newcomers after the deployment. It has fallen close to the pre-deployment level 10 days after deployment.

image.png (778×1 px, 90 KB)

Below table summarizes the skin status on 2023-02-01 in percentage of the users who created accounts on English Wikipedia after the deployment. The new accounts data is pulled from schema event.ServerSideAccountCreation, only including self-made creations and excluding creations via api and bot user agent.

Skin NameNumber of new comersPercentage of newcomers
vector-20225042678.24%
vector1308120.30%
monobook5190.81%
timeless3210.50%
minerva990.15%
cologneblue10.00%
modern10.00%

How the skin status changes by newcomer's registration cohort?

Below table summarizes the number of newcomers using each skin on 2023-02-02 by users registration cohort

Registration cohortcologneblueminervamodernmonobooktimelessvectorvector2022total_newcomers
2023-01-19113110251224847197135
2023-01-2001109930264541686953
2023-01-210805938175733975259
2023-01-220705330118032164486
2023-01-230803325100036894755
2023-01-24070221974037254513
2023-01-25030181757636564270
2023-01-26060231545334903987
2023-01-27050111038232133621
2023-01-28070161130928043147
2023-01-29050131129129303250
2023-01-30030152026535083811
2023-01-3107071122135103756
2023-02-0105091120332343462

Below table summarizes the skin percentage on 2023-02-02 by user's registration cohort

Registration cohortcologneblueminervamodernmonobooktimelessvectorvector2022total_newcomers
2023-01-190.01%0.18%0.01%1.43%0.71%31.51%66.14%100.00%
2023-01-200.00%0.16%0.00%1.42%0.43%38.04%59.95%100.00%
2023-01-210.00%0.15%0.00%1.12%0.72%33.41%64.59%100.00%
2023-01-220.00%0.16%0.00%1.18%0.67%26.30%71.69%100.00%
2023-01-230.00%0.17%0.00%0.69%0.53%21.03%77.58%100.00%
2023-01-240.00%0.16%0.00%0.49%0.42%16.40%82.54%100.00%
2023-01-250.00%0.07%0.00%0.42%0.40%13.49%85.62%100.00%
2023-01-260.00%0.15%0.00%0.58%0.38%11.36%87.53%100.00%
2023-01-270.00%0.14%0.00%0.30%0.28%10.55%88.73%100.00%
2023-01-280.00%0.22%0.00%0.51%0.35%9.82%89.10%100.00%
2023-01-290.00%0.15%0.00%0.40%0.34%8.95%90.15%100.00%
2023-01-300.00%0.08%0.00%0.39%0.52%6.95%92.05%100.00%
2023-01-310.00%0.19%0.00%0.19%0.29%5.88%93.45%100.00%
2023-02-010.00%0.14%0.00%0.26%0.32%5.86%93.41%100.00%

Graph of percentage of newcomers using each skin by registration cohort on English Wikipedia

image.png (1×1 px, 172 KB)

Takeaways:

  • As of 2022-02-02, among the newcomers who registered between 2023-01-19 and 2023-01-21, around 64% of them use vector-2022 skin, and 34% of them use vector skin.
  • The proportion of vector-2022 skin continues to increase after the deployment on more recent registered user groups
  • Among the recently registered users, 93% of them use vector-2022 skin.
  • One thing needed to mention: the skin status only reflects the current skin preference, and the earlier registered users have more time to opt-out or opt-in. If we run the data a few days later, the skin status of the same cohort might change.
jwang triaged this task as Unbreak Now! priority.Feb 7 2023, 4:16 AM
jwang lowered the priority of this task from Unbreak Now! to High.
jwang moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
jwang renamed this task from Analyze number of new comers who created account after deployment using each available skin on English Wikipedia to Analyze number of newcomers who created account after deployment using each available skin on English Wikipedia.Feb 8 2023, 9:06 PM

How soon newcomers updated skin preference after registration on English Wikipedia?

Methodology

Metrics we measured:

  • For each skin, number of newcomers who have updated skin preference after vector2022 deployment on English Wikipedia.
  • Skin update rate: number of users who updated skin pref out of total newcomers after vector2022 deployment on English Wikipedia.

Account creation events between 2023-01-15 and 2023-02-07 are pulled from event.ServerSideAccountCreation schema, only include self-made creation, exclude auto creation and creations by bot users and API.
Skin preference updates between 2023-01-15 and 2023-02-07 are pulled from event.prefupdate schema. Analysis reviewed the last skin user selected in 24 hours of registration, and after 7 days of registration.

Analysis result
Skin preference update within 24 hours of registration
  • Among the newcomers who registered within 4 days after deployment, around 35% of them updated skin preference within 24 hours after registration. Most of them switched to vector skin.They probably registered for the purpose of changing default skin setting.
  • Among the newcomers after Jan 30, only around 7% of them updated skin preference within 24 hours of registration. Most of them opted for vector skin.
registration cohort datecologneblueminervamodernmonobooktimelessvector-2022vectorNumber of users who updated skin preference in 24 hours of registrationNumber of newcomersopt rate in 24 hours of registration
2023-01-18060442760710847281830.06%
2023-01-1911311004913422152513713535.22%
2023-01-200110913411426222872695341.31%
2023-01-2109057396717291901525936.15%
2023-01-2206054344611561296448628.89%
2023-01-230803226439921101475523.15%
2023-01-24070231837742827451318.32%
2023-01-25030181833566638427014.94%
2023-01-26060221531448522398713.09%
2023-01-27060111014378419362111.57%
2023-01-28080161014309357314711.34%
2023-01-29060121118290337325010.37%
2023-01-3003015181026030638118.03%
2023-01-31070792021926237566.98%
2023-02-010508111520424334627.02%
2023-02-020201491419223134366.72%
2023-02-0304011111815519932636.10%
2023-02-040307121616220029726.73%
2023-02-050101172414518828926.50%
2023-02-060401372314519235605.39%
2023-02-07050971515018637614.95%
Skin preference update after 7 days of registration

Only 0.25% of newcomers changed skin preference 7 days after registration.

registration cohort datecologneblueminervamodernmonobooktimelessvector-2022vectorNumber of users who updated skin preference after 7 days of registrationNumber of newcomersopt rate after 7 days of registration
2023-01-1802003491828180.64%
2023-01-191005510234471350.62%
2023-01-20001216172769530.39%
2023-01-21000324152452590.46%
2023-01-22000200121444860.31%
2023-01-2300000571247550.25%
2023-01-2401003371445130.31%
2023-01-250001023642700.14%
2023-01-260000012339870.08%
2023-01-270100013536210.14%
2023-01-280000010131470.03%
2023-01-290000001132500.03%
2023-01-300000002238110.05%
2023-01-310000001137560.03%