Background
As we are not sure about the propagation time of adding the sameAs property, the most reliable way to determine whether any changes are observed is adding the property to a subset of pages and looking at any changes using an A/B test
Hypothesis
Adding the Schema.org sameAs property will increase pageviews coming from search engines
Acceptance criteria
- Determine how to set up an A/B test for the sameAs property
- Provide an estimate for how long the change will take (is a mid-November deployment viable)
- No deviation in behavior for logout / login state.
Notes and thoughts so far
- We can use page id to bucket pages
- We would like to target 50% of traffic by assigning the test or control group to a particular number of the page id (for example, even and odd page ID's)
- We would like the ability to avoid the top 100 articles from this test
- We don't know if we can do this client-side or if we have to do it server-side
- We should look into whether we should analyze the results on wikidata as well. From Tilman's commen: "I have to say that I don't know enough about the topic yet to fully understand the theory of change, i.e. the assumed mechanism that causes the addition of sameAs to increase views. E.g. would we also assume that the linked Wikidata benefits and will be ranked higher? If yes, should the analysis take Wikidata views into account as well?"