Page MenuHomePhabricator

High-level investigation on Google Search updates on featured snippets for Quarterly Insights
Closed, ResolvedPublic

Description

Jan 22, 2020, Google rolled out a significant update to the first page of search results. Web pages in a featured snippet position will no longer be repeated in regular organic listings. We want to measure impact of this recents change to featured snippets on Wikipedia traffic.

Key Takeaways:

  • This change applies to multiple languages and in both mobile and desktop.
  • Webpages in a featured snippet position have had their regular organic results pushed back to the top of Page 2, or even on Page 3, is not guaranteed. If lose featured snippet, the results will probably return to page 1
  • Google Search Console doesn’t differentiate featured snippets from regular results. In addition it can't be inferred from webrequest logs (it's cleansed from the referer)

For quarterly insights:

  • Look at overall traffic across wikipedias and in english wikipedia before and after the change.

Event Timeline

cchen created this task.Mar 3 2020, 6:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 3 2020, 6:25 AM
kzimmerman triaged this task as High priority.Mar 4 2020, 9:43 PM
kzimmerman moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.

Hey @Isaac , Connie is taking a preliminary look at whether the Google change around Featured Snippets impacts our traffic - at a high level, so far, we're not seeing any obvious differences. We'd like to speak to initial findings and plans for next steps in our Quarterly Insights presentation, which is scheduled for March 16. Is there anything you're working on that you'd like us to mention? Do you want us to add a slide or two (perhaps you could make a guest appearance) if you have related info to share?

Isaac added a comment.Mar 5 2020, 11:45 PM

Thanks @kzimmerman for reaching out! Yeah, the lack of change makes sense to me even given the results seen by business2community, which I think are pretty good methods though lacked a control dataset so it would not surprise me if the 2% median drop isn't borne out by other analyses. I'll do some thinking and get back to you if there's additional information worth sharing.

@cchen I already mentioned but if you build a list of articles and how they show up in search, I'd be interested in helping out. I was doing some more reading on featured snippets, which led to this study from 2017. Obviously that's a while ago and things easily could have changed, but a few thoughts:

  • Most featured snippets are from long-tail queries, so general queries like parasite don't return anything but when I searched parasite earnings it pulled up the appropriate section from the Wikipedia article as a featured snippet. Google Search Console won't really help though with featured snippets because it doesn't show long-tail queries.
  • The best look we have at what pages are associated with featured snippets (short of buying access to an SEO resource like ahrefs) is a list of top 100 URLs by # of featured snippets that they put out with that 2017 research. Looking through that, the topic area that stand out that we could focus on is List articles. We see this both explicitly and also in normal articles that are about things that are a list (Wonders of the World, Zodiac Signs). While other articles also probably appear frequently in featured snippets (e.g., from that 2017 list, subarticles about the United States), the list articles are easy to identify via categories, Wikidata properties, or even just title matching ("List of%"). So they might be a good place to start for narrowing down pageview trends.

Thanks @Isaac ! I've also added you to the meeting invite, in case you want to pop in.

High-level analysis was done for Quarterly Insights

kzimmerman renamed this task from Impact on Google Search updates on featured snippets to High-level investigation on Google Search updates on featured snippets for Quarterly Insights.Mar 17 2020, 10:21 PM
kzimmerman updated the task description. (Show Details)Mar 17 2020, 10:35 PM

@cchen can you link to your notebooks/code? once that's done we can close out this task