See also @Fuzheado's observations at https://www.facebook.com/AndrewLih/posts/10160297334830241

In T191429#4107254, @Tbayer wrote:

See also @Fuzheado's observations at https://www.facebook.com/AndrewLih/posts/10160297334830241

Those are not insignificant traffic increases. Eyballing just the sites Andrew included, that's something like 10k pvs per day, if it keeps up. For no effort on our part, that is a lot of mindshare.

In T191429#4107368, @jmatazzoni wrote:

In T191429#4107254, @Tbayer wrote:

See also @Fuzheado's observations at https://www.facebook.com/AndrewLih/posts/10160297334830241

Those are not insignificant traffic increases. Eyballing just the sites Andrew included, that's something like 10k pvs per day, if it keeps up. For no effort on our part, that is a lot of mindshare.

To clarify, the spike around February 3 that Andrew pointed out there is likely unrelated - it obviously predates Facebook's rollout, and its size is much higher than the ~500 views/daily that their earlier tests reportedly generated. In the linked post it is actually instead conjectured that it might have to do with YouTube instead, but there too it doesn't match the timing of their recent SXSW announcement. We should look into that in a separate task.

Initial results of the daily number of enwiki pageviews in the US with a referrer from Facebook.

There does not appear to be any significant changes in daily pageviews to English Wikipedia from Facebook immediately following the full rollout of the article context feature on April 3, 2018. There's an increase in pageviews with a facebook referrer prior to the rollout between March 31 and April 2nd. This may be due to higher facebook traffic over the Easter holiday weekend.

Pending potentially obtaining a full list of articles on which the feature is deployed, I did a quick breakdown of pageviews for some popular US new sources that I confirmed to have the facebook article context feature. The following graph shows a sudden increase of pageviews to these pages from Facebook on April 3rd, with a maximum of 425 pageviews to the English Wikipedia Washington Post page on April 4th. Further monitoring of pageviews over the coming weeks will help determine if there are any sustained overall increases in daily pageviews with a facebook referrer.

I'll also take a look at top N pages (item 3) and work with @Tbayer to try to estimate the number of additional pageviews.

Data via

daily_pageviews<- do.call(rbind, lapply(seq(start_date, end_date, "day"), function(date) {
  cat("Fetching webrequest data from ", as.character(date), "\n")
  clause_data <- wmf::date_clause(date)
  
query <- paste("
SELECT '", date, "' AS date, COUNT(1) as requests
FROM wmf.webrequest",
clause_data$date_clause,
" AND referer LIKE '%facebook.com%'
AND is_pageview = TRUE
--all referers from facebook should by classified as external
AND referer_class = 'external'
AND http_status IN ('200', '304')
--remove bots
AND agent_type = 'user'
-- only look at English Wikipedia pageviews in the US
AND normalized_host.project = 'en'  
AND normalized_host.project_class = 'wikipedia'

https://github.com/wikimedia-research/facebook-article-context-impacts

I took a look at the top facebook referred pages on English Wikipedia before and after the rollout date.

One week prior to the feature rollout date on April 3rd (March 26th through April 2nd), there are no news media pages in the top 30 facebook referred English Wikipedia pages in the U.S. On April 4th (one day following the rollout), there are 5 news media pages in the top 30 facebook referred pages. Between April 3rd through April 9th, this number declines to only 2 news media articles in the top 30. Results below:

Summary of Top Facebook Referred News Media Articles
Top Facebook referred new media articles on April 4th (day after rollout)

Top Facebook referred new media articles between April 3rd to April 9th (week after rollout)

I confirmed all of the above news media sources included the article context feature on Facebook.

Full List of Top 30 Facebook Referred Pages
Top 30 Facebook Referred Pages Between March 26th through April 2nd (1 week prior to rollout)

Top 30 Facebook Referred Pages Between April 3rd through April 9th (1 week after rollout)

Query

SELECT CONCAT('https://',uri_host,uri_path,uri_query) AS url, SUM(1) AS requests
FROM wmf.webrequest ",
clause_data$date_clause,
" AND is_pageview
AND referer LIKE '%facebook.com%' 
AND referer_class = 'external'
AND http_status IN ('200', '304')
AND agent_type = 'user'
AND normalized_host.project = 'en'  
AND normalized_host.project_family = 'wikipedia'
AND geocoded_data['country'] = 'United States'
GROUP BY uri_host, uri_path, uri_query
ORDER BY requests DESC LIMIT 30

Here are the updated daily facebook referred pageviews based on data through April 25th.

There are no significant changes following the full rollout of the article context feature on April 3, 2018. In addition, any potential effects from Facebook's article context feature appear to be too small to determine from the overall number of Facebook referrals.

Looking just at the daily pageviews to a set of some news media related articles more clearly display the effect of the article context feature. The plot below includes a selection 9 new-media related articles including all those found in the top 30 Facebook referred pages the week following April 4th.

A week prior to the feature rollout (March 26 - April 3rd), there was an average of about 3 daily pageviews to these pages with a Facebook referrer. There is a significant increase of daily pageviews on April 4th; however, pageviews quickly decline after April 4th to an average of around 40 pageviews between April 11 and April 18th. Some of the higher pageviews around April 4th seen for Breitbart and potentially other sources may be the result of Facebook posts linking to the article directly in the context of news related coverage released that day.

Potential next steps if needed could include trying to expand the set of new articles pages pending a complete list from Facebook or by using Wikipedia categories but overall it appears that the article context feature had a very small effect on Facebook referred pageviews. The excerpt displayed by Facebook in the article context feature is large so it's possible that many Facebook users who access this feature do not clickthrough to the Wikipedia articles.

Let me know if you have any questions or comments,

Codebase

Great work, @MNeisler! Some additional remarks inline.

In T191429#4164524, @MNeisler wrote:

Here are the updated daily facebook referred pageviews based on data through April 25th.

There are no significant changes following the full rollout of the article context feature on April 3, 2018. In addition, any potential effects from Facebook's article context feature appear to be too small to determine from the overall number of Facebook referrals.

Looking just at the daily pageviews to a set of some news media related articles more clearly display the effect of the article context feature. The plot below includes a selection 9 new-media related articles including all those found in the top 30 Facebook referred pages the week following April 4th.

A week prior to the feature rollout (March 26 - April 3rd), there was an average of about 3 daily pageviews to these pages with a Facebook referrer. There is a significant increase of daily pageviews on April 4th; however, pageviews quickly decline after April 4th to an average of around 40 pageviews between April 11 and April 18th. Some of the higher pageviews around April 4th seen for Breitbart and potentially other sources may be the result of Facebook posts linking to the article directly in the context of news related coverage released that day.

To add, for the record: The Breitbart-related coverage and social media attention which likely is a confounding factor here is summarized e.g. in https://www.haaretz.com/us-news/.premium-breitbart-declares-war-on-wikipedia-in-facebook-s-fight-against-fake-news-1.5991915 (paywalled, but may be accessible via Google).

Potential next steps if needed could include trying to expand the set of new articles pages pending a complete list from Facebook or by using Wikipedia categories

For the record: we have been thinking about using https://en.wikipedia.org/wiki/Category:Media_in_the_United_States

but overall it appears that the article context feature had a very small effect on Facebook referred pageviews. The excerpt displayed by Facebook in the article context feature is large so it's possible that many Facebook users who access this feature do not clickthrough to the Wikipedia articles.

To clarify: by now we can safely assert that the effect on overall Facebook-referred pageviews is very small. That said, we still have options for exploring question 2 ("Estimate the number of additional daily pageviews resulting from the feature") further. From F17344233, it appears that we could state a lower bound of about 400 daily pageviews for this just based on this fairly small sample of 9 articles, which we might be able to increase a lot when including the long tail of all articles in the aforementioned category. Focusing on the top-referred news media articles first was a great initial approach, but (besides the shape of the chart) the fact that the article about a comparatively small website like the Daily Wire surpassed those for e.g. the NYT or Fox News is another indicator that this was indeed dominated by the controversies/attention generated by Facebook's announcement itself, rather then the feature per se.

Let me know if you have any questions or comments,

Codebase

Some minor technical notes about the queries used (which should not affect the validity of the data):

• Instead of the regexes for uri_path, it's probably preferable to use the already parsed field pageview_info['page_title'].
• As discussed earlier, the condition http_status IN ('200', '304') is redundant to is_pageview (it's part of the pageview definition)