AFAIK all projects are included in the pageviews API. This seems to be something with the tool instead, so https://github.com/MusikAnimal/pageviews/issues would be a better place.

Hi,
Not all projects are included in the pageview API.
The definition of what is called a "pageview" can be found here, and more precisely here in the pseudo-code section.

Projects in .wikimedia.org that are included in the api are

• commons
• meta
• incubator
• species

Please include in the domain .wikimedia.org the projects nl and be as well in the pageview API.

@Romaine: the PageviewAPi doesn't return pageviews for all domains, as @JAllemandou
said there is a criteria behind what is consider a pageview.

@Ironholds, @DarTar
We do not have *.wikimedia.org domains on the pageview definition, do you remember the rationale of why those were excluded?

Because they're not "production" wikis; the goal was to avoid counting meta-pageviews (chapter traffic, say) towards our metrics.

We have gotten this request from pretty much everyone that's been excluded by that policy, though, @Nuria and @Ironholds. It might make sense to not report on those as part of our "look we're distributing knowledge to the world" metrics, but it doesn't make sense to exclude them from the API. I propose we compute them and put them in the API, just not include them in any "all-projects" type of aggregate we add in the future. Because just being able to access the statistics doesn't mean we report on it, right? If @Ironholds is *really* insistent, I guess we could make a whole separate pipeline for these pageviews and serve them via /metrics/pageviews-meta/ or something like that. It seems a bit weird to me, what does everyone else think?

@Milimetric: I think we can do that, it requires several modifications we need to plan for :

Let's keep those wikis with is_pageview= false (so as @Ironholds said it is clear what we count towards the metric of "distributting knowledge to the word").
but let's have a pipeline that puts those pageviews is a similar to pageview-hourly table and those can be loaded from there to the pageview API (just a sketch of what we can do here, let's tackle this concretely at our next tasking meeting)

Yeah, that's what I meant, Nuria, because the split might make sense on our back-end but not the front-end (pageview API)

Sounds good to me!

In T127804#2060090, @Milimetric wrote:

We have gotten this request from pretty much everyone that's been excluded by that policy, though, @Nuria and @Ironholds. It might make sense to not report on those as part of our "look we're distributing knowledge to the world" metrics, but it doesn't make sense to exclude them from the API. I propose we compute them and put them in the API, just not include them in any "all-projects" type of aggregate we add in the future. Because just being able to access the statistics doesn't mean we report on it, right? If @Ironholds is *really* insistent, I guess we could make a whole separate pipeline for these pageviews and serve them via /metrics/pageviews-meta/ or something like that. It seems a bit weird to me, what does everyone else think?

Sounds goods to me. If this would be implemented it would solve it I think for everyone.

Cool, we'll use this ticket then to prioritize that work.