30 minutes isn't really reasonable, and neither is spamming more purge traffic. If there's a constant risk of the page content breaking without invalidation, how is even 30 minutes acceptable? Doesn't this mean that on average they'll be broken for 15 minutes after an affecting change?

In T142944#2557015, @BBlack wrote:

30 minutes isn't really reasonable, and neither is spamming more purge traffic. If there's a constant risk of the page content breaking without invalidation, how is even 30 minutes acceptable? Doesn't this mean that on average they'll be broken for 15 minutes after an affecting change?

Well, I have no idea how often they change in a significant way, but it's probably rather days than minutes.

As you can see on https://kn.wikipedia.org/wiki/%E0%B2%B5%E0%B2%BF%E0%B2%B6%E0%B3%87%E0%B2%B7:AboutTopic/Q12345 that page includes information from a lot of Wikidata entities (mostly labels of these). In theory a change to any of them would require us to purge that page, but that's hardly implementable and will lead to a lot of purge traffic (especially given we have the placeholders on various wikis).

During Wikimania we (@Joe and I) thought about how bad it would be to just cache them for n hours. We could easily provide a ?action=purge on the special page (would be done on the MediaWiki site of things), so that people can manually purge them after editing, if they deem that to be necessary. Does that sound reasonable?

I think I'm lacking a lot of context here about these special pages and placeholders. But my bottom line thoughts are currently along these lines:

1. How do actual, real-world, anonymous users interact with these placeholders and special pages? What value is it providing the average reader, in what way? How does the scope of the new code and new invalidation problems (esp potential purge traffic) compare to that? Because I tend to think (with what little context I have) that this sounds like a ton of churn on our end for very little real value to the user. Maybe most of the value is to logged-in editors, who don't face invalidation problems in the first place?

2. For the most part, we can categorize the invalidation model of page content into one of two bins: either it's purged on relevant update nearly-immediately (at most, a few seconds' delay for asynchronicity and such), or it's something that sometimes goes stale for some real amount of time, where we really have to think about what happens when users read a stale page, and we need an upper bound on staleness to consider that question properly. Once you're in the latter bin of stale things, there needs to be a rational way to quantify the fallout of a stale view. Is a stale page broken itself, or does it have broken links, or simply outdated content? I tend to think that, in the examples I've seen so far, either something requires immediate invalidation, or staleness isn't a real issue within a reasonable (e.g. hours, days) timeframe. 30 minutes seems arbitrary and probably not tied to a real-world constraint on how broken a stale view is. It sounds more like a compromise because we really want immediate purging but we know the purge volume will be unreasonable.

In T142944#2560699, @BBlack wrote:

I think I'm lacking a lot of context here about these special pages and placeholders. But my bottom line thoughts are currently along these lines:

1. How do actual, real-world, anonymous users interact with these placeholders and special pages? What value is it providing the average reader, in what way? How does the scope of the new code and new invalidation problems (esp potential purge traffic) compare to that? Because I tend to think (with what little context I have) that this sounds like a ton of churn on our end for very little real value to the user. Maybe most of the value is to logged-in editors, who don't face invalidation problems in the first place?

ArticlePlaceholders have the most value for readers that want information about a certain topic. The idea is to add value to (primarily) small Wikipedias so that people see (and use) them more, which in turn, will hopefully gain us new editors.
For this, we also desire to get the placeholders into search engines, to drive more traffic to those small Wikipedias (which don't have much content on their own, thus few search engine traffic).

In T142944#2560699, @BBlack wrote:

2. For the most part, we can categorize the invalidation model of page content into one of two bins: either it's purged on relevant update nearly-immediately (at most, a few seconds' delay for asynchronicity and such), or it's something that sometimes goes stale for some real amount of time, where we really have to think about what happens when users read a stale page, and we need an upper bound on staleness to consider that question properly. Once you're in the latter bin of stale things, there needs to be a rational way to quantify the fallout of a stale view. Is a stale page broken itself, or does it have broken links, or simply outdated content? I tend to think that, in the examples I've seen so far, either something requires immediate invalidation, or staleness isn't a real issue within a reasonable (e.g. hours, days) timeframe. 30 minutes seems arbitrary and probably not tied to a real-world constraint on how broken a stale view is. It sounds more like a compromise because we really want immediate purging but we know the purge volume will be unreasonable.

Yeah, in a perfect world we would (obviously) always serve the most up to data content from Wikidata.
I think it's ok for these pages to get a little outdated, relevant links don't change that often (most probably go to Wikidata, which has stable ids, anyway). The only issue with longer cache duration I can see is vandalism and broken pages (for example in case a Wikipedia messes up their Lua module so that the special page produces garbage) getting caught up in the caches for too long.

In T142944#2560827, @hoo wrote:

For this, we also desire to get the placeholders into search engines, to drive more traffic to those small Wikipedias (which don't have much content on their own, thus few search engine traffic).

May I ask how this should work? As far as I understand your special-page generates an “article” from wikidata-data if there is no real article. So if for example there would be no article about “Horse” on english Wikipedia if someone would search for “Horse” your special-page would generate it.
But how can you feed that to a external search-engine? For that the search-engine would need to ask for a page that is not there– why should it do so and for which articles should it ask?

In T142944#2566353, @DaBPunkt wrote:

In T142944#2560827, @hoo wrote:

For this, we also desire to get the placeholders into search engines, to drive more traffic to those small Wikipedias (which don't have much content on their own, thus few search engine traffic).

May I ask how this should work? As far as I understand your special-page generates an “article” from wikidata-data if there is no real article. So if for example there would be no article about “Horse” on english Wikipedia if someone would search for “Horse” your special-page would generate it.
But how can you feed that to a external search-engine? For that the search-engine would need to ask for a page that is not there– why should it do so and for which articles should it ask?

Article placeholder doesn't redirect from the article namespace, but only works if linked to directly (for example on Special:Search or externally). We plan to feed a list of notable placeholders to search engines (possibly via an indexed special page).

I've created T144592: Search index a limited number of article placeholders on cywiki for testing and evaluation purposes for a trial of submitting placeholders to search engines. Based on that, we can (probably) do further performance evaluation.

Heads up: In T144592: Search index a limited number of article placeholders on cywiki for testing and evaluation purposes we decided to index exactly 1,000 placeholders on eowiki.

All other placeholders will not be linked to and also have <meta name="robots" content="noindex,nofollow"/> set, so this trial is a very limited trial.

We hope to incrementally increase the number of placeholders indexed later on, depending on the findings from this trial.

Nothing was ever resolved here. 30 minutes seems like an arbitrary number with no formal basis or reasoning, and is way shorter than we'd like for anything article-like.

I clicked Submit too soon :) Continuing:

We'd expect content to be at minimum a day, if not significantly longer. MW currently emits 2-week cache headers (with plans to eventually bring that down closer to a day, but those plans are still further off). Cache invalidation is a hard problem, but it's not something we can just ignore, either. Perhaps this should be tied into the broader X-Key effort to sweep these up when the underlying wikidata is updated?

@BBlack Given T109458: [Story] CDN cache article placeholders is not implemented, there is no caching for these pages right now. If you consider this a requirement for this limitted trial, we could look into caching them for one day (and provide a custom purge mechanism).

Hey :)

We'd really like to move forward with making the ArticlePlaceholder more useful. It not showing up in search engine results is the biggest blocker at the moment.
Given that there was no feedback on Marius' last comment can we go ahead? If we don't get a reply in the next two weeks I assume yes.