Page MenuHomePhabricator

[Hypothesis] WE2.3.8 Build capabilities to integrate Abstract Content into local Wikipedias
Open, HighPublic

Description

Hypothesis

If we implement the capabilities for article-level opt-in integration of Abstract Wikipedia content into local Wikipedias, we can demonstrate how the model works in practice and be ready to engage pilot communities in Q1.

This is a *readiness* hypothesis, not a measurement hypothesis. Its validation is the existence of a working end-to-end pipeline (M0 → M1 → M2) that a non-developer stakeholder can show to a candidate pilot community without any step requiring hand-holding from the engineering team.

The milestones below decompose "the capabilities for article-level opt-in integration" into three required layers plus one stretch milestone for downstream discoverability:

  • M0 — Storage layer. A durable place to put pre-rendered Abstract Content so that readers on local wikis see it at normal read latency.
  • M1 — Read preview surface. A special page on local wikis that renders Abstract Content from the M0 storage layer, used for internal dogfooding and demos, and as the rendering component that M2 reuses.
  • M2 — Read surface, opt-in. The mechanism by which a local sysop opts specific abstract articles into mainspace, plus the external-search-engine indexability metadata (robots / canonical / hreflang / sitemap inclusion) that lets crawlers treat opted-in articles as normal mainspace pages.
  • M3 — Internal search discoverability (stretch). Integration of opted-in abstract articles into the local wiki's CirrusSearch surfaces — title typeahead and main search — so that readers on the client wiki can find opted-in content through the normal on-wiki search box. Explicitly out of scope for the Q1 readiness hypothesis and depends on coordination with Search Platform; tracked in the same epic because it is the same end-to-end pipeline, but sequenced to follow once the core M0 → M1 → M2 flow is validated in pilot.

M2 is the milestone the Q1 readiness hypothesis is ultimately about. M0 and M1 exist to make M2 demonstrable and to give us a staged place to catch problems before they touch mainspace. M3 is strictly the post-pilot scale-up of the same pipeline and does not gate Q1 readiness; it is listed in this epic because splitting it into a separate epic would disguise that it is the same end-to-end pipeline the earlier milestones built, not independent work.

Readiness criteria for engaging pilot communities

The epic is "ready" for pilot engagement when, cumulatively:

  • A candidate pilot community can be shown a working opted-in abstract article in mainspace on Test Wikipedia, reached by normal navigation and not by a special URL hack. (Product has selected Test Wikipedia as the pilot wiki; the Beta Cluster is not a viable alternative because the WikiLambda back-end services — function-orchestrator and function-evaluator — do not run there.)
  • A sysop on that wiki can opt an article in and out via CommunityConfiguration without needing shell or database access.
  • The content loads from the storage layer in the common case, with the live-call fallback exercised only on cold paths.
  • Provenance, attribution, and deactivation CTAs are visible to readers and sysops respectively.
  • A kill-switch exists and has been exercised in at least one drill.

These are deliberately checklist-shaped rather than numeric, matching the readiness framing.

Terminology

  • Repo mode (existing $wgWikiLambdaEnableRepoMode): wiki hosts ZObjects (Wikifunctions).
  • Client mode (existing $wgWikiLambdaEnableClientMode): wiki calls a remote Wikifunctions repo, e.g. for {{#function:…}}.
  • Abstract mode (existing $wgWikiLambdaEnableAbstractMode): wiki *hosts* Abstract Content in its own namespaces (abstract.wikipedia.org).
  • Abstract client mode (new, introduced in Milestone 1; working name $wgWikiLambdaEnableAbstractClientMode): wiki *consumes* Abstract Content authored elsewhere and displays it to its readers.

"Abstract client mode" is orthogonal to the three existing modes. A local Wikipedia enabling this feature will typically have client mode on, abstract mode off, and abstract client mode on.

Prior art in the WikiLambda codebase

A non-trivial amount of the M0 infrastructure already exists and is in production on abstract.wikipedia.org:

  • includes/AbstractContent/AbstractWikiRequest.php already fetches a fragment from a remote Wikifunctions, sanitises it via WikifunctionsPFragmentSanitiserTokenHandler, and caches it in memcached with fresh/stale TTL tiers.
  • includes/Jobs/CacheAbstractContentFragmentJob.php is the async refresh job, keyed today on (qid, language, date, functionCall).
  • includes/Special/SpecialViewAbstract.php is the repo-side special page and is the starting point for the client-side equivalent.
  • includes/ActionAPI/ApiAbstractWikiRunFragment.php is the existing cross-wiki fragment entry point.
  • WikiLambdaAbstractNamespaces already maps namespaces to Wikidata concept types.

The work in this epic is therefore best read as *build a persistent storage layer for abstract content*, *generalising the special page for client use*, and *building the opt-in layer on top of the existing content pipeline* — not as greenfield construction.

Non-goals (for this epic)

  • Local editing of Abstract Content on client wikis (edits only happen on abstract.wikipedia.org).
  • Synchronous live regeneration on article reads in steady state; live orchestrator calls are a cold-path fallback only.
  • Custom skins, themes, or per-wiki content overrides of abstract articles.
  • Merging an opted-in abstract article with an existing local article on the same subject; local content always wins.
  • Quantitative measurement-style success metrics for this epic. We will collect telemetry (see M0 observability) but evaluation of whether the model is viable in practice is for the pilot, not for this build.

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
In ProgressJdforrester-WMF
In ProgressJdforrester-WMF
In Progressgengh
Opengengh
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
In Progressgengh
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone

Event Timeline

DSantamaria added a project: OKR-Work.