Page MenuHomePhabricator

Add a link engineering: support fetching recommendations from a remote wiki
Closed, InvalidPublic

Description

GrowthExperiments on test wikis can be configured to use articles from a production wiki for suggested edits; we want the same for suggested structured edits.

Once we have the code in place to do this, we'll want patch(es) to enable the remote fetch for the following beta wikis:

  • frwiki
  • enwiki
  • cswiki
  • arwiki
  • bnwiki
  • viwiki

Event Timeline

For link recommendations to work with remote articles:

  • the remote wiki's CirrusSearch index must contain all the data needed for finding suggested articles;
  • the link recommendations REST API GET and POST must work with remote articles.

This means that

  • the remote wiki must have GrowthExperiments with link recommendations enabled (the setup we had with maintenance-based tasks where the remote wiki didn't need GrowthExperiments at all won't work here) so that the search index gets populated.
  • when GETting recommendations, the API must proxy the request to the remote wiki.
  • when POSTing recommendation review results, the API must ignore that the article doesn't exist locally.

All in all, seems pretty simple.

We'd have to deploy the extension on enwiki to make beta enwiki work. That means we'd need a configuration flag for "backend only" mode as presumably we don't want to expose the suggested tasks interface to any users there.

For link recommendations to work with remote articles:

  • the remote wiki's CirrusSearch index must contain all the data needed for finding suggested articles;
  • the link recommendations REST API GET and POST must work with remote articles.

This means that

  • the remote wiki must have GrowthExperiments with link recommendations enabled (the setup we had with maintenance-based tasks where the remote wiki didn't need GrowthExperiments at all won't work here) so that the search index gets populated.
  • when GETting recommendations, the API must proxy the request to the remote wiki.
  • when POSTing recommendation review results, the API must ignore that the article doesn't exist locally.

All in all, seems pretty simple.

We would also need a way to view the article in the beta wiki, right?

We would also need a way to view the article in the beta wiki, right?

We have $wgGENewcomerTasksRemoteArticleOrigin for making the task card point to the remote wiki. If we also want to test the guidance / editing interface, importing articles might be less hassle.

We would also need a way to view the article in the beta wiki, right?

We have $wgGENewcomerTasksRemoteArticleOrigin for making the task card point to the remote wiki. If we also want to test the guidance / editing interface, importing articles might be less hassle.

Yeah, I imagine we do. So the setup would be something like this, perhaps?

  • import articles from production wiki
  • configuration flag to read link reco data from production wiki
  • saving link reco reviews happens to local beta cluster wiki

So the setup would be something like this, perhaps?

  • import articles from production wiki
  • configuration flag to read link reco data from production wiki
  • saving link reco reviews happens to local beta cluster wiki

Yes. The non-obvious part is how to do search, which we could do locally or on the production wiki. Doing it on the production wiki means having to run the extension there so the search index is maintained; otherwise it seems straightforward. For a large wiki like enwiki, that might mean having to deal with scaling issues earlier than we otherwise would.

Doing it locally would mean that 1) we'd have to make sure articles are indexed by ORES topics, 2) we'd have to have some replacement for the EventGate index update pipeline (direct CirrusSearch writes, probably).

...which both seem troublesome; the first kind of aligns with our plans to deploy in stealth mode on all wikis, the second can be reused for local setups.

So the setup would be something like this, perhaps?

  • import articles from production wiki
  • configuration flag to read link reco data from production wiki
  • saving link reco reviews happens to local beta cluster wiki

Yes. The non-obvious part is how to do search, which we could do locally or on the production wiki. Doing it on the production wiki means having to run the extension there so the search index is maintained; otherwise it seems straightforward. For a large wiki like enwiki, that might mean having to deal with scaling issues earlier than we otherwise would.

Doing it locally would mean that 1) we'd have to make sure articles are indexed by ORES topics, 2) we'd have to have some replacement for the EventGate index update pipeline (direct CirrusSearch writes, probably).

It sounds like we are sorting out a lot of these questions in T274198: Beta wiki configuration for add link project. In particular, we decided to go with article import, local search, ORES topic setting via a script.

So @Tgr I am marking this task as invalid but please reopen if you think there is work to be done here that is not captured in T274198

Restricted Application added a subscriber: hubaishan. · View Herald TranscriptJan 21 2026, 2:39 PM