Page MenuHomePhabricator

Monotonic increasing of strip markers allows information to be shared between parts of a page
Open, MediumPublic

Description

The final set of numbers in a strip marker starts at 0 and goes up by 1 for each occurrence of the tag that placed the strip marker. This can be used in conjunction with obscure extension tags to pass information between different parts of a page. An example of this is that from Lua, "return frame:extensionTag('hiero', 'foo'):sub(29, 36)" returns a number 1 higher than the last each time, even between different #invoke's.


Version: 1.24rc
Severity: normal

Details

Reference
bz65259

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:24 AM
bzimport added a project: MediaWiki-Parser.
bzimport set Reference to bz65259.
bzimport added a subscriber: Unknown Object (MLST).

An (almost) backward‑compatible solution would be to replace in these markers the incrementing “8 hex digits” part with 8 random hex digits. However, that would have only a 32‑bit space, which has far too much collision risk.

That random part should be increased to at least 16 hex digits, for a 64‑bit space that should not collide in practice, but to be really collision‑risk‑free we should target 32 hex digits, for a 128‑bit space.
This change would require updating, on wikis, the modules that search for strip markers. Though a bit tedious (or more precisely, boring), I think it would be reasonably doable.

But while we are at breaking the format, and as 32 hex digits is quite long, we may consider switching to a base‑64 encoding. In this case the length is deterministic—always 22 characters if the = padding is omitted.

Patterns to match in Lua:

from:
'\127\'\"`UNIQ%-%-tagname%-%x+%-QINU`\"\'\127'

to:
'\127\'\"`UNIQ%-%-tagname%-[A-Za-z0-9+/]+%-QINU`\"\'\127'

(it’s such a shame that Lua patterns don’t have {n,m} quantifiers…)

(we could replace [A-Za-z0-9+/] with [%w+/], but I don’t like Lua’s confusing %w class, which, somewhat unexpectedly, doesn’t include the _ character.)

Then, for convenience (and some future‑proofing), we might consider providing some method in the mw.text.* library to get a pattern to match such markers, with an optional parameter for the tag to match (and if the parameter is omitted, match all tags).

To clarify, the random part I suggested here is not the same as the random part in the old marker format, from before Gerrit 214404.

In the old marker format, the random part was generated at PHP class instantiation, with all the caveats that have been mentioned in the tickets. That part was then constant across the page.
In my suggestion, the random part is different in each marker.

We can make a fixed (thus cached, etc.) regex to match the markers, including the random part.

But the page cache miss remains. Every generation of a given page would produce different markers, whereas currently they are the same across page generations.

But how could we have both the bread and the butter?
Markers that have to be unique within a given page, and at the same time constant and independent from each other… Incrementing and random approaches each satisfy one need but not the other. What solution remains?

The solution could be to use a deterministic RNG with a fixed seed.

Fortunately, PHP 8.2+ provides a native Random API that fits this use case. For example:

use Random\Randomizer;
use Random\Engine\PcgOneseq128XslRr64;

$pageSeed = hash( 'sha512', $title->getPrefixedText(), true );

// Seed must be 16 bytes for PCG
$seed = substr( $pageSeed, 0, 16 );
$engine = new PcgOneseq128XslRr64( $seed );

$rng = new Randomizer( $engine );

// Each call produces deterministic pseudo-random bytes
$id1 = $rng->getBytes( 16 ); // 128 bits
$markerId1 = rtrim( base64_encode( $id1 ), '=' );

Note that we also need to take the following into account:

One of the long-term goals for Parsoid is to be able to reparse only the part of the page that was actually changed and then merge it into the existing HTML for the rest of the page.

Given that the random sequence remains the same, we should take care not to inject the same ID into the page twice. (Note that the exact same concern already exists with the current incrementing IDs.)

(Edit: Apologies, Anomie — Phabricator automatically added you as a subscriber. I didn’t do it intentionally.)

That being said:

  1. The leak only allows discovering the index of the marker within the page. Although it is still a leak, it's not as severe as something like allowing direct modification of the Lua environment.
  2. Since this is a deterministic RNG, the values could in fact be generated externally. The deterministic‑RNG approach is essentially a form of obfuscation—complex, but still only obfuscation. Someone determined enough could theoretically reproduce the sequence and hardcode it in a Lua module, although it would be extremely tedious and highly improbable in practice.
  3. Because the markers are only intermediate artifacts (i.e., they do not appear in the final output), would markers that vary across page renderings actually affect caching? If they don't affect caching, then we could use true randomness instead of deterministic randomness, which would:
    • eliminate point 2 above (no reproducibility at all),
    • and also address concerns related to the future Parsoid partial‑rendering feature, where the current code would inject the same ID twice.
  • Base64 encodes data in groups of 3 bytes using 4 characters. Therefore, for maximum encoding efficiency, we should pick a number of bytes that is a multiple of 3. A sweet spot would be 12 bytes (96 bits), encoded using 16 characters (and as a bonus, 16 is a nice number). 96 bits is still sufficient for practical collision resistance, but I don't want to go below that since the "random ID" method relies on collision avoidance for correct operation.
  • I've thought of an improved approach: we could store the generated random IDs in a set (O(1) lookup), and if an ID is generated twice, simply generate a new one (or, with a deterministic RNG, advance to the next value). This would even allow using a 32‑bit space while keeping the current [0-9a-f]{8} format; collisions would occur, of course, but they would be detected and avoided.
  • I would be very interested to know whether intermediate markers affect page caching, because that would determine whether the correct method is to use true randomness or a deterministic RNG.
Od1n lowered the priority of this task from High to Medium.Jan 22 2026, 9:41 PM