Page MenuHomePhabricator

flattenIfArray should just accept null/undefined result
Open, LowPublic


parsoid/lib/wt2html/tokenizer.utils.js has an annoying throw statement

			for (var i = 0; i < e.length; i++) {
				var v = e[i];
				if (Array.isArray(v)) {
					// Change in assumption from a shallow array to a nested array.
					if (res === null) { res = e.slice(0, i); }
					internalFlatten(v, res);
				} else if (v !== null && v !== undefined) {
					if (res !== null) {
				} else {
					throw new Error("falsy " + e);

Because of it, pegTokenizer.pegjs has a bunch of r:something { return r; }, making the compiled WikiPEG code inefficient.

We should just comment out the throw statement and remove { return r; } or similar code in pegTokenizer.pegjs if return tu.flattenStringlist(r); is called in downstream code.

Event Timeline

Change 508176 had a related patch set uploaded (by Dan1wang; owner: Dan1wang):
[mediawiki/services/parsoid@master] All NULL in flattenIfArray and remove superfluous return statements (T222560)

Change 599457 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Cleanup some superfluous actions in the grammar

The history of that throw is that the negative predicate return values were leaking into the token stream.

If we're going to rely on TokenizerUtils::flattenIfArray to remove them, we need to be more explicit about it and audit these changes carefully.

Change 599457 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Cleanup some superfluous actions in the grammar

Change 603571 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump Parsoid to v0.12.0-a16

Change 603571 merged by jenkins-bot:
[mediawiki/vendor@master] Bump Parsoid to v0.12.0-a16