Page MenuHomePhabricator
Paste P4166

ArchCom-RFC-2016W40-irc-E287.txt
ActivePublic
Actions

Authored by • RobLa-WMF on Oct 5 2016, 10:04 PM.
Tags
None
Referenced Files
F4561603: ArchCom-RFC-2016W40-irc-E287.txt
Oct 5 2016, 10:04 PM
Subscribers
None
21:01:02 <TimStarling> #startmeeting RFC meeting
21:01:02 <wm-labs-meetbot> Meeting started Wed Oct 5 21:01:02 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:02 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:02 <wm-labs-meetbot> The meeting name has been set to 'rfc_meeting'
21:01:05 <cscott> good afternoon
21:01:18 <TimStarling> #topic RFC: Future of magic links | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) |​ Logs: https://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/
21:01:54 <robla> #link https://phabricator.wikimedia.org/T145604
21:02:08 <robla> #link https://www.mediawiki.org/wiki/Requests_for_comment/Future_of_magic_links
21:02:49 <robla> hi everyone! hi legokt^H^H^H^H^H^Hcscott :-)
21:03:32 <cscott> i will be serving as your legokt today
21:04:28 <robla> there are three steps we discussed that are summarized in the description of T145604
21:04:29 <stashbot> T145604: [RfC] Future of magic links - https://phabricator.wikimedia.org/T145604
21:05:04 <Scott_WUaS> Hi
21:05:12 <robla> step 1: Disable the magic link functionality by default for the MediaWiki 1.28 release, and mark it as deprecated.
21:05:30 <robla> 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration
21:05:30 <robla> 3. Disable magic links functionality a year or so after the MediaWiki 1.28 release (in time for the next MediaWiki LTS release)
21:05:50 <robla> is there anyone that objects to step 1?
21:06:34 <TimStarling> is legoktm online?
21:06:43 <cscott> maybe we should find out who's here? (that is, actively participating in this discussion.)
21:07:03 <cscott> TimStarling: no, legoktm is in class, so I'm being his surrogate for this meeting.
21:07:16 <robla> legoktm is at a class right now, so if there are objections or things he needs to clarify, we'll need to plan a followup
21:08:02 <Scott_WUaS> (Hi CScott - I'm reading along, typing occasionally)
21:08:05 <robla> it seems, though, that this is a "yeah, we should just do this" kind of thing for step 1
21:08:08 * subbu is around and is generally happy with the direction, modulo details that might need tweaking
21:08:31 <TimStarling> is there going to be an extension to replace it?
21:08:39 <bd808> step 1 is normal deprecation procedure I think
21:09:02 <brion> a workalike extension could be created if someone wants one but i don't know if anyone does :)
21:09:08 <cscott> i think there's still a semi-open question w/ what the replacement should be: 1) ordinary template, 2) parser function, 3) extension, 4) interwiki link, 5) something else?
21:09:13 <TimStarling> if there's no migration path for external users then I don't think it should be deprecated
21:09:13 <bd808> Kunal wrote something about that in email/phab today...
21:09:21 <TimStarling> just disabled by default
21:09:50 <brion> my inclination is ordinary template is the right thing for wikipedia
21:09:54 <cscott> 1) and 2) look like {{ISBN|xxx}} and require content edits, 3) would be a workalike using a parser hook, 4) would look like [[isbn:xxx]] and require content edits.
21:09:58 <robla> legoktm's position is that disabling just disables the hyperlinks, but it's still readable
21:10:27 <brion> it is a relatively clean 'failure mode' yes
21:10:38 <brion> no explosions/php fatals :)
21:10:40 <cscott> i think we already have a config option for disabling it in the parser? legoktm wrote that patch. so it's just a matter of turning that config option off on WMF wikis.
21:11:27 <cscott> we can discuss removing the feature from the codebase and/or moving it to a parser extension as a separate thing.
21:11:31 <subbu> I don't like option (3)
21:11:38 <bd808> TimStarling: "We would move the Booksources code and ISBN parser function to an
21:11:39 <bd808> a extension." -- https://lists.wikimedia.org/pipermail/wikitech-l/2016-October/086716.html
21:12:11 <subbu> i.e. option (3) for parsing "ISBN ....." as a magic link with a parser hook
21:14:15 <cscott> i don't like it for WMF wikis, but it would be a reasonable thing to allow 3rd parties to continue to support magic links on their external wikis if they felt strongly about it
21:14:21 <TimStarling> using square brackets seems elegant to me
21:14:27 <cscott> re TimStarling's "if there's no migration path for external users then I don't think it should be deprecated"
21:14:32 <TimStarling> considering it is an actual link
21:15:03 <subbu> I can go with either {{isbn|..}} or [[isbn:..]] personally
21:15:04 <bd808> pmid and rfc are proposed to become default interwiki links
21:15:24 <bd808> isbn is the only oddball right?
21:15:27 <cscott> [[rfc:1234]] would work fine
21:15:37 <cscott> isbn is the oddball because it goes to *the wiki itself*
21:15:42 <bd808> *nod*
21:15:54 <TimStarling> yeah, it would be like a namespace alias
21:15:55 <cscott> although i don't think there's anything preventing interwiki links from being relative URLs?
21:15:58 <TimStarling> like media
21:16:29 <TimStarling> it could even be a negative namespace ID rather than an interwiki prefix
21:17:15 <brion> interesting idea
21:17:37 <brion> i think i prefer an interwiki, with special:booksources as a special service that happens to be the link target
21:18:27 <robla> is there a precedent for magic link functionality in other wiki markups? e.g. does anyone else automatically turn "ISBN \d+" into a hyperlink automatically?
21:18:36 <brion> but negative (magic) namespace has a certain elegance to it too
21:19:11 <cscott> robla: external link hyperlinking is the precedent
21:19:16 <cscott> ie, http://foo.bar/bat
21:19:34 <cscott> most other markup languages have that
21:19:43 <brion> bugzilla autolinks textual things like 'bug 1234'
21:19:49 <cscott> when given a bare url
21:19:57 <brion> and of course many mobile browsers autolink phone numbers (which i hate ;)
21:20:00 <cscott> github markdown autolinks hashes and bug references
21:20:25 <TimStarling> $RFCPattern = "RFC\\s?(\\d+)";
21:20:25 <TimStarling> $ISBNPattern = "ISBN:?([0-9- xX]{10,})";
21:20:32 <TimStarling> s/$RFCPattern/&StoreRFC($1)/geo;
21:20:38 <TimStarling> s/$ISBNPattern/&StoreISBN($1)/geo;
21:20:39 <cscott> https://help.github.com/articles/autolinked-references-and-urls/
21:20:40 <TimStarling> says usemod
21:20:49 <TimStarling> so this feature was in fact carried over from usemodwiki
21:21:08 <cscott> PMID was a more recent addition
21:21:47 <robla> that would imply a precedent that we're continuing rather than merely an oddball legacy feature of our own
21:22:17 <cscott> weeelll.
21:22:30 <robla> that said, I think I agree with brion's hatred of autolinked phone numbers
21:22:38 <cscott> precent becomes oddball legacy given enough time (and lack of external adoption)
21:22:45 <TimStarling> usemod is also responsible for the "free link" terminology
21:23:25 <robla> it's quite likely just an oddball legacy feature that we inherited from grandpa usemod :-)
21:24:45 <TimStarling> the argument for deprecation would be simpler parser code?
21:25:27 <cscott> simple parser spec & removal of an english-specific bit of markup
21:25:42 <TimStarling> but as long as we magically link free URLs, we will still have Parser::doMagicLinks()
21:25:48 <cscott> RFCs in particular are english-only and not really relevant for most of our projects
21:26:15 <TimStarling> that's the argument for step 2
21:26:16 <James_F> TimStarling: And the magic of image links -> remote transclusion when that evil flag is enabled.
21:26:20 <TimStarling> I mean the argument for step 1
21:26:23 <cscott> sure, but doMagicLinks() could be greatly simplified and only match https?: prefixes
21:26:30 <cscott> for example
21:26:39 <cscott> instead of the pretty complicated regexp for ISBNs
21:26:40 <TimStarling> I am fine with step 2 and 3 but not so much with step 1
21:27:02 <cscott> and our free URLs autolinking is pretty crazy complicated because of all the different protocol schemes we support in theory.
21:27:30 <subbu> I don't like "magic links" because they suddenly add special meaning to some text substrings ...
21:27:59 <cscott> it complicates WTS as well, you have to watch for all sorts of boundary conditions when serializing
21:28:01 <TimStarling> if it's disabled on WMF wikis then parsoid doesn't need to support it
21:28:19 <James_F> Parsoid is for more than just WMF users, eventually.
21:28:22 <subbu> instead of using uniform syntax .. and as someone argued, RFC xyz is ocfusing since mediawiki has RFCs.
21:28:25 <bd808> o_0 parsoid is Wikimedia only TimStarling?
21:28:29 <subbu> *confusing
21:28:44 <cscott> and if we're going to continue to do round-trips for content migration, whether to wikitext 2.0 or markdown or <fill in the blank>, then simplifying WTS (wikitext serialization) will continue to be desirable
21:29:02 <TimStarling> parsoid has a reduced feature set compared to MW
21:29:04 <robla> it seems like protocol links (like mailto:.*, http?s:.*) have ample precedent in both email and wikis. Magic words as arbitrary barewords are relatively rare
21:29:05 <subbu> and all the nowiki crap.
21:29:09 <TimStarling> it doesn't even support wikipedia properly
21:29:24 <bd808> well that I can't argue against
21:29:38 <cscott> robla: yes, but have you looked at the full list of protocols we support?
21:29:54 <TimStarling> there are no plans to make parsoid support every single MW parser extension
21:30:01 <robla> cscott: not in a while
21:30:05 * robla goes to look
21:30:22 <cscott> $wgUrlProtocols = [ 'bitcoin:', 'ftp://', 'ftps://', 'geo:', 'git://', 'gopher://', 'http:// ', 'https://', 'irc://', 'ircs://', 'magnet:', 'mailto:', 'mms://', 'news:' , 'nntp://', 'redis://', 'sftp://', 'sip:', 'sips:', 'sms:', 'ssh://', 'svn://', 'tel:', 'telnet://', 'urn:', 'worldwind://', 'xmpp:', '//' ];
21:30:27 <subbu> TimStarling, yes .. i've proposed that parser hooks that rely on parser internals should instead be deprecated.
21:30:46 <subbu> parsoid and php parser have different internals.
21:30:47 <cscott> i'm pretty sure we could drop gopher:// and worldwind://
21:31:05 <bd808> I hear gopher is going to make a comeback ;)
21:31:25 <TimStarling> free external links could support a reduced set of protocols compared to bracketed external links
21:31:27 <cscott> but in general, I like {{bitcoin|<hash>}} or [[bitcoin:hash]] better than autolinking bitcoin:cafebabe in text
21:31:42 <subbu> but, anyway ... we do want to get to a unified model .. and that parsoid doesn't support wikipedias is neither here nor there .. whatever parser it is that supports wikipedias will need to grapple with the roundtripping and html2wt issue.
21:31:45 <TimStarling> that's why we have so many protocols, because non-WMF users want to link to those things
21:32:25 * robla doesn't think the list cscott provided seems so crazy
21:32:29 <cscott> TimStarling: perhaps, but I think they could manage to use slightly different markup to accomplish the same task.
21:32:55 <TimStarling> like bracketed external links?
21:32:59 <cscott> exactly
21:33:23 <cscott> or double-bracketed links for some things perhaps
21:33:26 <TimStarling> [[Gopher (protocol)]] does of course have many gopher links on it
21:33:29 <Scott_WUaS> robla: agreed - cscott's list could make sense
21:33:40 <TimStarling> but all bracketed as far as I can see
21:33:50 <marktraceur> cscott: Pretty sure gopher:// is officially deprecated now, from what I vaguely recall
21:35:55 <cscott> so i'm going to be a poor advocate for legoktm and say I'm not particularly chuffed by magic links. i'd get rid of them for wikitext 2.0 as part of a broader simplification, but now that they are implemented and working it's not really helping us much to get rid of them.
21:36:24 <robla> so...perhaps we can agree to make the linking aspect off by default, without necessarily declaring them deprecated in 1.28
21:36:35 <cscott> i guess the question is whether you think we can slim down wikitext incrementally by turning off these weird crufty bits one by one, or whether we should treat it as a completely new markup language
21:36:48 <cscott> and use something like "parsoid2" to do round-trips between "wikitext 1" and "wikitext 2.0"
21:37:03 <subbu> what i think of wikitext 2.0 is probably different from what cscott thinks of wikitext 2.0 probably :)
21:37:15 * marktraceur may have been wrong, the minnpost article he must have seen was just a random summary of Gopher
21:37:27 <cscott> i imagine a language with a formal grammar, small enough to fit on a single sheet of paper.
21:37:36 * robla doesn't want to rabbithole on that topic, when we might actually be able to make a decision about Mediawiki 1.28
21:37:36 <cscott> but otherwise looking as much as possible like wikitext 1.0
21:37:36 <subbu> marktraceur, ah ... now it makes sense gopher came form UMn ... and that is why gopher
21:37:53 <marktraceur> cscott: How big is the piece of paper? Implementation details...
21:38:13 <robla> can we turn magic words off by default in 1.28 without deprecating?
21:38:27 <cscott> and so, listing 28 different external link prefixes and 12 or so separate productions for ISBN/RFC/PMID is not going to help wikitext 2.0 fit on a single sheet of paper
21:39:16 <robla> er...I suppose I should have said "magic links"
21:39:28 <robla> can we turn magic links off by default in 1.28 without deprecating?
21:40:01 <subbu> sure .. i guess the discussion is more about whether deprecation is required / desirable.
21:40:12 <cscott> https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pegTokenizer.pegjs.txt#L449 is the grammar for RFC/PMID/ISBN in parsoid, for reference.
21:40:46 <cscott> as far as i'm concerned the real question is: can we get the *wiki communities to start rewriting content to use whatever our preferred replacement markup is?
21:40:46 <robla> subbu: yes, but that seems to be sending us down the rabbithole of talking about Wikitext 2.0, which we aren't going to finish in this hour
21:41:06 <subbu> personally, i don't think we need to talk about wikitext 2.0 there.
21:41:20 <cscott> https://en.wikipedia.org/wiki/Template:ISBN was created by MZMcBride
21:42:01 <cscott> and it seems to already be used by quite a number of pages
21:42:15 <TimStarling> as a compromise, how about not deprecating it immediately, but revisit after a year or two and see how many people are turning on the option?
21:42:31 <subbu> i think it is a worthwhile discussion in its own merit. but, like cscott i don't have strong feelings ... but yes, it will simplify some code in parsoid .. but i won't be heartbroken if it is kept as is.
21:42:37 <cscott> one intermediate step might be to hack the parser to add a parser warning if you use magic link syntax, suggesting an appropriate rewrite
21:42:46 * robla likes TimStarling's suggestion
21:43:08 <cscott> we can do that w/o committing to deprecation, just encouraging people not to use that syntax.
21:43:14 <bd808> THen we'd just need a way of collecting that data
21:43:19 <cscott> and then, as TimStarling suggests, give it a year or so and see what our trends are.
21:43:30 <subbu> ya what bd808 says .. do we have a mechanism for collecting that data?
21:43:45 <cscott> we could probably hack together a dumpGrepper script that would collect repeatable numbers for long-term comparison purposes
21:43:47 <TimStarling> there's that pingback feature ori introduced...
21:44:06 <cscott> we could also add [[Category:Uses Magic Link Syntax]]
21:44:38 * robla looks for the link to the aforementioned pingback feature
21:44:44 <subbu> i know (or am i imagining it?) kaldari had strong feelings about magic links. curious if he is around.
21:44:49 <TimStarling> currently it doesn't send any configuration
21:45:31 <TimStarling> #link https://www.mediawiki.org/wiki/Manual:$wgPingback
21:46:03 <kaldari> My suggestion was to just kill the RFC magic linking entirely, as it's usefulness is very marginal
21:46:22 <kaldari> otherwise, I support making it configurable
21:46:42 <kaldari> and depricating
21:46:46 <bd808> I think that part has been done now. there's a feature flag for it
21:47:14 <bd808> and this discussion is about deprecation and removal from core
21:47:26 <TimStarling> the dynamic dates was removed, there is that precedent, but dynamic dates was never on by default and was rarely used by non-WMF wikis as far as we know
21:47:57 <robla> we could conceivably deprecate in 1.29 if the 1.28 release goes well
21:48:03 <TimStarling> whereas we've been told that magic links are regularly used
21:48:22 <subbu> ISBN in citations only from what I can tell.
21:48:31 <TimStarling> I mean outside WMF
21:48:35 <subbu> ah ..
21:48:40 <TimStarling> for WMF we can run bots, outside WMF not so much
21:48:57 <TimStarling> outside WMF we don't know if people even want to run bots, or if they like their magic links the way they are
21:49:04 <robla> the pingback feature would give us more certainty, but I think even just "did anyone complain?" might be a good enough test in this case
21:49:35 <bd808> or would anyone who would complain actually upgrade anyway...
21:50:35 <cscott> it would be nice if WMF published a bot framework and scripts for it w/ every major upgrade
21:50:53 <cscott> like how we upgrade database schemas, we could provide tools for external wikis to update their content
21:51:02 <bd808> I guess I'm not sure why it needs to die if there is a feature flag other than hypothetical future parser world that would like to not support the feature
21:51:18 <bd808> cscott: I think we call that pywikibot
21:51:22 <robla> I suppose deprecating in 1.28 would be the "be bold" way of doing it. we could be prepared to "undeprecate" if people hate that we turned it off
21:51:43 <cscott> bd808: sure, it's the "official" part and "with scripts for every major upgrade" which is missing.
21:51:47 <James_F> Do "off by default" changes need to go through the one-version-notice process?
21:52:07 <TimStarling> heh
21:52:14 <TimStarling> disable by default then silently remove in 1.29?
21:52:20 <James_F> Tut. :-)
21:53:05 <robla> James_F: I don't *think* we have a policy like that, but I suppose that may be what legoktm's other rfc covers in more depth
21:53:27 <bd808> cscott: I'm not sure I agree on "official" and WMF being intrinsically intertwined, but some suggested wikitext cleanup scripts would be an interesting thing to add when changing the syntax
21:53:38 <James_F> Sorry, yes, I more meant "would it be covered by the semi-in-practice policy?".
21:53:46 <robla> the deprecation rfc: T146965
21:53:47 <stashbot> T146965: [RfC] Deprecation policy for PHP code in MediaWiki - https://phabricator.wikimedia.org/T146965
21:53:52 <cscott> i'm always interested in process improvements to make it easier for us to evolve wikitext syntax
21:54:57 <TimStarling> so should we call the non-controversial parts of this approved?
21:55:13 * subbu cares less about syntax and more about underlying processing model / semantics except where the syntax gets in the way
21:56:09 <subbu> wfm .. but, explicitly listing out the non-controversial parts would be useful.
21:56:26 <TimStarling> #agreed 1. Disable the magic link functionality by default for the MediaWiki 1.28 release
21:56:58 <TimStarling> #agreed 2. Deprecate magic links on Wikimedia wikis (e.g. Wikipedia), providing alternatives for this functionality and tools to aid the migration
21:57:26 <robla> hmmm....I'd feel more comfortable with broader input on #2
21:57:42 <cscott> sure. and the controversial step 3 would be removing the magic link code from core (perhaps moving it to an extension)?
21:57:52 <robla> I suppose "start step #2" isn't yet controversial
21:58:01 <James_F> Yes.
21:58:10 <cscott> i'd propose 2(a) -- add a category and parser warning for magic links on wikimedia wikis, without officially deprecating the feature.
21:58:22 <James_F> cscott: To move to an extension it'd have to keep the horrible hooks into the PHP parser, though?
21:58:28 <cscott> then sit back and see if folks are getting the magic links cleaned up or not.
21:58:35 <TimStarling> #info no consensus on removing the functionality from MW or flagging its pending removal via deprecation
21:59:21 <TimStarling> no, step 3 is disabling magic links on WMF wikis
21:59:23 <cscott> James_F: yes, although in my big picture worldview we're adding a pluggable parser API, and gradually moving from the "old" PHP parser to a newer one.
21:59:32 * James_F nods.
21:59:34 <TimStarling> removing from MW core was 1b
21:59:45 <cscott> so the hooks would only stay in the legacy PHP parser. which, of course, you could keep running if you like and are willing to keep it maintained.
22:00:05 <bd808> that's one possible universe, yes cscott
22:00:06 <Scott_WUaS> (Thanks, All)
22:00:27 <cscott> bd808: i keep pushing the universe towards my platonic ideal of it. ;)
22:00:35 * cscott has to turn into a pumpkin
22:00:54 <bd808> I like the "everything in the core php app" universe better
22:01:03 <bd808> but that's why we have these chats
22:01:10 * robla plans to turn into a different type of squash
22:01:29 <bd808> its a dangerous time of year to be a pumpkin
22:01:37 <robla> srsly
22:01:46 <bd808> could be gutted and filled with candles at any point
22:01:47 <TimStarling> what was next week's RFC again robla?
22:01:57 <robla> next week we'll plan on talking about CREDITs file
22:01:59 * robla finds link
22:02:18 <robla> https://phabricator.wikimedia.org/T139300
22:02:20 <cscott> bd808: pluggable doesn't mean not monolithic or not php, btw
22:02:25 <cscott> just that things are decoupled
22:02:39 <cscott> so you can have a pure-PHP single-process markdown wiki if you like.
22:02:42 <TimStarling> ok, all done
22:02:45 <TimStarling> #endmeeting
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits