- Location: #wikimedia-office IRC channel
- Meeting type: Field narrowing (on migration plan)
- Time: 2016-06-08, Wednesday 21:00 UTC (2pm PDT, 23:00 CEST)
- Agenda:
- T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool
- Discussion about migration strategy
- T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool
- Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-06-08-21.01.html
- Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-06-08-21.01.txt
- Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-06-08-21.01.wiki
- Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-06-08-21.01.log.html
Meeting summary
- Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ (robla, 21:02:12)
- LINK: https://phabricator.wikimedia.org/E203 <-Phab event info, with links (robla, 21:03:17)
- LINK: https://www.mediawiki.org/wiki/Html5Depurate (robla, 21:12:22)
- LINK: https://github.com/wikimedia/html5depurate/blob/master/src/main/java/org/wikimedia/html5depurate/CompatibilitySerializer.java is a first step to teasing out exactly what tidy is doing, although I'd like to eventually document this properly on-wiki in english, not in code. (cscott, 21:12:54)
- LINK: https://github.com/tstarling/remex-html (TimStarling, 21:16:01)
- LINK: https://github.com/tstarling/remex-html parser that Tim is working on (robla, 21:16:47)
- first 15-20 minutes of the meeting have been about different Tidy alternatives (robla, 21:17:50)
- <subbu> we are replacing tidy with html5depurate .. that is where are headed right now. (robla, 21:18:16)
- <gwicke> we will have to do something about third party use sooner rather than later <robla> this is a topic for a different meeting (robla, 21:38:10)
- <cscott> i have (the start of a) slower pure-PHP implementation which might help with third party use. (note for that different meeting) (cscott, 21:39:18)
- <TimStarling> first it should be deployed to all wikis as a gadget or other opt-in tool (robla, 21:42:45)
- Tim intends to make the new html available to users as an experiment, by clicking a button to replace the old html with the new html, on request. (DanielK_WMDE__, 21:46:24)
- <TimStarling> we could have a special page as well, to make it linkable; /wiki/Special:NewParser/Barack_Obama or something (DanielK_WMDE__, 21:47:10)
- Tim suggests to run one instance of html5depurate on each app server, so mw can access it on localhost; html5depurate runs an integrated webserver (grizzly). (DanielK_WMDE__, 21:48:30)
- <robla> so, the order is 1) rollout html5depurate instances 2) rollout special page+gadget 3) rollout to first wikis 4) plan full rollout based on step 3 (?) (robla, 21:55:29)
- <robla> prerequisite for step 1: discussion with ops. prerequisite for step 3: discussion with liaisons, correct? <subbu> robla, sounds about right. (robla, 21:55:54)
- <DanielK_WMDE__> robla: (2) should also be discussed with comcom/liaisons, i think (robla, 21:57:21)
Meeting ended at 22:00:19 UTC.
People present (lines said)
- TimStarling (53)
- subbu (37)
- cscott (37)
- robla (36)
- gwicke (33)
- DanielK_WMDE__ (23)
- brion (6)
- wm-labs-meetbot` (3)
- stashbot (2)
- bd808 (1)
- Scott_WUaS (1)
- ebernhardson (1)
| 1 | 21:01:58 <robla> #startmeeting ArchCom RFC Meeting: T89331 Replace Tidy in MW parser with HTML 5 parse/reserialize |
|---|---|
| 2 | 21:01:58 <wm-labs-meetbot`> Meeting started Wed Jun 8 21:01:58 2016 UTC and is due to finish in 60 minutes. The chair is robla. Information about MeetBot at http://wiki.debian.org/MeetBot. |
| 3 | 21:01:58 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. |
| 4 | 21:01:58 <wm-labs-meetbot`> The meeting name has been set to 'archcom_rfc_meeting__t89331_replace_tidy_in_mw_parser_with_html_5_parse_reserialize' |
| 5 | 21:01:59 <stashbot> T89331: Replace Tidy in MW parser with HTML 5 parse/reserialize - https://phabricator.wikimedia.org/T89331 |
| 6 | 21:02:12 <robla> #topic Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ |
| 7 | 21:02:34 <robla> hi everyone! |
| 8 | 21:02:35 <subbu> o/ |
| 9 | 21:03:17 <robla> #link https://phabricator.wikimedia.org/E203 <-Phab event info, with links |
| 10 | 21:03:41 <cscott> hey there |
| 11 | 21:03:44 <TimStarling> I wrote a bit of an update in the task description |
| 12 | 21:04:25 <robla> TimStarling: you had wanted to have this discussion be about the migration, and not so much about the design choices, right? |
| 13 | 21:04:51 <TimStarling> yes |
| 14 | 21:05:25 <TimStarling> I think the design was decided last year, and we've gone ahead and implemented it, but there's still a fair bit of work to do to get it out the door |
| 15 | 21:05:48 <gwicke> could you perhaps give a short summary of how this fits into your larger-picture roadmap? |
| 16 | 21:06:35 <subbu> whose roadmap specifically? |
| 17 | 21:06:42 <TimStarling> in the forseeable future, we plan on maintaining both the MW parser and Parsoid |
| 18 | 21:06:45 <gwicke> the parsing team roadmap |
| 19 | 21:07:15 <TimStarling> so it makes sense to do some work to bring them closer together in terms of syntax |
| 20 | 21:08:14 <robla> there are two questions Tim brought up at the end of the description of T89331. Question 1 ... |
| 21 | 21:08:15 <stashbot> T89331: Replace Tidy in MW parser with HTML 5 parse/reserialize - https://phabricator.wikimedia.org/T89331 |
| 22 | 21:08:17 <robla> "Are we close enough now in visual diff testing to call that part of the project done? (96.79% showed less than 1% differences, 93.35% rendered with pixel-perfect accuracy.)" |
| 23 | 21:08:46 <subbu> I think there is benefit in and of itself to replace Tidy ... it has been requested for a long time. Plus, what TimStarling said about this bringing Parsoid and PHP parser output closer together. |
| 24 | 21:08:50 <gwicke> are you planning to add more DOM functionality in Html5Duperate? |
| 25 | 21:09:32 <TimStarling> cscott has some ideas about maybe using it for #balance |
| 26 | 21:09:40 <TimStarling> in which case I suppose it would need more DOM functionality |
| 27 | 21:09:46 <subbu> but, why is that relevant to this discussion? |
| 28 | 21:10:11 <gwicke> I'm mainly trying to understand where you are planning to go with Html5Duperate |
| 29 | 21:10:42 <subbu> ok .. |
| 30 | 21:11:02 <cscott> (one option for implementing #balance is to do the actual balancing in tidy or post-tidy, when we have clean serialized html5 with all tags matched, etc.) |
| 31 | 21:11:10 <gwicke> depending on your answers, it will be more or less required for third party users |
| 32 | 21:11:34 <TimStarling> currently we do not actually build a DOM in depurate, the parser gives us an event stream which we serialize |
| 33 | 21:12:10 <cscott> (one of *my* goals for a tidy replacement is obtaining better semantics for what the "tidy phase" does, exactly. tidy's actual transformations are not written down anywhere, and WMF wikis depend on their exact behavior.) |
| 34 | 21:12:22 <robla> #link https://www.mediawiki.org/wiki/Html5Depurate |
| 35 | 21:12:46 <subbu> gwicke, we want a tidy-replacement solution for 3rd party users and that replacement solution right now is html5depurate. but, other options are not ruled out for someone wanting to provide them. |
| 36 | 21:12:54 <cscott> https://github.com/wikimedia/html5depurate/blob/master/src/main/java/org/wikimedia/html5depurate/CompatibilitySerializer.java is a first step to teasing out exactly what tidy is doing, although I'd like to eventually document this properly on-wiki in english, not in code. |
| 37 | 21:13:18 <subbu> whatever that solution is has to solve the problems that we are solving right now, and for deployment on the wmf cluster, we need to solve the migration / how-to-roll-out problem. |
| 38 | 21:14:14 <brion> have we looked at https://github.com/Masterminds/html5-php for a pure-php solution? |
| 39 | 21:14:15 <cscott> if we can separate the "tidy compatibility" part from the html5 parsing/serialization part, we can (a) more easily use different html5 parsing/serialization solutions, depending on performance, ops considerations, etc, and (b) gradually deprecate the strangest corners of the "tidy compatiblity" part. |
| 40 | 21:14:25 <gwicke> so, are you planning to basically develop html5depurate to eventually be equivalent to Parsoid's DOM passes? |
| 41 | 21:14:38 <TimStarling> brion: yes, I think I had some notes about it on the task |
| 42 | 21:14:45 <brion> nice, i'll read up |
| 43 | 21:15:10 <TimStarling> yeah, show older comments and then search for it |
| 44 | 21:15:36 <TimStarling> I'm actually working on my own HTML 5 parser in PHP now |
| 45 | 21:15:42 <cscott> afaik, html5depurate is intended to be html5 parse+serialize only, with "as little as possible" of mediawiki-specific compat stuff. this means that some of what tidy is doing we end up moving into php. |
| 46 | 21:15:53 <subbu> gwicke, we haven't gotten that far yet since the core parser doesn't provide the functionality needed to implement Parsoid's DOM passes on top of that. |
| 47 | 21:16:01 <TimStarling> https://github.com/tstarling/remex-html |
| 48 | 21:16:22 <gwicke> so, you are planning to have a DOM in PHP, too? |
| 49 | 21:16:28 <subbu> so, short answer .. we won't be implementing those dom passes in depurate. |
| 50 | 21:16:34 <cscott> for example, https://gerrit.wikimedia.org/r/286928 moves some tidy-specific self-closing tag functionality into the sanitizer, so it doesn't have to be part of tidy (any of the possible tidy implementations) |
| 51 | 21:16:47 <robla> #link https://github.com/tstarling/remex-html parser that Tim is working on |
| 52 | 21:17:04 <subbu> gwicke, still trying to figure how that discussion is relevant to tidy replacement. |
| 53 | 21:17:08 <brion> ah looks like it (html5-php) may not handle all error corrections correctly per spec, which is worrying :) |
| 54 | 21:17:21 <gwicke> subbu: still trying to figure out where you are headed with this |
| 55 | 21:17:28 <cscott> and i think tim's doBlockLevels work is also intended to move more of the fixups done by tidy (sometimes incorrectly) into core. right? |
| 56 | 21:17:33 <subbu> we are replacing tidy with html5depurate .. that is where are headed right now. |
| 57 | 21:17:50 <robla> #info first 15-20 minutes of the meeting have been about different Tidy alternatives |
| 58 | 21:18:16 <robla> #info <subbu> we are replacing tidy with html5depurate .. that is where are headed right now. |
| 59 | 21:18:35 <TimStarling> doBlockLevels is actually generating invalid HTML from valid input |
| 60 | 21:18:43 <TimStarling> creating an awful mess for tidy to clean up |
| 61 | 21:18:44 <cscott> brion: just for completeness, https://gerrit.wikimedia.org/r/#/c/279669/7/includes/tidy/Balancer.php is another html5 "parser" (actually just the treebuilder phase) written in PHP. |
| 62 | 21:18:44 <cscott> "pp |
| 63 | 21:18:49 <TimStarling> I would like it to not do that in the first place |
| 64 | 21:18:59 <brion> cscott: tx |
| 65 | 21:19:44 <cscott> right, so the general idea is to gradually move to the place where "tidy" is just a standards-compliant html5 parse-and-reserialize, and all the other fixups are done in core (or avoided entirely, like with the doBlockLevels fixes) |
| 66 | 21:19:59 <gwicke> this sounds a lot like parsoid |
| 67 | 21:20:01 <cscott> we're not going to get there in one leap, the initial html5depurate will have some tidy compatibility hacks still. |
| 68 | 21:20:41 <cscott> gwicke: one goal is easier compatibility between php and parsoid, yes. but parsoid doesn't have a separate 'tidy' phase really. |
| 69 | 21:20:52 <TimStarling> I wonder if we should make Sanitizer into a proper balancing HTML parser |
| 70 | 21:21:26 <gwicke> it does beg the question if you are planning to gradually convert the PHP parser into a Parsoid port |
| 71 | 21:21:30 <TimStarling> it would simplify the main pass if the input were valid |
| 72 | 21:21:49 <cscott> well, parsoid is just differently structured. parsoid does token stream -> manipulations -> tree builder phase -> final manipulations. php does wikitext -> partially parsed html -> sanitizer -> doBlockLevels -> tidy -> languageconverter. or something like that. |
| 73 | 21:22:02 <TimStarling> gwicke: we've already talked about that twice, I don't really want to get into it again |
| 74 | 21:22:39 <robla> should we talk about visual diff test pass requirements? |
| 75 | 21:22:52 <subbu> i think this discussion is side-tracking into all the other things we can do with depurate / what the parsing team is doing / might be doing .. and is not about whether html5deuprate is an acceptable tidy replacement and what blocks its deployment. |
| 76 | 21:23:02 <gwicke> TimStarling: it would be interesting if you could clarify what your answer is |
| 77 | 21:23:04 <cscott> TimStarling: the input to removeHTMLtags is not fully-parsed HTML, so we can't actually do a proper HTML5 parse at that point. I tried that once. |
| 78 | 21:23:07 <TimStarling> robla: I think there are no comments on that |
| 79 | 21:23:21 <cscott> TimStarling: OTOH arlo has been gradually fixing the attribute regexps in the sanitizer (for example) to be html5-spec-compliant. |
| 80 | 21:23:38 <subbu> but https://phabricator.wikimedia.org/T134469#2281710 are my thoughts. |
| 81 | 21:24:15 <subbu> i'll prepare a wiki page addressing questions about parsing team roadmap / 2 parsers / etc .. i don't think that discussion is relevant to this rfc. |
| 82 | 21:24:27 <TimStarling> we need to start getting the community involved in text migration |
| 83 | 21:24:47 <gwicke> subbu: it matters wrt third party support, which is related to the migration |
| 84 | 21:24:53 <TimStarling> then they will have opinions on diff targets |
| 85 | 21:25:33 <subbu> gwicke, as it stands today .. TimStarling has written an abstract interface into which a tidy replacement can be dropped, and html5depurate is one viable option. |
| 86 | 21:25:37 <robla> TimStarling: I guess that's what I meant by requirements. I'm assuming you're trying to figure out how to get editors engaged on fixing the few bugs that are past the point of diminishing returns, right? |
| 87 | 21:26:27 <subbu> if a pure php alternative is avaialble that can be used .. if parsoid is available, then parsoid can be used ... but if parsoid is used for read views there, tidy is irrelevant ... so many possibilities. |
| 88 | 21:26:48 <subbu> but, for mediawiki installs that don't want parsoid, parsoid is not a solution for replacying tidy. |
| 89 | 21:26:49 <TimStarling> robla: yes, although diffs are not necessarily bugs |
| 90 | 21:27:28 <TimStarling> we don't really want to have exactly the same behaviour as tidy because we don't actually like tidy's behaviour |
| 91 | 21:27:49 <TimStarling> so it comes down to rolling out changes gradually, providing tools for editors |
| 92 | 21:27:57 <subbu> robla, https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy has a classification of diffs we currently see and possible strategies for addressing them. |
| 93 | 21:28:08 <TimStarling> perhaps James_F would have a sense of how the project would go |
| 94 | 21:28:27 <robla> do you envision a heterogeneous deployment for this? (i.e. one specific wiki first, then rolling out more widely)? |
| 95 | 21:28:28 <cscott> again, for completeness: https://gerrit.wikimedia.org/r/282733 is 90% of a pure-PHP implementation of html5depurate. It's not as fast as html5depurate, so WMF probably wouldn't use that in production, but it could ease the political issue for non-WMF users. |
| 96 | 21:28:52 <gwicke> if html5depurate is the equivalent of a HTML5 round-trip, then there are many alternative implementations |
| 97 | 21:29:15 <gwicke> I'm mostly worried about other features that require a DOM |
| 98 | 21:29:27 <TimStarling> it's an HTML parse with a custom serializer |
| 99 | 21:29:41 <TimStarling> *not* a compliant serializer |
| 100 | 21:30:00 <cscott> gwicke: sure. but only a pure PHP implementation really helps with the political problem. |
| 101 | 21:30:05 <subbu> right now, html5depuate it implements additional some simple tidy-like transforms to ease the migration from tidy .. otherwise the diffs would be numerous. the plan is to remove those passes one at a time using visual idffing and the process we come up with now ... that will migrate syntax gradually. |
| 102 | 21:30:21 <cscott> TimStarling: "not a compliant serializer" meaning the tidy-compatibility hacks? or the XHTML-compatibility hacks? or something else? |
| 103 | 21:30:35 <TimStarling> the tidy compatibility hacks |
| 104 | 21:30:39 <cscott> ok. sure. |
| 105 | 21:30:50 <TimStarling> the XHTML hacks will give you roughly the same thing if you reparse it |
| 106 | 21:31:01 <gwicke> cscott: there are two schools of thought there- one is that really the most important thing for third party users is a) resource needs, and b) setup / maintenance complexity |
| 107 | 21:31:08 <gwicke> the other is that it has to be PHP |
| 108 | 21:31:58 <cscott> +1 to subbu's comment about gradually migrating syntax. |
| 109 | 21:32:01 <brion> those are of course two separate things :) but related, in that extra daemons/languages/binaries add to the setup cost |
| 110 | 21:32:08 <DanielK_WMDE__> gwicke: the argument being that PHP still provides the best bang for the buck in terms of (b) |
| 111 | 21:32:11 <gwicke> how html5depurate fits into either remains to be seen; for the first case, it depends a lot on its resource needs |
| 112 | 21:32:25 <cscott> the "90% implementation" i mention above doesn't have tim's tidy compatibility hacks yet, that's the part that's still missing there. |
| 113 | 21:33:08 <gwicke> we can solve the setup issue with containers & automation (see mediawiki-containers) |
| 114 | 21:33:14 <subbu> gwicke, this is rehashing the first RFC discussion. |
| 115 | 21:33:38 <gwicke> what is the expected memory usage of html5depurate? |
| 116 | 21:33:41 <subbu> what are yo ugetting at exactly? |
| 117 | 21:34:06 <subbu> are you saying we shouldn't deploy html5depurate on the wmf cluster? |
| 118 | 21:34:20 <cscott> i didn't mean to distract us with my mention of the pure php option. in fact, what i meant was the opposite -- we should feel free to ignore some of the political issues wrt services because we might be able to do an end-run around them. |
| 119 | 21:34:30 <gwicke> memory isn't an issue on the cluster, but it is a critical resource for small VMs |
| 120 | 21:34:47 <cscott> so hopefully we can concentrate on wmf needs, and then have the "what's best for other deployments" as a separate discussion |
| 121 | 21:34:47 <robla> ok, I think this meeting is about deployment to the wmf cluster, no? |
| 122 | 21:35:33 <robla> so...I'd like to reask my earlier question: which production wiki would you plan to deploy to first? |
| 123 | 21:35:49 <DanielK_WMDE__> i'm not quite clear on where the service is going to run. it's not going to be a lonely tomcat instances somewhere, will it? |
| 124 | 21:36:20 <gwicke> we will have to do something about third party use sooner rather than later |
| 125 | 21:36:22 <TimStarling> it can run with one instance per MW host |
| 126 | 21:36:27 <TimStarling> listening on localhost |
| 127 | 21:36:33 <DanielK_WMDE__> can i read the plan for the service setup somewhere? |
| 128 | 21:36:34 <robla> gwicke: not in this meeting, though |
| 129 | 21:37:06 <gwicke> certainly not *doing*, but I think we'll need a plan soon |
| 130 | 21:37:09 <TimStarling> the plan is to basically install the existing debian packaging, and use it with the bundled configuration |
| 131 | 21:37:55 <DanielK_WMDE__> silly question: *are* we using tomcat? is it decent these days? |
| 132 | 21:38:10 <robla> #info <gwicke> we will have to do something about third party use sooner rather than later <robla> this is a topic for a different meeting |
| 133 | 21:38:17 <DanielK_WMDE__> (another silly question: can we use unix sockets instead of tcp-over-loopback?) |
| 134 | 21:38:27 <TimStarling> no, it's grizzly, it does its own daemonization and has an embedded grizzly webserver |
| 135 | 21:38:29 <subbu> TimStarling, can you also address robla's questions about deployment? |
| 136 | 21:38:50 <TimStarling> I started off in tomcat but switched to grizzly to make it easier to manage |
| 137 | 21:38:53 <DanielK_WMDE__> what servlet engine are we using for other stuff? |
| 138 | 21:38:55 <gwicke> if you are committing to addressing this issue as a team in a way that doesn't result in a significant change in overall MW system resource requirements, then this works for me |
| 139 | 21:39:13 <TimStarling> there is still a servlet class in the source tree but I'm not sure if it still works |
| 140 | 21:39:18 <cscott> #info <cscott> i have (the start of a) slower pure-PHP implementation which might help with third party use. (note for that different meeting) |
| 141 | 21:39:37 <bd808> DanielK_WMDE__: we don't have other servlets AFAIK |
| 142 | 21:39:43 <TimStarling> most java services do the same, they run an embedded webserver |
| 143 | 21:39:51 <TimStarling> that's how cirrus and gerrit work |
| 144 | 21:40:44 <DanielK_WMDE__> TimStarling: ah ok, i only saw that you talked about tomcat on the ticket, and I got worried ;) |
| 145 | 21:41:09 <robla> subbu: I think the answer to your question is "no" :-) |
| 146 | 21:41:21 <TimStarling> <robla> so...I'd like to reask my earlier question: which production wiki would you plan to deploy to first? |
| 147 | 21:41:30 <DanielK_WMDE__> TimStarling: also "is tomcat decent these days" - "no, it's grizzly" actually makes sense in it's own way ;) |
| 148 | 21:41:47 <TimStarling> first it should be deployed to all wikis as a gadget or other opt-in tool |
| 149 | 21:42:12 <TimStarling> when it is fully deployed, I'm not sure, maybe it would just follow the release train |
| 150 | 21:42:44 <TimStarling> the weekly train is not a bad deployment sequence |
| 151 | 21:42:45 <robla> #info <TimStarling> first it should be deployed to all wikis as a gadget or other opt-in tool |
| 152 | 21:42:45 <DanielK_WMDE__> TimStarling: how would the opt in work? split or skip the parser cache based on user preferences, or a cookie or something? |
| 153 | 21:42:51 <cscott> why not deploy to meta first? |
| 154 | 21:43:05 <gwicke> performance would likely be poor when used as a beta feature, because of parser cache misses |
| 155 | 21:43:06 <subbu> I think what we don't have full clarity is at what point we pull the switch on this being the default .. since there will be some rendering diffs that require fixing syntax on pages. |
| 156 | 21:43:29 <subbu> and when do we just deploy it and have some pages look broken and rely on editors fixing up the wikitext / templates. |
| 157 | 21:43:42 <TimStarling> maybe fetch a DocumentFragment via an API, switch it out when you click a button |
| 158 | 21:43:52 <gwicke> there are communities that are very open to trying new things, like Catalan |
| 159 | 21:43:53 <DanielK_WMDE__> i feel per-user opt-in makes this pretty tricky, and will be confusing to users. they can't easily show issues to each other, either... |
| 160 | 21:44:05 <TimStarling> maybe have a position:fixed button hovering over the page so you can flip back and forth |
| 161 | 21:44:13 <subbu> our visual diffing results and documentation on https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy gives us a good basis to make claims about what kind of things might possibly break. |
| 162 | 21:44:30 <gwicke> mw.org could also be a good testing ground |
| 163 | 21:44:31 <DanielK_WMDE__> TimStarling: ah, you want this on demand, with user interaction. that would be ok I guess. |
| 164 | 21:44:44 <subbu> gwicke, yes .. mw.org and catalan wikis are a good idea. |
| 165 | 21:44:48 <DanielK_WMDE__> otherwise, i'd have suggested to do per-namespace experiments |
| 166 | 21:45:01 <subbu> and cscott mentioned meta as well. |
| 167 | 21:45:35 <TimStarling> we could have a special page as well, to make it linkable |
| 168 | 21:45:58 <TimStarling> /wiki/Special:NewParser/Barack_Obama or something |
| 169 | 21:45:59 <cscott> commons ought to be safe-ish as well, since in theory most of the content there should be reasonably well-formed. |
| 170 | 21:46:05 * robla tries to remember who at WMF is most appropriate to loop into a rollout discussion |
| 171 | 21:46:22 <gwicke> liaisons and james might have other ideas for communities to target |
| 172 | 21:46:24 <DanielK_WMDE__> #info Tim intends to make the new html available to users as an experiment, by clicking a button to replace the old html with the new html, on request. |
| 173 | 21:46:37 <brion> nice |
| 174 | 21:47:10 <DanielK_WMDE__> #info <TimStarling> we could have a special page as well, to make it linkable; /wiki/Special:NewParser/Barack_Obama or something |
| 175 | 21:47:50 <robla> thanks DanielK_WMDE__ and thanks TimStarling . That clarifies the strategy a lot |
| 176 | 21:48:29 <subbu> TimStarling, robla what about deployment to individual wikis at some point .. like mw.org, meta .. before doing a full rollout? |
| 177 | 21:48:30 <DanielK_WMDE__> #info Tim suggests to run one instance of html5depurate on each app server, so mw can access it on localhost; html5depurate runs an integrated webserver (grizzly). |
| 178 | 21:49:35 <robla> subbu: yeah, I think that'll be a must. I think we'll need to get someone from liaisons to weigh on rollout ordering |
| 179 | 21:49:45 <subbu> k |
| 180 | 21:50:16 <DanielK_WMDE__> who is going to decide on where service instances live, how many we need, how they get managed, etc? |
| 181 | 21:50:21 <gwicke> mw.org has a good dogfooding factor |
| 182 | 21:50:22 <DanielK_WMDE__> that would be the next thing that needs planning, right? |
| 183 | 21:50:41 <gwicke> operational ownership as well |
| 184 | 21:50:43 <DanielK_WMDE__> hmmm dog food... |
| 185 | 21:50:58 <robla> om nom nom |
| 186 | 21:51:14 <TimStarling> yes, we will need to have a conversation with ops about it |
| 187 | 21:51:39 <gwicke> are there already metrics & logging? |
| 188 | 21:51:40 <TimStarling> I imagine the instances would be fully rolled out before the gadget deployment |
| 189 | 21:52:01 <TimStarling> logging yes, metrics no |
| 190 | 21:52:22 <TimStarling> the package installs a local log file |
| 191 | 21:52:27 <ebernhardson> w |
| 192 | 21:52:40 <DanielK_WMDE__> TimStarling: one per app server makes communication and setup easy, but may be overkill. do you think we need that many instances? |
| 193 | 21:53:28 <robla> so, the order is 1) rollout html5depurate instances 2) rollout special page+gadget 3) rollout to first wikis 4) plan full rollout based on step 3 (?) |
| 194 | 21:54:16 <DanielK_WMDE__> hm, most parsing doesn't happen on web accessible hosts, but on the jobqueue runners, right? |
| 195 | 21:54:22 <TimStarling> DanielK_WMDE__: I don't mind, I think that plan came out of a previous IRC discussion but it can go wherever ops wants it to go |
| 196 | 21:54:24 <subbu> step 0: talk with ops, puppetize, etc. i suspect. |
| 197 | 21:54:50 <robla> prerequisite for step 1: discussion with ops. prerequisite for step 3: discussion with liaisons, correct? |
| 198 | 21:54:51 <DanielK_WMDE__> TimStarling: yea, i'm just saying, we'll need a plan for that :) |
| 199 | 21:55:04 <subbu> robla, sounds about right. |
| 200 | 21:55:10 <cscott> 3) is a wmf-config change, should be easy to quickly rollback if there are any problems. |
| 201 | 21:55:29 <robla> #info <robla> so, the order is 1) rollout html5depurate instances 2) rollout special page+gadget 3) rollout to first wikis 4) plan full rollout based on step 3 (?) |
| 202 | 21:55:39 <gwicke> subbu: are you planning to help with third-party support work? |
| 203 | 21:55:54 <robla> #info <robla> prerequisite for step 1: discussion with ops. prerequisite for step 3: discussion with liaisons, correct? <subbu> robla, sounds about right. |
| 204 | 21:56:14 <DanielK_WMDE__> robla: (2) should also be discussed with comcom/liaisons, i think |
| 205 | 21:56:28 <cscott> 2a) is publicize the gadget well in tech news so that folks can help fix up bad markup, know what to look for, etc. |
| 206 | 21:56:34 <robla> we're coming up on the top of the hour |
| 207 | 21:57:08 <subbu> gwicke, we can have additional conversation about that .. but initially, a simple solution is: tidy replacement is not recommended for 3rd parties till we have some experience with this .. that is just my thought .. cscott and TimStarling might have other ideas. |
| 208 | 21:57:21 <robla> #info <DanielK_WMDE__> robla: (2) should also be discussed with comcom/liaisons, i think |
| 209 | 21:57:58 <TimStarling> yeah, not many benefits for third parties at this point |
| 210 | 21:58:13 <gwicke> vagrant is another question |
| 211 | 21:58:20 <cscott> there's a bit of a messaging question there: do we pitch this as "just a faster tidy, don't worry about it if you don't have a huge wiki"? |
| 212 | 21:58:27 <robla> TimStarling: anything else you want to make sure we cover in this hour, or should I plan on hitting #endmeeting in a minute? |
| 213 | 21:58:27 <Scott_WUaS> good discussion! |
| 214 | 21:58:39 <cscott> if we pitch it as "cleaner markup" or something like that, then folks may wish to run it immediately |
| 215 | 21:58:44 <subbu> ya .. adventurous wikis might decided to go with it, but it is at their own risk (like broken rendering, need to fix wikitext, templates, etc.). |
| 216 | 21:58:51 <TimStarling> there's no time for new topics |
| 217 | 21:59:03 <cscott> we'll have it packaged, so there's nothing *stopping* 3rd parties from running it |
| 218 | 21:59:11 <subbu> config option |
| 219 | 21:59:14 <robla> discussion can continue on #mediawiki-parosid? |
| 220 | 21:59:14 <cscott> it's just we won't push it as a "you should run this". not yet at least. |
| 221 | 21:59:23 <cscott> at some point in the future we'll probably deprecate the WMF fork of tidy. |
| 222 | 21:59:24 <subbu> robla, works for me. |
| 223 | 21:59:29 <robla> parsoid even? :-) |
| 224 | 21:59:30 <TimStarling> #mediawiki-parsoid perhaps |
| 225 | 21:59:32 <cscott> (since WMF doesn't even run stock tidy) |
| 226 | 21:59:46 * cscott has to turn into a pumpkin |
| 227 | 21:59:52 * DanielK_WMDE__ is getting parasoid |
| 228 | 21:59:53 <gwicke> to clarify, I strongly support the general direction of moving to HTML5 parsing over tidy; my concerns are mostly about the details of the implementation |
| 229 | 21:59:59 <cscott> i'm just saying let's be careful about messaging. |
| 230 | 22:00:00 <subbu> gwicke, thanks. |
| 231 | 22:00:02 <robla> alright meeting turning into a pumpkin in a few seconds :-) |
| 232 | 22:00:17 <robla> thanks everyone! |
| 233 | 22:00:19 <robla> #endmeeting |
| Architecture meetings | ||
|---|---|---|
| 13:00 PT ArchCom Planning Meetings | upcoming | all since 2016-03-30 |
| 14:00 PT ArchCom-RFC Meetings | upcoming | all since 2015-09-09 |