HomePhabricator

RFC Meeting: Review backlog, Thumb API (2016-03-16, #wikimedia-office) (NOTE: U.S. Daylight Savings shift)
ActivePublic

Hosted by daniel on Mar 16 2016, 9:00 PM - 10:00 PM.

Description

Agenda

Meeting summary

  • wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ | https://phabricator.wikimedia.org/E148 (robla, 21:00:53)
    • LINK: https://phabricator.wikimedia.org/T66214 (gwicke, 21:01:56)
    • topic discussed: where we add extra parameters (page, time, language, etc) is the format sufficiently clearly specified? (robla, 21:04:13)
    • to discuss in future: implementation details for small/large installs (backlog) (brion, 21:05:47)
    • avoiding cache fragmentation will require normalization of parameters (deterministic serialization of hashmap or other ordering) (brion, 21:19:50)
    • daniel suggests having the failed-validation case be a redirect, gwicke warns they're expensive (brion, 21:20:24)
    • normalizing parameters within varnish would be painful (brion, 21:20:43)
  • T78639: How to address the long tail of low priority tasks in active projects (robla, 21:21:36)

Meeting ended at 22:00:02 UTC.

People present (lines said)

  • gwicke (53)
  • DanielK_WMDE__ (47)
  • brion (45)
  • robla (28)
  • bd808 (19)
  • andre__ (17)
  • greg-g (16)
  • legoktm (10)
  • SMalyshev (7)
  • stashbot (6)
  • Purodha (4)
  • Scott_WUaS (4)
  • wm-labs-meetbot` (3)
  • jzerebecki (2)
  • ebernhardson (2)
  • chasemp (1)

IRC log

122:00:22 <TimStarling> #startmeeting RFC meeting
222:00:22 <wm-labs-meetbot`> Meeting started Wed Mar 9 22:00:22 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot.
322:00:22 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
422:00:22 <wm-labs-meetbot`> The meeting name has been set to 'rfc_meeting'
522:00:36 <TimStarling> #topic RFC meeting | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/
622:01:12 <TimStarling> so I guess we're doing notifications first
722:01:31 <brion> yup
822:01:33 <legoktm> hi
922:01:34 <robla> T128351
1022:01:34 <stashbot> T128351: RfC: Notifications in core - https://phabricator.wikimedia.org/T128351
1122:01:36 <brion> first in first out
1222:01:39 <gwicke> +1
1322:01:54 <TimStarling> and thumb URLs will be the end of this week plus maybe next week as well?
1422:02:32 <Scott_WUaS> great!
1522:02:57 <gwicke> TimStarling: yes, depending on where we end up today
1622:02:57 <legoktm> I updated https://www.mediawiki.org/wiki/Requests_for_comment/Notifications_in_core recently with open questions and implementation plan sections
1722:04:01 <legoktm> I thought consenus was mostly leaning to sticking most of Echo into core (with some specific open questions), but Krinkle posted something different on the bug just now
1822:04:16 <matt_flaschen> https://phabricator.wikimedia.org/T128351#2104701
1922:05:31 <matt_flaschen> Krinkle basically suggested keeping all the DB code and UI outside core, and making it pluggable.
2022:05:40 <robla> legoktm: there are five open questions there, should we step through them?
2122:06:10 <TimStarling> making it pluggable would be more work
2222:06:43 <matt_flaschen> Yes, my first question is whether anyone even wants to make an alterate implementation.
2322:06:58 <legoktm> robla: my questions assumed people were in agreement that we should be moving most of the Echo backend stuff into core, but if people agree with Krinkle, those questions don't make sense
2422:07:01 <gwicke> the main thing you need is to define a narrow interface
2522:07:22 <gwicke> which is useful in its own right
2622:07:25 <Krinkle> It could be as sample as taking Echo's base classes, converting to an interface and adding to core. And in the process we'd also force separation between backend and logic handling (similar to what we're working on within mw for some areas)
2722:07:29 <legoktm> so we should first figure out whether we want to have some kind of pluggable thing Krinkle proposed, or just have Echo as the core notification implementation
2822:07:39 <ashley> +2 to merging Echo into core (finally, if I may add)
2922:07:49 <Krinkle> the latter is a separate thing, but I think people do agree on should happen regardless.
3022:08:01 <TimStarling> and the rationale is to do a more gradual rollout to non-WMF users?
3122:08:39 <jdlrobson> +1 Krinkle that was my thoughts on this
3222:08:49 <brion> well if we have a plain API for it, we either need a core implementation (like Search and AuthPlugin) or to consider mediawiki as not requiring it
3322:08:50 <Krinkle> Yeah. It's been very little exposed to third parties. I think we should either halt the merge and do a bundle for a first release cycle first. Or only merge the interfaces and then bundle and stick with just that.
3422:08:51 <jdlrobson> the interface is needed by skins, but they don't need to care about how it's implemented
3522:09:05 <brion> what would be the behavior if an extension fires off notifications on a wiki with no notification backend?
3622:09:19 <gwicke> a no-op default has been proposed
3722:09:26 <bd808> brion: >/dev/null ?
3822:09:32 <brion> that sounds bad to me
3922:09:37 <matt_flaschen> jdlrobson, how is interface needed by skins? My understanding is people are talking about API interface.
4022:09:40 <Krinkle> brion: Yeah, just like core's RC notificaitons go no where by default.
4122:09:45 <TimStarling> jdlrobson: are you agreeing with Krinkle that echo should not be brought into core?
4222:09:45 <Krinkle> not to redis or UDP
4322:09:50 <gwicke> I think it is useful for this discussion to separate a) interface, and b) default / alternate implementations
4422:10:07 <gwicke> a) is fairly orthogonal to b)
4522:10:07 <matt_flaschen> See also my question at https://phabricator.wikimedia.org/T128351#2104841 . How do we handle places where Echo replaces core functoinality (e.g. email notif for user talk)?
4622:10:10 <Krinkle> But the interface for creating RC entries and having an RCFeedEngine are in core.
4722:10:11 <brion> Krinkle: RC events aren't user-facing for most values of user, whereas this is entirely user-focused
4822:10:21 <Krinkle> brion: True.
4922:10:36 <jdlrobson> TimStarling: i'm still collecting my thoughts but my immediate reaction when I saw the RFC was interfaces should be defined in core, not the entirety of the Echo system.
5022:10:49 <duploktm> sorry, my other client is lagging
5122:10:53 <Krinkle> brion: But I don't think we're too worried about exposing Echo as an API inside HTML. So that'll work the same way either way. Point being, what do we gain by having Echo in core?
5222:10:56 <brion> i can't envision many (any?) circumstances where as a core or extension author i would expect firing off a notification to a user to do nothing to be acceptable
5322:11:23 <brion> we gain a consistent way for core and extension authors to provide software->user communications path
5422:11:33 <brion> the consistent part comes from it always working, not just having an internal api :)
5522:11:34 <jdlrobson> Echo seems like an implementation of a NotificationsEngine and I could imagine other notifications engines involving email and even the orange bar of death ;-)
5622:11:51 <Krinkle> brion: Right, so you're comparing this similar to EmailUser. It can't go nowhere by default since the application intent is to reach a user, not just to provide information in general.
5722:12:00 <matt_flaschen> Krinkle, main reason i think is to avoid duplicate implementations (e.g. email for user talk notif) with hooks galore, second to that is that it's an important framework, and thus arguably important enough to go into core.
5822:12:01 <brion> *nod*
5922:12:04 <duploktm> Echo has a lot of things inside of it, and I think those things are being confused.
6022:12:13 <jdlrobson> matt_flaschen: Right now Echo adds itself to menus via horrible hooks and I'd prefer to see it replace some concept in core. e.g. NotificationsEngine = 'Echo'
6122:12:22 <matt_flaschen> jdlrobson, in theory you can make anything pluggable, but in practice does anyone actually plan to write that? It's a lot of code.
6222:12:36 <matt_flaschen> jdlrobson, and what happens if there is no engine and no Echo. Do you just not get email notif for user talk change?
6322:12:58 <duploktm> Echo is a notifications framework, which includes a pluggable backend for how to deliver notifications (web, email, etc.), a system for formatting those notifications for each output type
6422:13:30 <gwicke> matt_flaschen: defining interfaces does not necessarily imply doing the legwork to set up fancy plugging infrastructure behind those interfaces
6522:13:50 <TimStarling> duploktm: that is an excellent nickname
6622:13:54 <duploktm> The main problem I see is that there are already notification types in core (enotifs, watchlists) which sometimes conflict with Echo, and require bad hook hacks to fight with
6722:14:03 <matt_flaschen> gwicke, I'm responding to Krinkle and jdlrobson (both of whom did propose making it pluggable)
6822:14:05 <brion> so it sounds like echo *is* the thing that defines pluggable backend interfaces
6922:14:37 <matt_flaschen> brion, note Krinkle even proposed making the DB backend pluggable. Echo does not support that and there has been no prior discussion of that.
7022:15:04 <TimStarling> brion: which brings into question the idea of using echo base classes as the core notification API
7122:15:09 <ori> what is the point of moving it into core, or define an interface in core?
7222:15:11 <duploktm> I'm all for interfaces and separating business and frontend logic and whatnot, but I don't think that separation needs to be enforced by extension vs. core and physical git repos
7322:15:24 <Krinkle> matt_flaschen: You can scrap that detail. I'm not pushing for that. That could be injected to the handler instead of being known to core directly.
7422:15:24 <ori> is there a clear articulation of the value somewhere?
7522:15:29 <TimStarling> echo would become a collection of backends instead of a single thing
7622:15:38 <matt_flaschen> Okay, fair enough.
7722:15:48 <duploktm> ori: https://www.mediawiki.org/wiki/Requests_for_comment/Notifications_in_core#Problem
7822:15:49 <brion> ori: to ensure it is always available so can be used and relied upon by core and extension authors making features for users
7922:15:49 <Krinkle> And I agree there isn't much value in anticipating a backend and handling logic from different authors at this point.
8022:15:51 <brion> least that's my goal ;)
8122:15:56 <jdlrobson> Note that MediaWiki doesn't ship with a default skin, you have to install a skin. I'd expect a similar kind of abstraction with Echo.
8222:16:03 <matt_flaschen> ori, I think main concrete benefit is that we don't have to have to versions of the same thing (e.g. user talk notification). Later on, we could also look at unifying watchlist and Echo if it makes sense.
8322:16:15 <matt_flaschen> The idea "important infrastructure should be in core" might also apply, but isn't concrete.
8422:16:24 <duploktm> ori: Another thing people noted on the phab task is that Echo and notifications in general have become an integral part of using MediaWiki - at least on Wikimedia wikis where it is installed
8522:17:04 <ori> Suppose we move it into core. The extensions that want to make use of the new Echo functionality in Core have to either:
8622:17:29 <ori> (a) drop the old code that works with the Echo extension, and express a dependency on the version of MediaWiki that introduced this work
8722:17:57 <ori> (b) keep the old code, and add _additional_ class_exists & co checks
8822:18:03 <brion> a) is how we usually roll
8922:18:26 <matt_flaschen> jdlrobson, again, why? At the time the decision was made to stop shipping skins, there were already many skins (in core and elsewhere). There is only one notification framework I know of.
9022:18:26 <ori> so why not just express a dependency on Echo?
9122:18:29 <TimStarling> yup (a)
9222:18:43 <duploktm> ori: sorry, but did you read the RfC? I proposed keeping a back-compat shim with old class names so neither a nor b would be necessary
9322:19:11 <TimStarling> ori: because of the reasons on the wiki page that duploktm just linked to
9422:19:47 <ori> you mentioned "a back-compat layer for things that haven't transitioned to core notifs yet " as something that would stay in Echo
9522:19:54 <ori> not that there would be a shim in core
9622:20:07 <matt_flaschen> ori, he's talking about a shim in Echo
9722:20:07 <TimStarling> "Being in an extension, Echo has to use ugly hooks to suppress some core features"
9822:20:15 <matt_flaschen> So if you're already depending on Echo, just keep doing it until you feel like changing your extension to drop the Echo dependency.
9922:20:19 <brion> (in wmf-maintained extensions folks love to remove old interfaces and clean them up. in third-party-maintained extensions folks tend to run back-compat. one of my concerns is making sure third-party extension authors have one fewer compat choice to make in the future)
10022:20:19 <Krinkle> Talkpage notifications are a good use case to figure where the boundary is. It's the kind of notification we expect to be delivered (non-optional) and is in core currently but aborted by Echo to use its own instead (still sent as e-mail but formatted by echo instead of core)
10122:20:20 <TimStarling> which implies that core should depend on Echo, not just other extensions
10222:20:54 <duploktm> ori: if it already has notifications, it already depends upon Echo...?
10322:20:57 <brion> *nod* core sends notifications too, not just other extensions
10422:21:16 <matt_flaschen> +1 to Krinkle and brion. This is the main concrete benefit I see.
10522:21:34 <DanielK_WMDE_> if i understand correctly ,the idea is to have all the actual code in core, and just a shim in the extension, for compat
10622:21:44 <DanielK_WMDE_> that way, core wouldn't depend on the extension
10722:21:45 <Krinkle> I think notifications could have an indication (if not already) whether delivery is important. People don't use the wiki always. So I disagree Echo is a meant of direct delivery. I"m sure many people never look at it.
10822:21:52 <brion> now we _could_ have an abstract interface with a simple concrete implementation. i just feel like the echo code maybe is a better way to do that than inventing a new default implementation
10922:21:57 <Krinkle> And if so, then the default implementation in core is to send an e-mail like it does now
11022:22:07 <Krinkle> It would allow Echo to replace it much more cleanly
11122:22:08 <TimStarling> DanielK_WMDE_: yes, I was just in ori's universe for a minute
11222:22:16 <DanielK_WMDE_> heh
11322:22:22 <matt_flaschen> Krinkle, how, though? Would core have a simple $wgNotificationHandler?
11422:22:23 <Krinkle> I think merging all of Echo is overestimated. It's not going to solve as many problems as we might think.
11522:22:28 <matt_flaschen> Note Echo also has email notifications.
11622:22:33 <matt_flaschen> And they can be enabled by default.
11722:22:36 <Krinkle> matt_flaschen: Yes, and it effectively does already
11822:22:55 <matt_flaschen> Krinkle, yes, not in a deliberately pluggable way. But yeah, it's a proposal to consider.
11922:22:57 <Krinkle> matt_flaschen: Exactly. It replaces that functionality with a nicer preference and pretty e-mail.
12022:23:08 <TimStarling> email is presumably not going to provide as many features as echo
12122:23:12 <RoanKattouw> Krinkle: So you're proposing only the email part of Echo be merged into core, and the web notification stuff be kept in the extension?
12222:23:18 <TimStarling> for example merging of duplicate messages
12322:23:55 <RoanKattouw> Yeah there are some features unique to rendering notifications on the web
12422:24:12 <Krinkle> Non-important notifications that are web-only by default would go nowhere without Echo installed.
12522:24:21 <TimStarling> so if email is the default backend and an API user is expecting message merging then you will end up making non-optimal design choices
12622:24:21 <jdlrobson> matt_flaschen: maybe the confusion is around the use of the word "echo". I am advocating for a notifications engine in core, but not the entirety of the Echo extension. So far Krinkle is not saying anything I disagree with.. :)
12722:24:21 <RoanKattouw> Such as what TimStarling said ("merging of duplicate messages", which we call "bundling"), and also i18n-y things like dealing with different languages and bidi embedding etc
12822:24:31 <TimStarling> yeah that
12922:24:32 <DanielK_WMDE_> FWIW, i think it would be very good for core to have the notion (that is, an interface) of a notification service that can be used to send notifications to users. The interface can, for now, be implemented directly by echo, no need for a fancy registry mechanism just yet. Would be nice if we could fit email notifications in there too, though.
13022:24:37 <jdlrobson> apologies if my terminology is confusing
13122:24:38 <duploktm> jdlrobson: what does "notifications engine" mean?
13222:25:09 <gwicke> DanielK_WMDE_: +1
13322:25:54 <matt_flaschen> DanielK_WMDE_, how can we handle existing core functoinality like core talk page email notifications without a registry system or unconditionally replacing that functionality with Echo?
13422:26:13 <bd808> Is anyone proposing an alternative implementation of a "notifications engine" or are we just hearing that code in core really shouldn't be shitty?
13522:26:16 <Krinkle> When installed, Echo would take over all e-mail notifications. That seems reasonable.
13622:26:20 <matt_flaschen> Krinkle has proposed (IIUC) a registry system with a simple implementation in core, and then Echo replacing that core implementation.
13722:26:25 <DanielK_WMDE_> matt_flaschen: fittign in email notification would have to wait until we do have some kind of registrytion mechansim
13822:26:28 <ori> the expected value of this work doesn't seem commensurate with the effort involved
13922:26:36 <matt_flaschen> DanielK_WMDE_ then to me we're not getting any benefit to the RFC.
14022:26:49 <ori> but if legoktm can / wants to do it without breaking things in the process then so be it
14122:26:59 <Krinkle> matt_flaschen: If you call "refactor existing core code" a simple implementation, then es.
14222:26:59 <Krinkle> yes*
14322:26:59 <DanielK_WMDE_> matt_flaschen: not getting benefit from what?
14422:27:13 <RoanKattouw> "simple" relative to echo
14522:27:18 <duploktm> Krinkle: maintaining duplicate implementations of similar functionality, except one of which is never used on the Wikimedia cluster is one of the things I'd like to move away from
14622:27:25 <brion> ori: are you referring to the 'split everything into interfaces and implementations more than it already is' or the 'merge to core' scenarios?
14722:27:31 <matt_flaschen> DanielK_WMDE_ if we have a core notification service but the existing email notifs (already in core) don't play by the rules of that service, what's the point?
14822:27:31 <Krinkle> duploktm: No duplicate functionality.
14922:27:43 <matt_flaschen> Krinkle, simple compared to effort, not simple refactoring work.
15022:27:48 <matt_flaschen> simple compared to Echo
15122:27:50 <matt_flaschen> Sorry
15222:28:03 <duploktm> Krinkle: so core would have no email notifs then? What is Echo taking over?
15322:28:14 <ori> brion: Both. These sorts of projects don't tend to go well unless they are tied to a feature goal
15422:28:22 <DanielK_WMDE_> matt_flaschen: my idea is to fit in Echo, using an interface that can later also be used by ENotif. Once that is done, we can refactor thigns and add a registry, and hook in ENotif too. The interface should be designed with that in mind
15522:28:44 <brion> ori: feature goal is to ensure that server->user notifications are consistently available as a service to other features in both core and extensions.
15622:28:53 <DanielK_WMDE_> I believe the main point of contention is how core talks to the notification engine or whatever. What does that interface look like?
15722:29:12 <Krinkle> duploktm: The interface for creating notifications would move, not be duplicated.
15822:29:22 <jdlrobson> duploktm my definition would be something that allows registering notifications for users and is either agnostic to how they are delivered or uses email to deliver them by default
15922:29:41 <duploktm> ori: this is mostly a tech-debt cleanup RfC
16022:29:41 <Krinkle> duploktm: And used by core instead of what it does now for email notifications.
16122:29:41 <matt_flaschen> DanielK_WMDE_, also in contention whether to move UI to core.
16222:29:41 <duploktm> Krinkle: so we'd still end up with two implementations of email notifs right?
16322:29:41 <brion> ori: so while other extensions can declare dependency on echo without that merge, features in core cannot do that reliably today
16422:29:53 <jdlrobson> ori: you do have a point on that - but its my understanding the collaboration team have goals around Echo no (judging on latest cross wiki notifications) so I'd hope it would be tied to those?
16522:29:53 <RoanKattouw> If you want a feature goal around this, consistent emails is one
16622:29:56 <duploktm> jdlrobson: ...that's what Echo currently is.
16722:29:58 <RoanKattouw> enotif emails and Echo emails look different right now
16822:30:02 <Krinkle> duploktm: There will likely be a class to be extended by core, but I doubt it would actually duplicate anything.
16922:30:06 <DanielK_WMDE_> matt_flaschen: ...and the storage backend. but my impression was that that wasn't so controversial.
17022:30:31 <gwicke> it's not like our feature-driven projects have a stellar track record
17122:30:50 <matt_flaschen> DanielK_WMDE_, I think Krinkle withdrew the idea of making that separately pluggable, so he's now suggesting just one pluggable (notification system) with an implementation in core that Echo would replace if enabled.
17222:31:32 <gwicke> I agree that the benefits should clearly outweigh the costs, but don't think that features are the only way to get focus and good cost / benefit ratios
17322:31:33 <ori> brion: yeah, I think it's valid, just not commensurate with the risks
17422:31:48 <brion> ori: fair enough, I just come down on other side of that :)
17522:31:56 <DanielK_WMDE_> matt_flaschen: sounds good, except that i would be fine with echo being that implementation in core. Rewriting it doesn't sound too appealing.
17622:32:17 <matt_flaschen> Yes, it's still open whether that's worth doing.
17722:32:22 <matt_flaschen> Maybe an alterate proposal would be:
17822:32:39 <matt_flaschen> 1. Bundle Echo in tarball for a release, as I think Krinkle suggested.
17922:33:04 <matt_flaschen> 2. Merge Echo (either in entirety, or just the logic part) into core.
18022:33:08 <jdlrobson> duploktm: well there's a lot of frontend code too that doesn't seem to fit into that definition. I don't think formatters fit into that definition.
18122:33:23 <matt_flaschen> The web UI part could potentially stay separate even if there was no other notification system in core, but that still adds complexity.
18222:33:29 <DanielK_WMDE_> matt_flaschen: we could also have a dummy implementation be the default.
18322:33:48 <ori> (2) seems like a good idea. It would be very ugly at first, but then the things about Echo that are extension-y could be replaced gradually
18422:33:58 <ori> so there isn't a big, risky mega-commit
18522:34:14 <duploktm> Yes, I'd like to do this gradually, not one giant thing
18622:34:38 <Krinkle> The main things we advertise as gains are the lack of good extension points in core and duplicated code right now between core and echo
18722:34:39 <duploktm> There's a lot of small areas that can be worked on like moving the "Notifications" preferences tab into core for example
18822:34:43 <matt_flaschen> Yeah, the RFC actually suggests to do it in entirety, but piece by piece.
18922:34:46 <Krinkle> We can (and will either way) fix that.
19022:35:01 <Krinkle> But I don't think there's any positive value gained out of merging Echo itself into core.
19122:35:20 <matt_flaschen> Krinkle, not having to maintain another notification implementation (the simpler email system).
19222:35:30 <RoanKattouw> When you say "Echo itself", what do you mean?
19322:35:39 <brion> i find the 'itself' to be the sticking point
19422:35:39 <RoanKattouw> Evidently there are some things you think should move in core and some things you think shouldn't
19522:35:39 <duploktm> jdlrobson: if it has to send emails, it needs to be able to format. And through experience, the formatting and backend code is tied together, you can't change one without changing the other
19622:35:53 <Krinkle> matt_flaschen: That's not true. It wouldn't be a separate system.
19722:36:29 <matt_flaschen> Krinkle, I'm talking about the existing enotifs. Aren't you proposing core would provide a wgNotificationHandler to do that, and Echo would have its own if enabled?
19822:36:48 <duploktm> Quick question: How many people have actually read through the current Echo interfaces and how they are used to create and format notifications?
19922:36:51 <Krinkle> matt_flaschen: Yeah, but why would Echo have to duplicate the core one? Extensions can extend.
20022:36:58 <RoanKattouw> Also, let me point out that emails also means email digests. The logic around that is "fun"
20122:37:08 <duploktm> I think this might be easier if I outline what the interfaces would look like in core.
20222:37:20 <brion> ++ outline
20322:37:30 <RoanKattouw> I know the existing interfaces but +1 for an outline
20422:37:36 <duploktm> The RfC focuses mainly on the icky parts of Echo, not the actual good things it does
20522:37:39 <RoanKattouw> Because I always like concrete proposals better
20622:37:48 <matt_flaschen> Okay, that's fair. It wouldn't have to be a totally separate implementation but it's still a valid question if we need "base email" and "Echo email".
20722:38:19 <TimStarling> so I guess discussion of this will continue on phabricator
20822:38:25 <Krinkle> matt_flaschen: The Email notification class woudl likely be extended by Echo and overload a method or two.
20922:38:26 <gwicke> duploktm: +1 for defining those interfaces
21022:39:08 <TimStarling> could the main people expressing opinions here write up a position statement as a phabricator comment?
21122:39:35 <brion> i believe that's TimStarling-speak for "tactfully, sirs, we are out of time for this subject and must move on"
21222:39:52 <TimStarling> brion, RoanKattouw, matt_flaschen
21322:40:02 * brion goes to phab
21422:40:03 <matt_flaschen> Yep, will do
21522:40:07 <TimStarling> Krinkle has already written some stuff so can probably reply to the things other people write
21622:40:27 <TimStarling> yeah I'm aiming for topic switch between 40 and 45 past
21722:40:31 <gwicke> good move to directly summarize on phabricator
21822:41:21 <TimStarling> are there any action items for duploktm, e.g. expanding the RFC document?
21922:41:40 <TimStarling> I guess most people didn't read it so maybe it's a waste of time ;)
22022:42:29 <duploktm> #action legoktm to outline/define what the core interfaces would look like, and more specificity on what the different parts of Echo actually do
22122:43:47 <TimStarling> #topic Content hash based thumb URLs | RFC meeting | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ (Meeting topic: RFC meeting)
22222:43:55 <TimStarling> #link https://phabricator.wikimedia.org/T66214
22322:44:23 <TimStarling> I think gwicke has some urgent questions that need to be addressed, then we will probably have a followup meeting next week
22422:44:36 <brion> sounds about right :)
22522:44:45 <gwicke> okay, let me give some background on this one
22622:45:14 <gwicke> this task started out discussing content hashes instead of names, but morphed into a general discussion about a thumb API
22722:45:46 <gwicke> https://phabricator.wikimedia.org/T66214#2087098 describes how those two relate
22822:46:46 <paravoid> (are you done gwicke?)
22922:46:55 <gwicke> the central problem of the api portion is letting clients select the size and possibly quality of thumb images, so that we can stop wasting time, cpu cycles and documentation complexity on API end points that return different-sized thumb references
23022:46:55 <brion> the one downside of treating the opaque-id (has etc) and the stable-predictable-URL-based-thumbnailing-API questions separately is that it raises back the question of cache stability fo rthe URLs
23122:47:07 <paravoid> can I make a comment on process for this?
23222:47:29 <gwicke> paravoid: permission granted
23322:47:35 <paravoid> I'll preface this with a statement that I'm not particularly opposed to this idea
23422:47:44 <paravoid> and it does make sense to me to some extent
23522:47:57 <paravoid> but, it seems that we have two issues conflated into one RfC, which has created some confusion
23622:48:29 <paravoid> and moreover, I think it's essential when designing an API to talk about its envisioned use cases, and I don't see much there
23722:48:31 <gwicke> paravoid: I agree, and I did consider actually splitting the task
23822:48:33 <paravoid> on that task, I mean
23922:48:56 <paravoid> I see e.g. Android and iOS apps tags on the task, but not actually a description of their needs
24022:49:18 <paravoid> and finally, it's still completely unclear how/where this would be implemented (and by whom)
24122:49:49 <paravoid> that's all
24222:50:32 <gwicke> yeah, a summary of use cases would be nice; there is a lot of detail in the discussion, but no short summary of the most important points
24322:50:47 <paravoid> nod
24422:50:47 <Scott_WUaS> Paravoid: thanks
24522:50:55 <brion> good notes :) on the apps, main thing they need is to be able to take a given image and fetch it at a given size reliably, with a minimum number of network round-trips
24622:51:15 <gilles> how is that not currently possible?
24722:51:19 <brion> it's hard to rely on the raw thumbnail URLs currently as asking for too large an image may fail etc, or you may have a funny format that's ahrd to predict
24822:51:22 <paravoid> yeah, I've heard this before but it's not very clear from the task -- and I'm not sure what /else/ would they need for example
24922:51:29 <brion> eg you hvae to know to map .svg to .svg.png
25022:51:32 <DanielK_WMDE_> how does purging work for different sized thumbnails?
25122:51:39 <gwicke> we are discussing the right size to serve on tasks like https://phabricator.wikimedia.org/T125983, but ultimately none will work for every consumer
25222:51:46 <DanielK_WMDE_> is the current mechanism sufficient?
25322:52:13 <gwicke> I am not even sure whether old images can contain slashes in their name
25422:52:16 <DanielK_WMDE_> brion: yea, ran into that back in the toolserver days :)
25522:52:18 <brion> there are also secondary concerns like wanting to be able to ask for sizes differently (mobile web/apps frequently want cropped-to-square thumbnails which is different from the max-bounding-box)
25622:52:30 <brion> where just having a clean way to extend the URL-based api would be nice
25722:52:30 <paravoid> as an example: I had previously heard a need for thumbnails that would crop/center on faces
25822:52:42 <gwicke> client-side quality selection based on bandwidth, device specs etc is a common need
25922:52:49 <TimStarling> pretty sure image names can't contain slashes
26022:53:05 <brion> yeah image names cannot contain slashes :)
26122:53:21 <gilles> yes, because the whole image path is suppose to correspond to the actual file path on basic mediawiki instances
26222:53:22 <brion> let's make sure we capture these notes on the task!
26322:53:22 <gwicke> definitely not new ones, I'm less sure about historical ones
26422:54:07 <gwicke> the filerepo code seems to support slashes
26522:54:51 <brion> filerepo needs to be able to handle relative paths in some places, but identifiers for files *should* not allow /s. if they do.... that may be a bug ;)
26622:54:57 <gwicke> in any case, rewriting thumb URLs is not something that was designed as a sane API, and is probably not something we should promote to one
26722:55:07 <paravoid> slashes in filenames sounds like actively looking for trouble
26822:55:27 <Vulpix> https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads#Known_problems_with_database_names_having_non-alphanumeric_characters :)
26922:55:42 <brion> ok we're running short on time, anything else to add?
27022:56:10 <gwicke> so the task proposes a simple strawman, and I'd like some input on that
27122:56:43 <gilles> if it's going to include all the needs we've just mentioned (destination format, cropping) it's looking more and more like the IIIF feature set
27222:56:53 <gilles> maybe a bit more than that, but it's a format we can participate in
27322:57:06 <gwicke> a prototype of this would be fairly easy to implement; a production version should not duplicate cache entries, so the translation between old & new style URLs should ideally happen in Varnish
27422:57:21 <gilles> thumbor also has its own url format that covers most, if not all of that. but that's not a standard. can be used as inspiration, though
27522:57:55 <brion> yeah IIIF is something i'm looking into actually as well :)
27622:58:00 <gwicke> brion realized that most of those advanced features are normally server-supplied
27722:58:03 <brion> so i'll read up on that and add more notes
27822:58:12 <gilles> doing the translation in varnish is how you're going to give ops nightmares. what's the VCL going to look like when it's version 6, 7 of the API and all the old URLs still have to redirect?
27922:58:22 <Scott_WUaS> great meeting and developing process, all! thank you :)
28022:58:33 <bd808> paravoid: images cropped around the focal point of the image is a use case for "hero images" such as the native apps and hovercards use. In the apps they do it client side using native libs but on the web we need a backend (thumbor) to do it.
28122:58:35 <tgr> two things that weren't clear in the task
28222:58:46 <tgr> is this going to happen in MediaWiki core?
28322:58:53 <TimStarling> tgr: maybe next week or on the task?
28422:58:53 <paravoid> nod, I'm not very fond of adding this kind of complexity at the edge
28522:58:55 <TimStarling> it's 58 past now
28622:59:00 <tgr> ok
28722:59:20 <gwicke> paravoid: it's the only way to use edge-cached content, afaik
28822:59:24 <paravoid> depends on the amount of the complexity of course
28922:59:34 <TimStarling> we'll presumably have a second meeting on this next week
29022:59:41 <paravoid> sort of -- you could go through a transition with the old URLs redirecting in theory
29122:59:41 <robla> next week: https://phabricator.wikimedia.org/E66/26
29223:00:05 <paravoid> that's something that we'll need to do at some point if we decide to do content hashing URLs, right?
29323:00:21 <gwicke> yes, there will need to be a translation service & redirects
29423:00:36 <gilles> don't design this API with only existing use cases in mind and assume it's rarely going to get updated. that's exactly how we got our ugly thumbnail urls
29523:00:40 <paravoid> I hope noone suggests that we do old URL -> content hash URL mappings in varnish :)
29623:00:54 <gwicke> gilles: thumb urls weren't even designed as an api
29723:00:54 <Scott_WUaS> gwicke: translation service?
29823:00:54 <gwicke> they were just a way to somehow internally reference images
29923:01:08 <TimStarling> ok to end the meeting now?
30023:01:20 <paravoid> not even that afaik -- they were just the directory structure on the web server that served them
30123:01:20 <gwicke> in any case, I would appreciate your input on the task
30223:01:27 <paravoid> will do
30323:01:36 <gilles> what you propose isn't an API, it's a URL format. let's call it what it is. semantically a REST API for images should support PUTs, DELETEs, etc.
30423:01:49 <gilles> if it's only GETs it's another URL scheme that's all
30523:02:19 <TimStarling> #endmeeting

Other meetings

See the Architecture meetings page for more general information about this meeting (also: Phab query: list of upcoming RFC meetings, Phab query: list of all RFC meetings).

Recurring Event

Event Series
This event is an instance of E66: ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office), and repeats every week.

Event Timeline

RobLa-WMF renamed this event from RFC Meeting: <topic TBD> (<see "Starts" field>, #wikimedia-office) to RFC Meeting: Review backlog, Thumb API (2016-03-16, #wikimedia-office) (NOTE: U.S. Daylight Savings shift).Mar 10 2016, 1:02 AM
RobLa-WMF updated the event description. (Show Details)
daniel updated the event description. (Show Details)Mar 16 2016, 4:20 PM

this meeting is happening now in #wikimedia-office. Links:

3:00 PM <wm-labs-meetbot`> Minutes: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-03-16-21.00.html
3:00 PM <wm-labs-meetbot`> Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-03-16-21.00.txt
3:00 PM <wm-labs-meetbot`> Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-03-16-21.00.wiki
3:00 PM <wm-labs-meetbot`> Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2016/wikimedia-office.2016-03-16-21.00.log.html

Meeting summary

  • wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ | https://phabricator.wikimedia.org/E148 (robla, 21:00:53)
    • LINK: https://phabricator.wikimedia.org/T66214 (gwicke, 21:01:56)
    • topic discussed: where we add extra parameters (page, time, language, etc) is the format sufficiently clearly specified? (robla, 21:04:13)
    • to discuss in future: implementation details for small/large installs (backlog) (brion, 21:05:47)
    • avoiding cache fragmentation will require normalization of parameters (deterministic serialization of hashmap or other ordering) (brion, 21:19:50)
    • daniel suggests having the failed-validation case be a redirect, gwicke warns they're expensive (brion, 21:20:24)
    • normalizing parameters within varnish would be painful (brion, 21:20:43)
  • T78639: How to address the long tail of low priority tasks in active projects (robla, 21:21:36)

Meeting ended at 22:00:02 UTC.

People present (lines said)

  • gwicke (53)
  • DanielK_WMDE__ (47)
  • brion (45)
  • robla (28)
  • bd808 (19)
  • andre__ (17)
  • greg-g (16)
  • legoktm (10)
  • SMalyshev (7)
  • stashbot (6)
  • Purodha (4)
  • Scott_WUaS (4)
  • wm-labs-meetbot` (3)
  • jzerebecki (2)
  • ebernhardson (2)
  • chasemp (1)
Nikerabbit added a subscriber: Nikerabbit.EditedMar 17 2016, 10:58 AM

Speaking of timezones, May I ask why did we stop rotating the time of this event? Currently I am not able to attend.

Even the announcement of this event came so late that I had already finished for the day and was not able to comment beforehand.

RobLa-WMF updated the event description. (Show Details)Jul 28 2016, 12:25 AM
daniel renamed this event from RFC Meeting: Review backlog, Thumb API (2016-03-16, #wikimedia-office) (NOTE: U.S. Daylight Savings shift) to ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office).Nov 21 2016, 6:11 PM
daniel changed the host of this event from RobLa-WMF to daniel.
daniel invited: ; uninvited: .
daniel updated the event description. (Show Details)
daniel updated the event description. (Show Details)Dec 9 2016, 7:44 AM
daniel renamed this event from ArchCom RFC Meeting Wxx: <topic TBD> (<see "Starts" field>, #wikimedia-office) to RFC Meeting: Review backlog, Thumb API (2016-03-16, #wikimedia-office) (NOTE: U.S. Daylight Savings shift).