Page MenuHomePhabricator
Paste P3640

ArchCom-RFC-2016W10-irc-E147.txt
ActivePublic

Authored by RobLa-WMF on Aug 4 2016, 12:32 AM.
Tags
None
Referenced Files
F4334949: ArchCom-RFC-2016W10-irc-E147.txt
Aug 4 2016, 12:32 AM
Subscribers
None
22:00:22 <TimStarling> #startmeeting RFC meeting
22:00:22 <wm-labs-meetbot`> Meeting started Wed Mar 9 22:00:22 2016 UTC and is due to finish in 60 minutes. The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:22 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:22 <wm-labs-meetbot`> The meeting name has been set to 'rfc_meeting'
22:00:36 <TimStarling> #topic RFC meeting | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/
22:01:12 <TimStarling> so I guess we're doing notifications first
22:01:31 <brion> yup
22:01:33 <legoktm> hi
22:01:34 <robla> T128351
22:01:34 <stashbot> T128351: RfC: Notifications in core - https://phabricator.wikimedia.org/T128351
22:01:36 <brion> first in first out
22:01:39 <gwicke> +1
22:01:54 <TimStarling> and thumb URLs will be the end of this week plus maybe next week as well?
22:02:32 <Scott_WUaS> great!
22:02:57 <gwicke> TimStarling: yes, depending on where we end up today
22:02:57 <legoktm> I updated https://www.mediawiki.org/wiki/Requests_for_comment/Notifications_in_core recently with open questions and implementation plan sections
22:04:01 <legoktm> I thought consenus was mostly leaning to sticking most of Echo into core (with some specific open questions), but Krinkle posted something different on the bug just now
22:04:16 <matt_flaschen> https://phabricator.wikimedia.org/T128351#2104701
22:05:31 <matt_flaschen> Krinkle basically suggested keeping all the DB code and UI outside core, and making it pluggable.
22:05:40 <robla> legoktm: there are five open questions there, should we step through them?
22:06:10 <TimStarling> making it pluggable would be more work
22:06:43 <matt_flaschen> Yes, my first question is whether anyone even wants to make an alterate implementation.
22:06:58 <legoktm> robla: my questions assumed people were in agreement that we should be moving most of the Echo backend stuff into core, but if people agree with Krinkle, those questions don't make sense
22:07:01 <gwicke> the main thing you need is to define a narrow interface
22:07:22 <gwicke> which is useful in its own right
22:07:25 <Krinkle> It could be as sample as taking Echo's base classes, converting to an interface and adding to core. And in the process we'd also force separation between backend and logic handling (similar to what we're working on within mw for some areas)
22:07:29 <legoktm> so we should first figure out whether we want to have some kind of pluggable thing Krinkle proposed, or just have Echo as the core notification implementation
22:07:39 <ashley> +2 to merging Echo into core (finally, if I may add)
22:07:49 <Krinkle> the latter is a separate thing, but I think people do agree on should happen regardless.
22:08:01 <TimStarling> and the rationale is to do a more gradual rollout to non-WMF users?
22:08:39 <jdlrobson> +1 Krinkle that was my thoughts on this
22:08:49 <brion> well if we have a plain API for it, we either need a core implementation (like Search and AuthPlugin) or to consider mediawiki as not requiring it
22:08:50 <Krinkle> Yeah. It's been very little exposed to third parties. I think we should either halt the merge and do a bundle for a first release cycle first. Or only merge the interfaces and then bundle and stick with just that.
22:08:51 <jdlrobson> the interface is needed by skins, but they don't need to care about how it's implemented
22:09:05 <brion> what would be the behavior if an extension fires off notifications on a wiki with no notification backend?
22:09:19 <gwicke> a no-op default has been proposed
22:09:26 <bd808> brion: >/dev/null ?
22:09:32 <brion> that sounds bad to me
22:09:37 <matt_flaschen> jdlrobson, how is interface needed by skins? My understanding is people are talking about API interface.
22:09:40 <Krinkle> brion: Yeah, just like core's RC notificaitons go no where by default.
22:09:45 <TimStarling> jdlrobson: are you agreeing with Krinkle that echo should not be brought into core?
22:09:45 <Krinkle> not to redis or UDP
22:09:50 <gwicke> I think it is useful for this discussion to separate a) interface, and b) default / alternate implementations
22:10:07 <gwicke> a) is fairly orthogonal to b)
22:10:07 <matt_flaschen> See also my question at https://phabricator.wikimedia.org/T128351#2104841 . How do we handle places where Echo replaces core functoinality (e.g. email notif for user talk)?
22:10:10 <Krinkle> But the interface for creating RC entries and having an RCFeedEngine are in core.
22:10:11 <brion> Krinkle: RC events aren't user-facing for most values of user, whereas this is entirely user-focused
22:10:21 <Krinkle> brion: True.
22:10:36 <jdlrobson> TimStarling: i'm still collecting my thoughts but my immediate reaction when I saw the RFC was interfaces should be defined in core, not the entirety of the Echo system.
22:10:49 <duploktm> sorry, my other client is lagging
22:10:53 <Krinkle> brion: But I don't think we're too worried about exposing Echo as an API inside HTML. So that'll work the same way either way. Point being, what do we gain by having Echo in core?
22:10:56 <brion> i can't envision many (any?) circumstances where as a core or extension author i would expect firing off a notification to a user to do nothing to be acceptable
22:11:23 <brion> we gain a consistent way for core and extension authors to provide software->user communications path
22:11:33 <brion> the consistent part comes from it always working, not just having an internal api :)
22:11:34 <jdlrobson> Echo seems like an implementation of a NotificationsEngine and I could imagine other notifications engines involving email and even the orange bar of death ;-)
22:11:51 <Krinkle> brion: Right, so you're comparing this similar to EmailUser. It can't go nowhere by default since the application intent is to reach a user, not just to provide information in general.
22:12:00 <matt_flaschen> Krinkle, main reason i think is to avoid duplicate implementations (e.g. email for user talk notif) with hooks galore, second to that is that it's an important framework, and thus arguably important enough to go into core.
22:12:01 <brion> *nod*
22:12:04 <duploktm> Echo has a lot of things inside of it, and I think those things are being confused.
22:12:13 <jdlrobson> matt_flaschen: Right now Echo adds itself to menus via horrible hooks and I'd prefer to see it replace some concept in core. e.g. NotificationsEngine = 'Echo'
22:12:22 <matt_flaschen> jdlrobson, in theory you can make anything pluggable, but in practice does anyone actually plan to write that? It's a lot of code.
22:12:36 <matt_flaschen> jdlrobson, and what happens if there is no engine and no Echo. Do you just not get email notif for user talk change?
22:12:58 <duploktm> Echo is a notifications framework, which includes a pluggable backend for how to deliver notifications (web, email, etc.), a system for formatting those notifications for each output type
22:13:30 <gwicke> matt_flaschen: defining interfaces does not necessarily imply doing the legwork to set up fancy plugging infrastructure behind those interfaces
22:13:50 <TimStarling> duploktm: that is an excellent nickname
22:13:54 <duploktm> The main problem I see is that there are already notification types in core (enotifs, watchlists) which sometimes conflict with Echo, and require bad hook hacks to fight with
22:14:03 <matt_flaschen> gwicke, I'm responding to Krinkle and jdlrobson (both of whom did propose making it pluggable)
22:14:05 <brion> so it sounds like echo *is* the thing that defines pluggable backend interfaces
22:14:37 <matt_flaschen> brion, note Krinkle even proposed making the DB backend pluggable. Echo does not support that and there has been no prior discussion of that.
22:15:04 <TimStarling> brion: which brings into question the idea of using echo base classes as the core notification API
22:15:09 <ori> what is the point of moving it into core, or define an interface in core?
22:15:11 <duploktm> I'm all for interfaces and separating business and frontend logic and whatnot, but I don't think that separation needs to be enforced by extension vs. core and physical git repos
22:15:24 <Krinkle> matt_flaschen: You can scrap that detail. I'm not pushing for that. That could be injected to the handler instead of being known to core directly.
22:15:24 <ori> is there a clear articulation of the value somewhere?
22:15:29 <TimStarling> echo would become a collection of backends instead of a single thing
22:15:38 <matt_flaschen> Okay, fair enough.
22:15:48 <duploktm> ori: https://www.mediawiki.org/wiki/Requests_for_comment/Notifications_in_core#Problem
22:15:49 <brion> ori: to ensure it is always available so can be used and relied upon by core and extension authors making features for users
22:15:49 <Krinkle> And I agree there isn't much value in anticipating a backend and handling logic from different authors at this point.
22:15:51 <brion> least that's my goal ;)
22:15:56 <jdlrobson> Note that MediaWiki doesn't ship with a default skin, you have to install a skin. I'd expect a similar kind of abstraction with Echo.
22:16:03 <matt_flaschen> ori, I think main concrete benefit is that we don't have to have to versions of the same thing (e.g. user talk notification). Later on, we could also look at unifying watchlist and Echo if it makes sense.
22:16:15 <matt_flaschen> The idea "important infrastructure should be in core" might also apply, but isn't concrete.
22:16:24 <duploktm> ori: Another thing people noted on the phab task is that Echo and notifications in general have become an integral part of using MediaWiki - at least on Wikimedia wikis where it is installed
22:17:04 <ori> Suppose we move it into core. The extensions that want to make use of the new Echo functionality in Core have to either:
22:17:29 <ori> (a) drop the old code that works with the Echo extension, and express a dependency on the version of MediaWiki that introduced this work
22:17:57 <ori> (b) keep the old code, and add _additional_ class_exists & co checks
22:18:03 <brion> a) is how we usually roll
22:18:26 <matt_flaschen> jdlrobson, again, why? At the time the decision was made to stop shipping skins, there were already many skins (in core and elsewhere). There is only one notification framework I know of.
22:18:26 <ori> so why not just express a dependency on Echo?
22:18:29 <TimStarling> yup (a)
22:18:43 <duploktm> ori: sorry, but did you read the RfC? I proposed keeping a back-compat shim with old class names so neither a nor b would be necessary
22:19:11 <TimStarling> ori: because of the reasons on the wiki page that duploktm just linked to
22:19:47 <ori> you mentioned "a back-compat layer for things that haven't transitioned to core notifs yet " as something that would stay in Echo
22:19:54 <ori> not that there would be a shim in core
22:20:07 <matt_flaschen> ori, he's talking about a shim in Echo
22:20:07 <TimStarling> "Being in an extension, Echo has to use ugly hooks to suppress some core features"
22:20:15 <matt_flaschen> So if you're already depending on Echo, just keep doing it until you feel like changing your extension to drop the Echo dependency.
22:20:19 <brion> (in wmf-maintained extensions folks love to remove old interfaces and clean them up. in third-party-maintained extensions folks tend to run back-compat. one of my concerns is making sure third-party extension authors have one fewer compat choice to make in the future)
22:20:19 <Krinkle> Talkpage notifications are a good use case to figure where the boundary is. It's the kind of notification we expect to be delivered (non-optional) and is in core currently but aborted by Echo to use its own instead (still sent as e-mail but formatted by echo instead of core)
22:20:20 <TimStarling> which implies that core should depend on Echo, not just other extensions
22:20:54 <duploktm> ori: if it already has notifications, it already depends upon Echo...?
22:20:57 <brion> *nod* core sends notifications too, not just other extensions
22:21:16 <matt_flaschen> +1 to Krinkle and brion. This is the main concrete benefit I see.
22:21:34 <DanielK_WMDE_> if i understand correctly ,the idea is to have all the actual code in core, and just a shim in the extension, for compat
22:21:44 <DanielK_WMDE_> that way, core wouldn't depend on the extension
22:21:45 <Krinkle> I think notifications could have an indication (if not already) whether delivery is important. People don't use the wiki always. So I disagree Echo is a meant of direct delivery. I"m sure many people never look at it.
22:21:52 <brion> now we _could_ have an abstract interface with a simple concrete implementation. i just feel like the echo code maybe is a better way to do that than inventing a new default implementation
22:21:57 <Krinkle> And if so, then the default implementation in core is to send an e-mail like it does now
22:22:07 <Krinkle> It would allow Echo to replace it much more cleanly
22:22:08 <TimStarling> DanielK_WMDE_: yes, I was just in ori's universe for a minute
22:22:16 <DanielK_WMDE_> heh
22:22:22 <matt_flaschen> Krinkle, how, though? Would core have a simple $wgNotificationHandler?
22:22:23 <Krinkle> I think merging all of Echo is overestimated. It's not going to solve as many problems as we might think.
22:22:28 <matt_flaschen> Note Echo also has email notifications.
22:22:33 <matt_flaschen> And they can be enabled by default.
22:22:36 <Krinkle> matt_flaschen: Yes, and it effectively does already
22:22:55 <matt_flaschen> Krinkle, yes, not in a deliberately pluggable way. But yeah, it's a proposal to consider.
22:22:57 <Krinkle> matt_flaschen: Exactly. It replaces that functionality with a nicer preference and pretty e-mail.
22:23:08 <TimStarling> email is presumably not going to provide as many features as echo
22:23:12 <RoanKattouw> Krinkle: So you're proposing only the email part of Echo be merged into core, and the web notification stuff be kept in the extension?
22:23:18 <TimStarling> for example merging of duplicate messages
22:23:55 <RoanKattouw> Yeah there are some features unique to rendering notifications on the web
22:24:12 <Krinkle> Non-important notifications that are web-only by default would go nowhere without Echo installed.
22:24:21 <TimStarling> so if email is the default backend and an API user is expecting message merging then you will end up making non-optimal design choices
22:24:21 <jdlrobson> matt_flaschen: maybe the confusion is around the use of the word "echo". I am advocating for a notifications engine in core, but not the entirety of the Echo extension. So far Krinkle is not saying anything I disagree with.. :)
22:24:21 <RoanKattouw> Such as what TimStarling said ("merging of duplicate messages", which we call "bundling"), and also i18n-y things like dealing with different languages and bidi embedding etc
22:24:31 <TimStarling> yeah that
22:24:32 <DanielK_WMDE_> FWIW, i think it would be very good for core to have the notion (that is, an interface) of a notification service that can be used to send notifications to users. The interface can, for now, be implemented directly by echo, no need for a fancy registry mechanism just yet. Would be nice if we could fit email notifications in there too, though.
22:24:37 <jdlrobson> apologies if my terminology is confusing
22:24:38 <duploktm> jdlrobson: what does "notifications engine" mean?
22:25:09 <gwicke> DanielK_WMDE_: +1
22:25:54 <matt_flaschen> DanielK_WMDE_, how can we handle existing core functoinality like core talk page email notifications without a registry system or unconditionally replacing that functionality with Echo?
22:26:13 <bd808> Is anyone proposing an alternative implementation of a "notifications engine" or are we just hearing that code in core really shouldn't be shitty?
22:26:16 <Krinkle> When installed, Echo would take over all e-mail notifications. That seems reasonable.
22:26:20 <matt_flaschen> Krinkle has proposed (IIUC) a registry system with a simple implementation in core, and then Echo replacing that core implementation.
22:26:25 <DanielK_WMDE_> matt_flaschen: fittign in email notification would have to wait until we do have some kind of registrytion mechansim
22:26:28 <ori> the expected value of this work doesn't seem commensurate with the effort involved
22:26:36 <matt_flaschen> DanielK_WMDE_ then to me we're not getting any benefit to the RFC.
22:26:49 <ori> but if legoktm can / wants to do it without breaking things in the process then so be it
22:26:59 <Krinkle> matt_flaschen: If you call "refactor existing core code" a simple implementation, then es.
22:26:59 <Krinkle> yes*
22:26:59 <DanielK_WMDE_> matt_flaschen: not getting benefit from what?
22:27:13 <RoanKattouw> "simple" relative to echo
22:27:18 <duploktm> Krinkle: maintaining duplicate implementations of similar functionality, except one of which is never used on the Wikimedia cluster is one of the things I'd like to move away from
22:27:25 <brion> ori: are you referring to the 'split everything into interfaces and implementations more than it already is' or the 'merge to core' scenarios?
22:27:31 <matt_flaschen> DanielK_WMDE_ if we have a core notification service but the existing email notifs (already in core) don't play by the rules of that service, what's the point?
22:27:31 <Krinkle> duploktm: No duplicate functionality.
22:27:43 <matt_flaschen> Krinkle, simple compared to effort, not simple refactoring work.
22:27:48 <matt_flaschen> simple compared to Echo
22:27:50 <matt_flaschen> Sorry
22:28:03 <duploktm> Krinkle: so core would have no email notifs then? What is Echo taking over?
22:28:14 <ori> brion: Both. These sorts of projects don't tend to go well unless they are tied to a feature goal
22:28:22 <DanielK_WMDE_> matt_flaschen: my idea is to fit in Echo, using an interface that can later also be used by ENotif. Once that is done, we can refactor thigns and add a registry, and hook in ENotif too. The interface should be designed with that in mind
22:28:44 <brion> ori: feature goal is to ensure that server->user notifications are consistently available as a service to other features in both core and extensions.
22:28:53 <DanielK_WMDE_> I believe the main point of contention is how core talks to the notification engine or whatever. What does that interface look like?
22:29:12 <Krinkle> duploktm: The interface for creating notifications would move, not be duplicated.
22:29:22 <jdlrobson> duploktm my definition would be something that allows registering notifications for users and is either agnostic to how they are delivered or uses email to deliver them by default
22:29:41 <duploktm> ori: this is mostly a tech-debt cleanup RfC
22:29:41 <Krinkle> duploktm: And used by core instead of what it does now for email notifications.
22:29:41 <matt_flaschen> DanielK_WMDE_, also in contention whether to move UI to core.
22:29:41 <duploktm> Krinkle: so we'd still end up with two implementations of email notifs right?
22:29:41 <brion> ori: so while other extensions can declare dependency on echo without that merge, features in core cannot do that reliably today
22:29:53 <jdlrobson> ori: you do have a point on that - but its my understanding the collaboration team have goals around Echo no (judging on latest cross wiki notifications) so I'd hope it would be tied to those?
22:29:53 <RoanKattouw> If you want a feature goal around this, consistent emails is one
22:29:56 <duploktm> jdlrobson: ...that's what Echo currently is.
22:29:58 <RoanKattouw> enotif emails and Echo emails look different right now
22:30:02 <Krinkle> duploktm: There will likely be a class to be extended by core, but I doubt it would actually duplicate anything.
22:30:06 <DanielK_WMDE_> matt_flaschen: ...and the storage backend. but my impression was that that wasn't so controversial.
22:30:31 <gwicke> it's not like our feature-driven projects have a stellar track record
22:30:50 <matt_flaschen> DanielK_WMDE_, I think Krinkle withdrew the idea of making that separately pluggable, so he's now suggesting just one pluggable (notification system) with an implementation in core that Echo would replace if enabled.
22:31:32 <gwicke> I agree that the benefits should clearly outweigh the costs, but don't think that features are the only way to get focus and good cost / benefit ratios
22:31:33 <ori> brion: yeah, I think it's valid, just not commensurate with the risks
22:31:48 <brion> ori: fair enough, I just come down on other side of that :)
22:31:56 <DanielK_WMDE_> matt_flaschen: sounds good, except that i would be fine with echo being that implementation in core. Rewriting it doesn't sound too appealing.
22:32:17 <matt_flaschen> Yes, it's still open whether that's worth doing.
22:32:22 <matt_flaschen> Maybe an alterate proposal would be:
22:32:39 <matt_flaschen> 1. Bundle Echo in tarball for a release, as I think Krinkle suggested.
22:33:04 <matt_flaschen> 2. Merge Echo (either in entirety, or just the logic part) into core.
22:33:08 <jdlrobson> duploktm: well there's a lot of frontend code too that doesn't seem to fit into that definition. I don't think formatters fit into that definition.
22:33:23 <matt_flaschen> The web UI part could potentially stay separate even if there was no other notification system in core, but that still adds complexity.
22:33:29 <DanielK_WMDE_> matt_flaschen: we could also have a dummy implementation be the default.
22:33:48 <ori> (2) seems like a good idea. It would be very ugly at first, but then the things about Echo that are extension-y could be replaced gradually
22:33:58 <ori> so there isn't a big, risky mega-commit
22:34:14 <duploktm> Yes, I'd like to do this gradually, not one giant thing
22:34:38 <Krinkle> The main things we advertise as gains are the lack of good extension points in core and duplicated code right now between core and echo
22:34:39 <duploktm> There's a lot of small areas that can be worked on like moving the "Notifications" preferences tab into core for example
22:34:43 <matt_flaschen> Yeah, the RFC actually suggests to do it in entirety, but piece by piece.
22:34:46 <Krinkle> We can (and will either way) fix that.
22:35:01 <Krinkle> But I don't think there's any positive value gained out of merging Echo itself into core.
22:35:20 <matt_flaschen> Krinkle, not having to maintain another notification implementation (the simpler email system).
22:35:30 <RoanKattouw> When you say "Echo itself", what do you mean?
22:35:39 <brion> i find the 'itself' to be the sticking point
22:35:39 <RoanKattouw> Evidently there are some things you think should move in core and some things you think shouldn't
22:35:39 <duploktm> jdlrobson: if it has to send emails, it needs to be able to format. And through experience, the formatting and backend code is tied together, you can't change one without changing the other
22:35:53 <Krinkle> matt_flaschen: That's not true. It wouldn't be a separate system.
22:36:29 <matt_flaschen> Krinkle, I'm talking about the existing enotifs. Aren't you proposing core would provide a wgNotificationHandler to do that, and Echo would have its own if enabled?
22:36:48 <duploktm> Quick question: How many people have actually read through the current Echo interfaces and how they are used to create and format notifications?
22:36:51 <Krinkle> matt_flaschen: Yeah, but why would Echo have to duplicate the core one? Extensions can extend.
22:36:58 <RoanKattouw> Also, let me point out that emails also means email digests. The logic around that is "fun"
22:37:08 <duploktm> I think this might be easier if I outline what the interfaces would look like in core.
22:37:20 <brion> ++ outline
22:37:30 <RoanKattouw> I know the existing interfaces but +1 for an outline
22:37:36 <duploktm> The RfC focuses mainly on the icky parts of Echo, not the actual good things it does
22:37:39 <RoanKattouw> Because I always like concrete proposals better
22:37:48 <matt_flaschen> Okay, that's fair. It wouldn't have to be a totally separate implementation but it's still a valid question if we need "base email" and "Echo email".
22:38:19 <TimStarling> so I guess discussion of this will continue on phabricator
22:38:25 <Krinkle> matt_flaschen: The Email notification class woudl likely be extended by Echo and overload a method or two.
22:38:26 <gwicke> duploktm: +1 for defining those interfaces
22:39:08 <TimStarling> could the main people expressing opinions here write up a position statement as a phabricator comment?
22:39:35 <brion> i believe that's TimStarling-speak for "tactfully, sirs, we are out of time for this subject and must move on"
22:39:52 <TimStarling> brion, RoanKattouw, matt_flaschen
22:40:02 * brion goes to phab
22:40:03 <matt_flaschen> Yep, will do
22:40:07 <TimStarling> Krinkle has already written some stuff so can probably reply to the things other people write
22:40:27 <TimStarling> yeah I'm aiming for topic switch between 40 and 45 past
22:40:31 <gwicke> good move to directly summarize on phabricator
22:41:21 <TimStarling> are there any action items for duploktm, e.g. expanding the RFC document?
22:41:40 <TimStarling> I guess most people didn't read it so maybe it's a waste of time ;)
22:42:29 <duploktm> #action legoktm to outline/define what the core interfaces would look like, and more specificity on what the different parts of Echo actually do
22:43:47 <TimStarling> #topic Content hash based thumb URLs | RFC meeting | Wikimedia meetings channel | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/ (Meeting topic: RFC meeting)
22:43:55 <TimStarling> #link https://phabricator.wikimedia.org/T66214
22:44:23 <TimStarling> I think gwicke has some urgent questions that need to be addressed, then we will probably have a followup meeting next week
22:44:36 <brion> sounds about right :)
22:44:45 <gwicke> okay, let me give some background on this one
22:45:14 <gwicke> this task started out discussing content hashes instead of names, but morphed into a general discussion about a thumb API
22:45:46 <gwicke> https://phabricator.wikimedia.org/T66214#2087098 describes how those two relate
22:46:46 <paravoid> (are you done gwicke?)
22:46:55 <gwicke> the central problem of the api portion is letting clients select the size and possibly quality of thumb images, so that we can stop wasting time, cpu cycles and documentation complexity on API end points that return different-sized thumb references
22:46:55 <brion> the one downside of treating the opaque-id (has etc) and the stable-predictable-URL-based-thumbnailing-API questions separately is that it raises back the question of cache stability fo rthe URLs
22:47:07 <paravoid> can I make a comment on process for this?
22:47:29 <gwicke> paravoid: permission granted
22:47:35 <paravoid> I'll preface this with a statement that I'm not particularly opposed to this idea
22:47:44 <paravoid> and it does make sense to me to some extent
22:47:57 <paravoid> but, it seems that we have two issues conflated into one RfC, which has created some confusion
22:48:29 <paravoid> and moreover, I think it's essential when designing an API to talk about its envisioned use cases, and I don't see much there
22:48:31 <gwicke> paravoid: I agree, and I did consider actually splitting the task
22:48:33 <paravoid> on that task, I mean
22:48:56 <paravoid> I see e.g. Android and iOS apps tags on the task, but not actually a description of their needs
22:49:18 <paravoid> and finally, it's still completely unclear how/where this would be implemented (and by whom)
22:49:49 <paravoid> that's all
22:50:32 <gwicke> yeah, a summary of use cases would be nice; there is a lot of detail in the discussion, but no short summary of the most important points
22:50:47 <paravoid> nod
22:50:47 <Scott_WUaS> Paravoid: thanks
22:50:55 <brion> good notes :) on the apps, main thing they need is to be able to take a given image and fetch it at a given size reliably, with a minimum number of network round-trips
22:51:15 <gilles> how is that not currently possible?
22:51:19 <brion> it's hard to rely on the raw thumbnail URLs currently as asking for too large an image may fail etc, or you may have a funny format that's ahrd to predict
22:51:22 <paravoid> yeah, I've heard this before but it's not very clear from the task -- and I'm not sure what /else/ would they need for example
22:51:29 <brion> eg you hvae to know to map .svg to .svg.png
22:51:32 <DanielK_WMDE_> how does purging work for different sized thumbnails?
22:51:39 <gwicke> we are discussing the right size to serve on tasks like https://phabricator.wikimedia.org/T125983, but ultimately none will work for every consumer
22:51:46 <DanielK_WMDE_> is the current mechanism sufficient?
22:52:13 <gwicke> I am not even sure whether old images can contain slashes in their name
22:52:16 <DanielK_WMDE_> brion: yea, ran into that back in the toolserver days :)
22:52:18 <brion> there are also secondary concerns like wanting to be able to ask for sizes differently (mobile web/apps frequently want cropped-to-square thumbnails which is different from the max-bounding-box)
22:52:30 <brion> where just having a clean way to extend the URL-based api would be nice
22:52:30 <paravoid> as an example: I had previously heard a need for thumbnails that would crop/center on faces
22:52:42 <gwicke> client-side quality selection based on bandwidth, device specs etc is a common need
22:52:49 <TimStarling> pretty sure image names can't contain slashes
22:53:05 <brion> yeah image names cannot contain slashes :)
22:53:21 <gilles> yes, because the whole image path is suppose to correspond to the actual file path on basic mediawiki instances
22:53:22 <brion> let's make sure we capture these notes on the task!
22:53:22 <gwicke> definitely not new ones, I'm less sure about historical ones
22:54:07 <gwicke> the filerepo code seems to support slashes
22:54:51 <brion> filerepo needs to be able to handle relative paths in some places, but identifiers for files *should* not allow /s. if they do.... that may be a bug ;)
22:54:57 <gwicke> in any case, rewriting thumb URLs is not something that was designed as a sane API, and is probably not something we should promote to one
22:55:07 <paravoid> slashes in filenames sounds like actively looking for trouble
22:55:27 <Vulpix> https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads#Known_problems_with_database_names_having_non-alphanumeric_characters :)
22:55:42 <brion> ok we're running short on time, anything else to add?
22:56:10 <gwicke> so the task proposes a simple strawman, and I'd like some input on that
22:56:43 <gilles> if it's going to include all the needs we've just mentioned (destination format, cropping) it's looking more and more like the IIIF feature set
22:56:53 <gilles> maybe a bit more than that, but it's a format we can participate in
22:57:06 <gwicke> a prototype of this would be fairly easy to implement; a production version should not duplicate cache entries, so the translation between old & new style URLs should ideally happen in Varnish
22:57:21 <gilles> thumbor also has its own url format that covers most, if not all of that. but that's not a standard. can be used as inspiration, though
22:57:55 <brion> yeah IIIF is something i'm looking into actually as well :)
22:58:00 <gwicke> brion realized that most of those advanced features are normally server-supplied
22:58:03 <brion> so i'll read up on that and add more notes
22:58:12 <gilles> doing the translation in varnish is how you're going to give ops nightmares. what's the VCL going to look like when it's version 6, 7 of the API and all the old URLs still have to redirect?
22:58:22 <Scott_WUaS> great meeting and developing process, all! thank you :)
22:58:33 <bd808> paravoid: images cropped around the focal point of the image is a use case for "hero images" such as the native apps and hovercards use. In the apps they do it client side using native libs but on the web we need a backend (thumbor) to do it.
22:58:35 <tgr> two things that weren't clear in the task
22:58:46 <tgr> is this going to happen in MediaWiki core?
22:58:53 <TimStarling> tgr: maybe next week or on the task?
22:58:53 <paravoid> nod, I'm not very fond of adding this kind of complexity at the edge
22:58:55 <TimStarling> it's 58 past now
22:59:00 <tgr> ok
22:59:20 <gwicke> paravoid: it's the only way to use edge-cached content, afaik
22:59:24 <paravoid> depends on the amount of the complexity of course
22:59:34 <TimStarling> we'll presumably have a second meeting on this next week
22:59:41 <paravoid> sort of -- you could go through a transition with the old URLs redirecting in theory
22:59:41 <robla> next week: https://phabricator.wikimedia.org/E66/26
23:00:05 <paravoid> that's something that we'll need to do at some point if we decide to do content hashing URLs, right?
23:00:21 <gwicke> yes, there will need to be a translation service & redirects
23:00:36 <gilles> don't design this API with only existing use cases in mind and assume it's rarely going to get updated. that's exactly how we got our ugly thumbnail urls
23:00:40 <paravoid> I hope noone suggests that we do old URL -> content hash URL mappings in varnish :)
23:00:54 <gwicke> gilles: thumb urls weren't even designed as an api
23:00:54 <Scott_WUaS> gwicke: translation service?
23:00:54 <gwicke> they were just a way to somehow internally reference images
23:01:08 <TimStarling> ok to end the meeting now?
23:01:20 <paravoid> not even that afaik -- they were just the directory structure on the web server that served them
23:01:20 <gwicke> in any case, I would appreciate your input on the task
23:01:27 <paravoid> will do
23:01:36 <gilles> what you propose isn't an API, it's a URL format. let's call it what it is. semantically a REST API for images should support PUTs, DELETEs, etc.
23:01:49 <gilles> if it's only GETs it's another URL scheme that's all
23:02:19 <TimStarling> #endmeeting