Page MenuHomePhabricator

Wikimedia Technical Conference 2019 Session: Front-end modernization and standardization
Closed, ResolvedPublic

Description

Session

  • Track: Standardization Decisions
  • Topic: Front-end modernization and standardization

Description

This session will discuss the efforts that are in progress creating and evaluating a plan towards modernizing the Front-End architecture of Wikimedia stack. It will share the process that is being done by the FAWG working group, what the expected outcomes may look like, and requested insights from the participants.

Since the Front-end stack is not strictly just JavaScript, and is dependent on output from the PHP and the structure that it outputs, the consideration for modernizing JavaScript touch on the PHP and backend operations, and all considerations for modernization and componentization must take that into account.

Questions to answer and discuss

Question: How would we want to build modern user interfaces, in a perfect world?

Question: Why haven’t we done it yet? What type of challenges should we consider? What are things we can learn from failed experiments in the past?

Question: What should we think about when we’re considering for the next steps?

Pre-reading for all Participants


Post-event summary:

  • ...

Post-event action items:

  • ...

Session notes:

Session Leader: Moriel; Facilitator: Brooke; Scribe: Chris, Amir (文Α🐅🏔️)
https://phabricator.wikimedia.org/T234649

Session Attendees
Grant, Fisch, Amir, Dan, Joe, Cormac, Wil, Kaldari, Niharika, Daniel, DJ, Galder, Leszek, Máté, Stephen, Leszek, Monica, Pablo, Volker E.

Notes:

  • Hello from Moriel. Engineer in the Product team at the WMF. Front end architecture working group FAWG ("fog").
  • Cross-departmental working group. Working to modernize the frontend tech stack.
  • The separation between UI and content in MediaWiki is completely non-clear. Line between what is "interface" and what is "content" is blurred.
  • At this session the WG will think how to do, but hopefully we won't do it as it always happens—think for ten years.
  • Putting the responsibility on our selves to come out with a plans soon that is iterable.
  • Open up the conversation and see more perspectives, diversity of voices, and where to go to continue having discussion
  • Introduction
    • Eight people in the working group and a facilitator (Kate Chapman) and a consultant Diana Montalion to help challenge assumptions ("This can't happen").
    • Look on MW.org for info on the group
    • Goal (still defining) creating experiences for users that create a modern, delightful, and trusted source of knowledge.
    • Not just picking a framework, but a workflow and complimentary tooling.
    • What the user interacts with and how we enable that to be modern - but also on the sever, isomorphic, what does it mean, what does the future look like?
    • Meets weekly, and split up into three groups
      • frameworks (What could be done, what does it mean)
      • big picture group (fast and easy for the engineer or for a gadget developer? or maybe both?)
      • integration and migration (challenges we will encounter and what to consider with actual migration).
    • What we've been asking ourselves?
    • Q: Have we considered design as a group as well to consider?
      • A: Yes, 100%. Among other functions. Non technical, but product focused to be included.
    • Q: Where are you gathering your output? What is it? When is it?
      • A: A lot of the discussions are early we didn't want to drop on a wiki because that would be confusing. Will be put on-wiki. Eventually we'll have a plan. An implementation plan. Something that we can follow up on.
    • Q: Will this go through an RfC process?
      • A: Yes, this is not just one RfC, but many. What will APIs look like. what about rendering, composition layer – multiple RfCs in an iterable process.
    • Q: What is out of scope? (Flutter mentioned, are we examining those?)
      • A: How are work will look like? But also think that decisions and migrations need forever, so we need to allow iterations. We need to consider the future and whatever decisions/changes to migrate will be easier because of early planning. More focus on web technologies at the moment.
    • Three questions to focus discussion on.
      • We want to build a modern user interface. Forget everything, if you had all the resources, how would you envision a user frontend system?
      • What's in the API and visualization of the data. Have an interface where anyone can build something. A clear API exposing the data in a ??? way
      • Flexibility of where the interface is rendered. Isomorphic rendering. Frontend rendered on server or client. Gives me flexibility to discover where I want to optimize.
      • Modular frontend, components, attributes, different interfaces build on the same component set. Make new interfaces by creating new components
      • Reuse the same components on frontend, backend, apps, etc. Same codebase.
        • It strikes me that our experience of doing this so far has not been entirely positive - see OOUI, parsoid. What makes it possible to do this now when we we have failed before? Are we sure this is a worthwhile aspiration?
      • Moriel's personal view - Want isomorphic for the use in third-parties where folks can't run node.js, etc. Where folks can run JS only, or parts only.
      • Interface that doesn't penalize editors (logged-in users). Right now whatever you see when logged in it takes time to render the page. Logged out users it's faster. Edge-level composition may solve this.
      • Dan: I support isomorphic for the record. My perfect framework/architecture would produce code, component interfaces, and a general structure that's so simple that even a non-programmer could read it, get some aesthetic pleasure, and understand in some capacity. Expands the universe of who can participate.
        • Clarifying question: Daniel: can you expand
          • We should consider aesthetics and optimize heavily for code simplicity. For example, and these are just my aesthetics so it's just a hypothetical, but I find reactive state management easier to understand than event-driven state management. (event-driven: when something happens, an event is fired, something is listening and changes state. Reactive: when something happens, we modify an object, the framework itself takes care of resolving state or other objects that depend on the object that changed, so you only have to think about how data relates to other data instead of how to communicate those changes. Higher level without loss of coherence is my definition of simple) So we should look at how those types of decisions would affect code simplicity in our shared understanding of "simple" and optimize for it.
      • Interface should be extensible.
    • Why haven't we done it yet? We seem to be in agreement generally. What type of challenges should we consider now? What can we learn from failed experiments in the past?
      • Máté: There's investment in in-house frameworks (OOUI).
      • TheDJ: Many reasons. One is that we are very backwards compatible. The gap to cross from status quo to something new is large. Separation of concerns between data and view has been bad in the past. Moving UI is difficult. We have a lot of requirements based on language, server-side rendering, RTL/LTR CSS, where we have so many extra demands that an out-of-the-box framework would not meet. It's difficult to go into an existing project and make changes to meet our needs.
      • The extension mechanism is complicated and locks innovation by fear of breaking other tools. Our tools have not kept up with how other practices have evolved. We can't do certain things ???
        • What we make can not be fed back into the system
      • There is question on how to do isomorphic when there is no javascript (JS) running on the server side.
      • Requirement that MW has to work without JS on the server side. On a vanilla LAMP stack without root access. On devices that don't have JS - not just reading, but editing/maintenance operations.
        • Joe: Response - having some content functioning without JS is important - privacy, TOR as example of access.
      • Volker E.: Migration path??? Problems with OOUI is that it came from one project and then was put onto all MediaWiki without enough resourcing. Make sure having leadership with clear direction and majority of developers working in this one framework.
      • Moriel's: We have a culture of 100% agreement and that makes it difficult to progress - circular arguments that make it easier to stay in place.
      • Do you think that it's agreement or suppression of disagreement? I felt that in the past sometimes my disagreements were ignored.
      • One thing is that there are a lot of aspirations. OOUI tries and doesn't do it all the way. Maybe one of the things [large aspirations] that may hold us back
      • TheDJ: We did not have infrastructure for making migrations before. We now have more of that at least. Even OOJS. Now we at least know where our widgets are!
      • Máté: Can we use ResourceLoader with other frameworks that are used in compiling of code.
    • What should we think about when we consider the next steps?
      • One of the big things we should just get started somewhere. There has to be something we can reasonably conclude. Keep some movement. What's the smallest thing we can get moving on?
      • Kaldari: Remember why we're a successful movement - we are easy to get started with and jump into. Other's can build interfaces for us. If we can create something that is easy to jump in and create something, then people will create things for us.
      • We have very different use cases, volunteer, professional, mobile apps (developers), wikidata. There may be solutions that give some benefits, but allow others to build their tools.
      • Q: How does this project relate to the overarching goals of the WMF?
      • A: Part of all of it
      • Q: Are there other priorities in Product that influence?
      • A: Yes, like desktop refresh - they need something new - can't do with existing stack.
      • Q: Asking who's actually going to do the work? Not on anyone's roadmap.
      • Let's get Toby and Grant to put this on the roadmap.
        • DONE! [tongue in cheek comment]
      • Amir: How to get the migration on despite all the legacy gadgets and user scripts. This will break them. We will need to give time to migrate. Can't simply break them. Allow people to stay with the old skin / frontend / whatever is needed to make gadgets work. Please start writing, but give a deadline to switch. Maybe three years, maybe three months :)
      • TPT: We're going to break the gadgets. Create a way to write them in the new framework.
      • Dan: We have thousands of lines of JS, but we eat that for breakfast. Maybe we can just rewrite them. It might be useful to measure stuff. Determine how much there is so we know. And then maybe we can just rewrite it and it won't be broken.
        • Q: what would we want to measure specifically
          • Roughly understand what we'd need to migrate. Don't be afraid of a big number you don't know. It might not be that big.
          • Also how many people are using?
          • How many lines of code are being executed?
      • Daniel K.: One comment on breaking gadgets. Many bind to skin structure. Might be a way we change how we implement Special: pages. If we focus on those first we might not break gadgets/skins first. People would be more excited to break their code if we give them a more stable foundation.
      • Máté: Take a look at the HTML rendering (skin classes) how we could better separate data flow and actual rendering. Might not be depended on framework and could pave the way toward futuer iterations.
      • TheDJ: More about gadgets/user scripts. There are many dependencies. Using import script is not the same as requiring a module. Maybe we could reuse or use the code in a better way. If we have user scripts on any page. we can say this is more forward looking, but if that means at some point in the future we have to load 3 frameworks and we promise that code will keep working that's not going to reconcile itself. In the most generic way. An App system. Every ??? That app can be the one thing that does the one thing, but that doesn't help with adding a link in a toolbar. Or maybe that is the separation we need to make? If you want to interface with the UI - use the framework - more complex things outside of the gadget space.

most gadget authors are trying to use jquery.ui instead of OOUI. Why? We need to understand it.

      • TheDJ: That might not be true any more - I only see it in older complex scripts. Newer smaller, rewritten stuff is in OOUI.
      • Amir: Last time I checked, they would like to use OOUI, but found it too hard to get started because documentation is lacking. This may no longer be true.
    • Volker E.: Standard UI APIs - how much do we want gadget users to be able to do? With OOUI there has been a limitation in order to not end up with all kinds of custom interfaces confusing users. We should be clear how much we allow?
    • Moriel: We have a lot of requirements and want to make things beautiful and easy to use - that can put things in contention with one another. Issues with documentation.
      • That kind of balance we need to figure out. Let folks build what they want and provide tools to handle things they don't have to worry about (language/internationalization)
    • TheDJ: QUESTION We should define the allowed interactions of gadgets/userscripts with the UI/page/etc (development guidelines)
      • TheDJ: ACTION For this ^^^ we should index what interaction points gadgets/userscripts have right now (toolbar button, portletlink, p-personal), watchlist line-link-widgets. Then we can better assess impact of changes as we define stable interfaces for these gadgets/userscripts.
    • Is any of this going to be run by the community? That might take years?
      • Moriel, try and find incremental changes. We have need like desktop refresh to move. Have to find a balance between input and making a decision.
    • Leave feedback on the talk page!
    • For next steps: what are the ways we can de-risk these dections? How can we make them lest fraught? What can we do as the foundation. FAWG, engineers?
  • In closing One thing that that we discovered in the group is that we have many assumptions. This tendencies to ask "Really? is it?" Or "Why" was to discover many things we can do incrementally. Do that yourselves.
    • Any extra questions
      • Galder: I was silent till now because I was trying to understand what "frontend" means for you. If there is an RfC it will be important to define what "Front end" means. I thought that it means the design, but for you it may mean "the technology that implements the design".
        • Moriel: We are continuously working on defining that. That is one of the issues. The fact that the UI and the technology are so entangled in each other is part of the problem. Working to define all of these.
      • On gadgets. Would it be helpful to define standard levels of support? Deprecation policies.
  • Thanks to 文Α🐅🏔️ for helping with notes!

Event Timeline

debt created this task.Oct 4 2019, 3:39 PM

Can't wait to help as much as I can with this effort. I think it's going to be a big challenge with an even bigger payoff. I didn't get to be part of the FAWG but I've used several mainstream and several weird frameworks in my career and have hopefully useful opinions about how to understand pros and cons and decide on this specific kind of technology. Looking forward to help in any way I can.

kaldari updated the task description. (Show Details)Oct 5 2019, 1:01 AM
Gilles added a subscriber: Gilles.Oct 7 2019, 8:59 AM

Something missing in the announcements about the FAWG and this task is the possibility that none of the available options are good trade-offs/long-term investments. There is a maturity of the environment required to do such a migration and I don't know if the current frontend framework landscape has reached that stage yet.

For example, it's possible that the most useful parts of these frameworks are things that should be made part of the web platform itself and that resources and efforts in terms of long-term investments would be better made in the standards space (W3C, WHATWG, etc.).

By framing the discussion on "which framework should we pick", it might lock the discussion into a close-minded view in that respect.

If there is a need so common that most websites end up using similar frameworks to achieve it, it might be something that should be part of JS/CSS. We've seen that process from beginning to end with jQuery being less and less useful as features have made their way into JS itself. Now we're still paying the price of jQuery's weight because we're locked into it, it's completely baked into all our code, even though we use very little of it.

We might be early in such a cycle with frontend frameworks. It would be unfortunate to invest a lot of resources in the jQuery-like phase when we could be helping and pushing towards the endgame. I.e. that it becomes part of the web platform and using those features becomes a non-issue. Worse, we could make a heavy frontend framework a page load requirement that will be very difficult to undo, with a potential usefulness lifespan of a few years.

daniel added a subscriber: daniel.Oct 8 2019, 10:13 PM

@Mooeypoo @daniel @Krinkle - Any of y'all want to be session leaders or facilitators for this? Feel free to ping other folks as well.

daniel added a comment.Oct 9 2019, 9:30 AM

@Mooeypoo @daniel @Krinkle - Any of y'all want to be session leaders or facilitators for this? Feel free to ping other folks as well.

I'm happy to contribute, but I'm already leading T234636: Wikimedia Technical Conference 2019 Session: API Integration Testing, so maybe someone else could lead this one?

@Mooeypoo, @Krinkle - We need someone to own this by Friday or it probably won't get approved as an official session.

I'll facilitate this. I'll talk to the group about whether there are specifics we want to make sure are delivered within the given topic.

kaldari updated the task description. (Show Details)Oct 10 2019, 10:57 PM
kaldari reassigned this task from kaldari to Mooeypoo.Oct 15 2019, 6:05 PM
debt triaged this task as Medium priority.Oct 22 2019, 7:00 PM

Something missing in the announcements about the FAWG and this task is the possibility that none of the available options are good trade-offs/long-term investments. There is a maturity of the environment required to do such a migration and I don't know if the current frontend framework landscape has reached that stage yet.
For example, it's possible that the most useful parts of these frameworks are things that should be made part of the web platform itself and that resources and efforts in terms of long-term investments would be better made in the standards space (W3C, WHATWG, etc.).
By framing the discussion on "which framework should we pick", it might lock the discussion into a close-minded view in that respect.
If there is a need so common that most websites end up using similar frameworks to achieve it, it might be something that should be part of JS/CSS. We've seen that process from beginning to end with jQuery being less and less useful as features have made their way into JS itself. Now we're still paying the price of jQuery's weight because we're locked into it, it's completely baked into all our code, even though we use very little of it.
We might be early in such a cycle with frontend frameworks. It would be unfortunate to invest a lot of resources in the jQuery-like phase when we could be helping and pushing towards the endgame. I.e. that it becomes part of the web platform and using those features becomes a non-issue. Worse, we could make a heavy frontend framework a page load requirement that will be very difficult to undo, with a potential usefulness lifespan of a few years.

This is good perspective and important to keep in mind. But we're not on vanilla JS contemplating a move to a framework. We're using a framework we built and that comes with some drawbacks that more mainstream frameworks can help with. We shouldn't load a big framework on every page load, we should look at each framework's plans to "melt away" into evolving W3 standards, of course. But we should also think about developer productivity, the ability to attract new talent, time saved not having to maintain our own framework, and so on. We can incorporate all of the fantastic work that went into OOUI into anything else that has a sensible architecture for components, traits, etc. So I think we have to first come up with a tradeoff formula and then plug in our options. Something like:

(performance change) + (developer productivity) + (ease of hiring) + (cost of migration) + (predicted cost of next migration)

Gilles added a comment.EditedOct 23 2019, 6:47 AM

You would have to migrate your entire stack to a new framework every 4 years if you wanted to keep up with the flavour-of-the-day trend that makes hiring "easier". For the backend, in 2011 Ruby on Rails would have looked like a great choice by that standard. What's RoR's reputation today? What does it look like in terms of hiring RoR developers today, even? Pretty terrible. Today's ease of hiring is tomorrow's huge technical debt and difficulty of hiring.

In my opinion, ease of hiring should not be a criteria for something that is less than a decade old, given the duration of hype cycles (and hiring market apparent dominance) that have been seen for most frameworks in the past. The likelihood that react et al. will fall out of grace in the next 5 years is very high, based on past long-term value of all JS frameworks.

The market, the hype and the hiring dominance are dominated by startups who constantly write new things from scratch. They never have to pay the cost of an expensive huge legacy migration like we do.

It's easy to forget how completely dominant jQuery was at one time and how little time it took for it to become bloat. Let's learn from that mistake, our dependency on frameworks should be loosely coupled, not something that affects the way all our code is written.

greg added a comment.Oct 23 2019, 9:39 PM

(Programming note)

This session was accepted and will be scheduled.

Notes to the session leader

  • Please continue to scope this session and post the session's goals and main questions into the task description.
    • If your topic is too big for one session, work with your Program Committee contact to break it down even further.
    • Session descriptions need to be completely finalized by November 1, 2019.
  • Please build your session collaboratively!
    • You should consider breakout groups with report-backs, using posters / post-its to visualize thoughts and themes, or any other collaborative meeting method you like.
    • If you need to have any large group discussions they must be planned out, specific, and focused.
    • A brief summary of your session format will need to go in the associated Phabricator task.
    • Some ideas from the old WMF Team Practices Group.
  • If you have any pre-session suggested reading or any specific ideas that you would like your attendees to think about in advance of your session, please state that explicitly in your session’s task.
    • Please put this at the top of your Phabricator task under the label “Pre-reading for all Participants.”

Notes to those interested in attending this session

(or those wanting to engage before the event because they are not attending)

  • If the session leader is asking for feedback, please engage!
  • Please do any pre-session reading that the leader would like you to do.
debt updated the task description. (Show Details)Oct 25 2019, 8:56 PM
Mooeypoo updated the task description. (Show Details)Oct 31 2019, 5:29 PM
kaldari updated the task description. (Show Details)Nov 11 2019, 5:02 PM
kaldari updated the task description. (Show Details)
kaldari updated the task description. (Show Details)Nov 11 2019, 5:05 PM
Quiddity updated the task description. (Show Details)Nov 12 2019, 10:46 PM
Quiddity updated the task description. (Show Details)
Pablo-WMDE updated the task description. (Show Details)Nov 13 2019, 4:24 AM
Pablo-WMDE awarded a token.
Anomie added a subscriber: Anomie.Nov 13 2019, 7:57 PM
  • Reuse the same components on frontend, backend, apps, etc. Same codebase.
    • It strikes me that our experience of doing this so far has not been entirely positive - see OOUI, parsoid. What makes it possible to do this now when we we have failed before? Are we sure this is a worthwhile aspiration?

To whoever pointed out the sub-bullet: Thank you!

Volker_E updated the task description. (Show Details)Nov 13 2019, 9:57 PM
abi_ awarded a token.Nov 18 2019, 11:29 AM
abi_ added a subscriber: abi_.
  • Reuse the same components on frontend, backend, apps, etc. Same codebase.
    • It strikes me that our experience of doing this so far has not been entirely positive - see OOUI, parsoid. What makes it possible to do this now when we we have failed before? Are we sure this is a worthwhile aspiration?

As far as I can tell, OOUI is an example of the opposite (you have to write code for backend and frontend separately), and parsoid was never intended to run on the client (ok the idea was floated initially, but as far as I know, never targeted in the implementation). The only instance of code reuse between client and server I am aware of is mustache templates, and that seems to work fine. Am I missing something?

greg closed this task as Resolved.Dec 17 2019, 10:56 PM

Thanks for making this a good session at TechConf this year. Follow-up actions are recorded in a central planning spreadsheet (owned by me) and I'll begin farming them out to responsible parties in January 2020.