Page MenuHomePhabricator

Where should design tokens live? Separate repository or monorepo with Design System?
Closed, ResolvedPublic

Description

Location (repository) of design tokens

Separate repos

Currently Wikimedia Foundation is building on top of WikimediaUI Base variables that are “living” in a separate repository on Gerrit.

Advantages:

  • Separation of concerns
  • Small and lean Git repo
  • Separation of Git history

Disadvantages:

  • Tokens are not easily updated in duo repo setup when directly working on components, possibly leading to an easily growing list of variables that need to be upstreamed and not the other way round (tokens available ahead)
Monorepo

Wikit in contrast has tokens part of their library in a monorepo, but with two different npm package definitions for the library and the tokens

Advantages:

  • Updates are immediately visible in the component library
  • Still exist as lone-standing npm package
  • It could be an advantage to roll-out the library to MW core as one, to make the tokens more easily accessible beyond the component library

Disadvantages:

  • Mixed Git history, that might long-term be hard to digest when for example unrelated design system changes are happening in the library, see Wikit log
  • The tokens should be technically agnostic is requested in a different task, but then they live alongside one tech-choice implementation.

A separate solution could be a Git submodule approach. Although Foundation has seen trouble with those setups in the past and possibly in the future T268283.


WiKit ADR on monorepo vs separate repos.

Acceptance criteria

  • Decide on repository location, single or joint monorepo. Monorepo it is, see last comment.
    • Ensure that tokens are available not only for component library, but for MediaWiki core (& skins & extensions), and code projects outside MW as well

Event Timeline

I like the idea of using a monorepo approach here. And not just for tokens – I could see the following things potentially existing as separate packages/workspaces within the repo:

  • Design tokens
  • The Vue component library itself (would pull in the tokens as a dependency)
  • A possible "lite" version of some components using petite-vue (T291666)
  • Documentation/Demo site (would pull in the components as a dependency)

We might want to look into using Yarn to manage this, it seems to be a popular approach these days and there would be no need for an additional tool in that case (see https://yarnpkg.com/features/workspaces). Yarn's zero-install feature may also be worth looking into, but that's a topic for a different discussion.

Would like to understand the considerations by WMDE in their approach better? Have you considered Git submodules as well for example? @ItamarWMDE @Jakob_WMDE @Tonina_Zhelyazkova_WMDE

Also, for better management of the several packages Wikit makes use of Lerna.

I'm not convinced of the monorepo approach necessarily right now. Most of the pros and cons come down to process. When I look at the idea above, I feel a next request or logical step in the same path would be to also have DSG in the same monorepo.

Two more thoughts

  1. Git history? Has this ever been a concern for WMDE? Repeating the Wikit Git log link from task description. It's hard to keep good overview of changes with so many different things packed together in one repo, even though we might be able to mitigate some of this with a strict commit message policy. – I'm thinking of packing Style Guide together with all assets.
  2. Down the road when the tokens are pretty settled, and the library will mainly see improvements/bug fixes in the current layout of Wikit the tokens and the components lib versions are in sync and get updated on every release of latter. I find this counterintuitive/confusing.

Currently with OOUI we're bringing the different repo/different packaged/different version base variables to MW core synced on OOUI updates and still have other consumers of vars/tokens satisfied. This seems saner to me.

Aside from process, this comes down to a fundamental question of how we want to structure and present the different parts of our design system. I am in favor of a holistic approach in which the resources included in the design system—the Design Style Guide, design tokens, and current component library(ies)—are housed together. To me, it makes sense to have a single source of truth for design standards, design tokens, and working components for the following reasons:

  • It makes understanding the design system easier, since all of the resources it offers are housed together
  • It lessens the gap between design and development, signalling to everyone that we are all using a shared system
  • It creates a single place for most of the documentation needed by someone on a product team or a volunteer who wants to learn how to use our system to build a user interface. There is often confusion now over where to go to find the right resources (on-wiki, figma, design style guide, OOUI demos, OOUI docs, wikimedia UI base...) and we could mitigate most of this. Of course, some things might not live within the repo (like designs housed in figma, or design-system-agnostic documentation that should like on-wiki e.g. Vue docs) but we could link out to those when necessary.

I'm talking about presentation here, but I think it makes sense from a process and development standpoint as well. Our current processes would have us updating the Design Style Guide and component library completely separately, and trusting that we'll keep them both in sync somehow, which is critical to avoid confusion.

Specific to design tokens, I think we risk a lot of manual work that we have to track to keep things in sync. I've had an item on my to-do list for months to update some variables for the WVUI Radio component once they're added to WMUI base (which I'm pretty sure has happened, I just haven't updated the component yet)—this will happen more and more as we add components and will be especially hard to track since we're hoping to get component contributions from a bunch of different people. I feel like sometimes our solution to this problem is "rely on Volker to remember to do all of this himself," which seems unnecessarily inefficient and a poor use of our lead UX engineer's time.

In terms of being technology-agnostic, I think this is just a matter of thinking of this repository as our design system resources, not as a component library. Within this repo, the component library is just whatever component library(ies) we're currently using. If we deprecate the Vue library eventually, or build a separate component library to use alongside it, that new library could just be added to the monorepo.

I would like to see us produce something like these design system sites:

+1 to everything Anne wrote.

We wrote an ADR for choosing the monorepo approach for WiKit which answers some of those questions for us.

Have you considered Git submodules as well for example?

Yes, in the ADR you'll find that the main advantage we saw in keeping everything in one repo is the ability to preview the effect a token change has on the components. With the current setup in WiKit we can check this a) manually via the branch previews created for every PR, and b) automatically via visual regression tests through Chromatic. With submodules you would first "blindly" commit a token change, and then need an additional commit bumping the submodule hash to see the effect.

I'm not convinced of the monorepo approach necessarily right now. Most of the pros and cons come down to process. When I look at the idea above, I feel a next request or logical step in the same path would be to also have DSG in the same monorepo.

Two more thoughts

  1. Git history? Has this ever been a concern for WMDE? Repeating the Wikit Git log link from task description. It's hard to keep good overview of changes with so many different things packed together in one repo, even though we might be able to mitigate some of this with a strict commit message policy. – I'm thinking of packing Style Guide together with all assets.

Not a huge concern IMO. The boundaries within the monorepo are very clear, each package within one directory. git log -- vue-components/ gives me the history of the components, git log -- tokens/ for that of the tokens. Commits spanning multiple packages (tokens, components, docs) without having to leave other repo's commit hashes/patch IDs are another benefit.

  1. Down the road when the tokens are pretty settled, and the library will mainly see improvements/bug fixes in the current layout of Wikit the tokens and the components lib versions are in sync and get updated on every release of latter. I find this counterintuitive/confusing.

Currently with OOUI we're bringing the different repo/different packaged/different version base variables to MW core synced on OOUI updates and still have other consumers of vars/tokens satisfied. This seems saner to me.

The decision to keep the version numbers in sync was made independently from the monorepo decision for WiKit. We can easily keep things in a monorepo and have different version numbers for tokens and components, or we can have synced version numbers and separate repos.

We chose to have them synced because we expect our applications to often want to use system components, while also using design tokens for application-specific components that should have a consistent look and feel. Applications using different versions of both packages may have inconsistencies caused by the system components internally using different tokens than the application itself. Syncing the version numbers helps to easily check whether the two are aligned.

Thanks for the inputs here, very helpful. I find most of the points above wishful, at the same time reasonable and in reach. I've already shared my concern about version syncing above. For me it's the question how much the tokens will be used outside the library and it seems reasonable to think about de-syncing it at some point.
One other thing that I think is important to emphasize that we don't want to end up in a developer-first catered style guide. It should be design-centered and with the strong impetus of the connection between DSG and the one exemplary standard implementation it might turn into “technical documentation first”.

Volker_E renamed this task from Where should design tokens live? to Where should design tokens live? Separate repository or monorepo with Design Syste,?.Oct 8 2021, 5:54 PM
Volker_E renamed this task from Where should design tokens live? Separate repository or monorepo with Design Syste,? to Where should design tokens live? Separate repository or monorepo with Design System?.

The decision from Vue.js taskforce convening was clear to use Monorepo with sub-package(s) tokens and icons in different stylesheet outputs taking into account all arguments above. We will look into finding out if CI or GitLab will need special attention on the Monorepo solution.

https://github.com/wikimedia/vue-component-library/pull/3 was merged into library as result of this decision.

Volker_E updated the task description. (Show Details)