Page MenuHomePhabricator

Make Kartotherian a monolith
Closed, ResolvedPublic2 Estimated Story Points

Description

Background information

Kartotherian has become a monorepo using lerna to manage package dependency, it allowed the code to be moved to Gerrit and use the CI pipeline on it. However, Kartotherian is still difficult to deploy and have a lot of duplicated code.

What

Refactor code and move away from plugin strategy, make kartotherian a real monorepo with code properly componentized and shared between kartotherian/tilerator

Outcome

Reduces tech debt by removing duplicated code and make it compliant to current services patterns in the Foundation;

How

Without re-architect or code changes, we should merge the package codes as libraries of a monorepo

  • Deprecate server package and merge it into kartotherian/tilerator
  • Deprecate geoshapes package and merge it into kartotherian
  • Deprecate snapshot package and merge it into kartotherian
  • Deprecate babel package and merge it into kartotherian
  • Deprecate cassandra package and merge it into kartotherian/tilerator
  • Deprecate core package and merge it into kartotherian/tilerator
  • Deprecate err package and merge it into kartotherian/tilerator
  • Deprecate autogen and merge it into kartotherian
  • Deprecate input-validator and merge it into kartotherian
  • Deprecate jobprocessor and merge it into tilerator
  • Deprecate layermixer and merge it into kartotherian
  • Deprecate maki and merge it into kartotherian
  • Deprecate module-loader and merge it into kartotherian/tilerator
  • Deprecate overzoom and merge it into kartotherian
  • Deprecate postgres package and verifiy that it's not used and phase it down
  • Deprecate substantial package and merge it into kartotherian
  • Deprecate tilerator package and merge it into kartotherian

Open questions

...

Acceptance criteria

  • Kartotherian is fully monorepo and can be deployed in our infrastructure without changing any code

Event Timeline

Change 588477 had a related patch set uploaded (by MSantos; owner: MSantos):
[mediawiki/services/kartotherian@master] Stop loading geoshapes with server

https://gerrit.wikimedia.org/r/588477

Change 588478 had a related patch set uploaded (by MSantos; owner: MSantos):
[mediawiki/services/kartotherian@master] Stop loading tiles endpoint within server

https://gerrit.wikimedia.org/r/588478

MSantos renamed this task from [EPIC] Make Kartotherian a real monorepo to [EPIC] Make Kartotherian a monolith.Sep 15 2020, 8:43 AM
LGoto triaged this task as Medium priority.Sep 16 2020, 3:38 PM
LGoto moved this task from Needs triage to Epics on the Product-Infrastructure-Team-Backlog board.

Lerna is officially abandoned, so the work in this epic probably includes moving to another monorepo framework?

Change 761666 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move autogen into monolith

https://gerrit.wikimedia.org/r/761666

Change 761678 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move kartotherian-core into the monolith

https://gerrit.wikimedia.org/r/761678

Change 761734 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] Move babel into monolith

https://gerrit.wikimedia.org/r/761734

Change 761740 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move geoshapes into monolith

https://gerrit.wikimedia.org/r/761740

Change 761745 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move snapshot into monolith

https://gerrit.wikimedia.org/r/761745

Change 761899 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move overzoom into the monoith

https://gerrit.wikimedia.org/r/761899

For some of these components we were planning to "merge it into kartotherian/tilerator", but now that component has been removed. Can you update these lines? Should the remaining components all be folded into kartotherian?

Change 762065 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move substantial into monolith

https://gerrit.wikimedia.org/r/762065

Change 762066 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] Move maki into monolith

https://gerrit.wikimedia.org/r/762066

Change 762069 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move server into monolith

https://gerrit.wikimedia.org/r/762069

Change 762070 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move module-loader into monolith

https://gerrit.wikimedia.org/r/762070

Change 762071 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move input-validator into monolith

https://gerrit.wikimedia.org/r/762071

Change 762072 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Move err into monolith

https://gerrit.wikimedia.org/r/762072

Change 762073 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Convert kartotherian from a monorepo

https://gerrit.wikimedia.org/r/762073

I've smoke-tested the basic use case locally, and can still render maps with maki.

awight renamed this task from [EPIC] Make Kartotherian a monolith to Make Kartotherian a monolith.Feb 15 2022, 9:46 AM
awight claimed this task.

Change 761678 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move kartotherian-core into the monolith

https://gerrit.wikimedia.org/r/761678

Change 761734 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move babel into monolith

https://gerrit.wikimedia.org/r/761734

Change 762066 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move maki into monolith

https://gerrit.wikimedia.org/r/762066

Change 762065 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move substantial into monolith

https://gerrit.wikimedia.org/r/762065

Change 761899 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move overzoom into monolith

https://gerrit.wikimedia.org/r/761899

Change 761666 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move autogen into monolith

https://gerrit.wikimedia.org/r/761666

Change 761740 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move geoshapes into monolith

https://gerrit.wikimedia.org/r/761740

Change 761745 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move snapshot into monolith

https://gerrit.wikimedia.org/r/761745

Change 762069 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move server into monolith

https://gerrit.wikimedia.org/r/762069

Change 762070 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move module-loader into monolith

https://gerrit.wikimedia.org/r/762070

awight claimed this task.
awight moved this task from Doing to Tech Review on the WMDE-TechWish-Sprint-2022-03-02 board.
awight changed the point value for this task from 8 to 2.
awight set Final Story Points to 8.

Change 762071 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move input-validator into monolith

https://gerrit.wikimedia.org/r/762071

Change 762072 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Move err into monolith

https://gerrit.wikimedia.org/r/762072

Change 762073 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Convert kartotherian from a monorepo

https://gerrit.wikimedia.org/r/762073

awight removed awight as the assignee of this task.Mar 4 2022, 1:22 PM
awight moved this task from Tech Review to Done on the WMDE-TechWish-Sprint-2022-03-02 board.

Nice work! The next step is to get rid of maps/kartotherian/package and move the needed functionality to the monorepo, I'll get a starter patch for that.

Jgiannelos claimed this task.
Jgiannelos added a subscriber: Jgiannelos.

Today we pushed to production kartotherian from the monorepo version. Feel free to reopen if something is missing.