Change Details

**Goal:** Identify the most urgent high-level architectural questions / priorities that we need to address. === Criteria for priorities: * Impact: * Touches many important areas * Dependent decisions need to be made soon / are already being made & are hard to change later * Vision: Makes for a compelling building block for a shared technical vision / story. Filtering questions: * Are decisions already being made in practice? * In absence of organizational work around it, would task X get done well or cause more technical debt? -> choose battles! ### Top priorities - {T99088} === Further candidates: * Shared service infrastructure * Standard solutions / APIs for state / storage and replication * Long-term storage strategy; can we abstract backends with good APIs? * Pub-Sub event bus & API (both private & public) * Low-latency event notifications (ex: WebSockets) * Efficient catch-up for dumps, Instant Commons, Wikidata, RC (PuSH, Atom, SyncML, ...) * API integration, versioning and security (auth, headers, content sanitization) * Deployment, CI, staging and service isolation: * can we find an integrated solution? Can we use the same infrastructure for prod & labs? * Dependency management * Extension installation & setup * "build" step in deployment * rolling deploys, health checks etc * NoSQL / Graph / SPARQL services * Configuration management -- etcd? * Can we eliminate Single Points of Failure? * Security: architectural solutions to common problems * Isolating sensitive information: password hashes, mail addresses, ip logs * Auth{n,z} APIs for high-volume service use * System lock-down: Least privilege per service, containers / VMs, VLans (see deployment / CI) * Systematic sanitization: templating, HTML+SVG content (even if wrapped in JSON) * Process * How can we ensure reliability and security while empowering teams to move faster? * Right balance between autonomy and shared infrastructure / standards? * Who is responsible when things break? * Who makes decisions on deployments? * "light weight" RFCs * RFC delegation -> topic "owners" / teams * RFC "red teams" ([[https://en.wikipedia.org/wiki/Red_team)]]. Or at least requiring RFCs to provide multiple options rather than seeking sign-off for a particular direction. * Change management * **Deprecation policy. Especially for JavaScript APIs that are used in user and site JS.** * think about narrower APIs for extensions/gadgets that would be more stable but more limited (compare to Firefox JS add-ons vs old xpcom extensions) * discover current users of deprecated APIs, post notifications / offer help * mwgrep - search through user scripts - useful for starting on deprecation decisions * add more automation & communication tools! * Feature removal. Pursuing simplicity via a process of "leaving or taking things out by constantly examining the difference between inherent and actual complexity, questioning and reducing the number of features, and questioning and reducing the amount of code." ([[http://www.artima.com/weblogs/viewpost.jsp?thread=362327)]] * Versioning APIs * Testability & Modularity * move towards "simple" dependency injection * promote use of "dumb" value objects / messages * promote interface segregation * Release & deployment * Dependency management * Extension installation & setup * "build" step in deployment * Multimedia * may want/need cleaner separation for embedding of fancy media objects (make it simpler for parsoid etc to treat them all alike) * have a stable name for resource, and API to fetch representations * (think about web vs print vs video editions variants?) * does this tie in with maps (and infoboxes?)? * (sending data into an embedded item?) ==== From the summit prep notes: Quick summary of topics that people are interested in: - [ ] Internationalization - [ ] Front-end standardization <- can vary from small to potentially huge changes to interaction - [x] Structured data <- data popular! content model may be less popular. but let's introduce it soon and get more feedback - [ ] Multimedia - [ ] Parsoid/content-api/etc - [ ] MediaWiki Vagrant/QA - [ ] Swift/thumbnails/tech debt - [ ] Browser testing - [ ] ElasticSearch (outside arch?) - [ ] Configuration - [ ] Flow - content model? -> lessons learned on content model issues should be shared! (roan & kunal will poke a doc up later) - [ ] Maps/OSM - [ ] A/B testing ==== Random notes: * web components -- useful model for transclusion and extensions? or keep to UI :) === Old notes [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015]] [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015-Topics]] [[https://etherpad.wikimedia.org/p/MWFixme]] [[https://etherpad.wikimedia.org/p/MWFixme2]] [[https://etherpad.wikimedia.org/p/ArchCom]] [[ https://www.mediawiki.org/wiki/Architecture_guidelines | mw.org Architecture guidelines]] - summarizable as "Incremental changes, Separation of concerns, and You aren't gonna need it" [[ https://www.mediawiki.org/wiki/Talk:Architecture_guidelines | mw.org Architecture guidelines discussion]]

**Goal:** Identify the most urgent high-level architectural questions / priorities that we need to address. === Criteria for priorities: * Impact: * Touches many important areas * Dependent decisions need to be made soon / are already being made & are hard to change later * Vision: Makes for a compelling building block for a shared technical vision / story. Filtering questions: * Are decisions already being made in practice? * In absence of organizational work around it, would task X get done well or cause more technical debt? -> choose battles! ### Top priorities These are the priorities we identified in the discussion so far (which includes a session at the Lyon Hackathon): - {T99088} === Further candidates: * Shared service infrastructure * Standard solutions / APIs for state / storage and replication * Long-term storage strategy; can we abstract backends with good APIs? * Pub-Sub event bus & API (both private & public) * Low-latency event notifications (ex: WebSockets) * Efficient catch-up for dumps, Instant Commons, Wikidata, RC (PuSH, Atom, SyncML, ...) * API integration, versioning and security (auth, headers, content sanitization) * Deployment, CI, staging and service isolation: * can we find an integrated solution? Can we use the same infrastructure for prod & labs? * Dependency management * Extension installation & setup * "build" step in deployment * rolling deploys, health checks etc * NoSQL / Graph / SPARQL services * Configuration management -- etcd? * Can we eliminate Single Points of Failure? * Security: architectural solutions to common problems * Isolating sensitive information: password hashes, mail addresses, ip logs * Auth{n,z} APIs for high-volume service use * System lock-down: Least privilege per service, containers / VMs, VLans (see deployment / CI) * Systematic sanitization: templating, HTML+SVG content (even if wrapped in JSON) * Process * How can we ensure reliability and security while empowering teams to move faster? * Right balance between autonomy and shared infrastructure / standards? * Who is responsible when things break? * Who makes decisions on deployments? * "light weight" RFCs * RFC delegation -> topic "owners" / teams * RFC "red teams" ([[https://en.wikipedia.org/wiki/Red_team)]]. Or at least requiring RFCs to provide multiple options rather than seeking sign-off for a particular direction. * Change management * **Deprecation policy. Especially for JavaScript APIs that are used in user and site JS.** * think about narrower APIs for extensions/gadgets that would be more stable but more limited (compare to Firefox JS add-ons vs old xpcom extensions) * discover current users of deprecated APIs, post notifications / offer help * mwgrep - search through user scripts - useful for starting on deprecation decisions * add more automation & communication tools! * Feature removal. Pursuing simplicity via a process of "leaving or taking things out by constantly examining the difference between inherent and actual complexity, questioning and reducing the number of features, and questioning and reducing the amount of code." ([[http://www.artima.com/weblogs/viewpost.jsp?thread=362327)]] * Versioning APIs * Testability & Modularity * move towards "simple" dependency injection * promote use of "dumb" value objects / messages * promote interface segregation * Release & deployment * Dependency management * Extension installation & setup * "build" step in deployment * Multimedia * may want/need cleaner separation for embedding of fancy media objects (make it simpler for parsoid etc to treat them all alike) * have a stable name for resource, and API to fetch representations * (think about web vs print vs video editions variants?) * does this tie in with maps (and infoboxes?)? * (sending data into an embedded item?) ==== From the summit prep notes: Quick summary of topics that people are interested in: - [ ] Internationalization - [ ] Front-end standardization <- can vary from small to potentially huge changes to interaction - [x] Structured data <- data popular! content model may be less popular. but let's introduce it soon and get more feedback - [ ] Multimedia - [ ] Parsoid/content-api/etc - [ ] MediaWiki Vagrant/QA - [ ] Swift/thumbnails/tech debt - [ ] Browser testing - [ ] ElasticSearch (outside arch?) - [ ] Configuration - [ ] Flow - content model? -> lessons learned on content model issues should be shared! (roan & kunal will poke a doc up later) - [ ] Maps/OSM - [ ] A/B testing ==== Random notes: * web components -- useful model for transclusion and extensions? or keep to UI :) === Old notes [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015]] [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015-Topics]] [[https://etherpad.wikimedia.org/p/MWFixme]] [[https://etherpad.wikimedia.org/p/MWFixme2]] [[https://etherpad.wikimedia.org/p/ArchCom]] [[ https://www.mediawiki.org/wiki/Architecture_guidelines | mw.org Architecture guidelines]] - summarizable as "Incremental changes, Separation of concerns, and You aren't gonna need it" [[ https://www.mediawiki.org/wiki/Talk:Architecture_guidelines | mw.org Architecture guidelines discussion]]

**Goal:** Identify the most urgent high-level architectural questions / priorities that we need to address. === Criteria for priorities: * Impact: * Touches many important areas * Dependent decisions need to be made soon / are already being made & are hard to change later * Vision: Makes for a compelling building block for a shared technical vision / story. Filtering questions: * Are decisions already being made in practice? * In absence of organizational work around it, would task X get done well or cause more technical debt? -> choose battles! ### Top priorities - {T99088}These are the priorities we identified in the discussion so far (which includes a session at the Lyon Hackathon): - {T99088} === Further candidates: * Shared service infrastructure * Standard solutions / APIs for state / storage and replication * Long-term storage strategy; can we abstract backends with good APIs? * Pub-Sub event bus & API (both private & public) * Low-latency event notifications (ex: WebSockets) * Efficient catch-up for dumps, Instant Commons, Wikidata, RC (PuSH, Atom, SyncML, ...) * API integration, versioning and security (auth, headers, content sanitization) * Deployment, CI, staging and service isolation: * can we find an integrated solution? Can we use the same infrastructure for prod & labs? * Dependency management * Extension installation & setup * "build" step in deployment * rolling deploys, health checks etc * NoSQL / Graph / SPARQL services * Configuration management -- etcd? * Can we eliminate Single Points of Failure? * Security: architectural solutions to common problems * Isolating sensitive information: password hashes, mail addresses, ip logs * Auth{n,z} APIs for high-volume service use * System lock-down: Least privilege per service, containers / VMs, VLans (see deployment / CI) * Systematic sanitization: templating, HTML+SVG content (even if wrapped in JSON) * Process * How can we ensure reliability and security while empowering teams to move faster? * Right balance between autonomy and shared infrastructure / standards? * Who is responsible when things break? * Who makes decisions on deployments? * "light weight" RFCs * RFC delegation -> topic "owners" / teams * RFC "red teams" ([[https://en.wikipedia.org/wiki/Red_team)]]. Or at least requiring RFCs to provide multiple options rather than seeking sign-off for a particular direction. * Change management * **Deprecation policy. Especially for JavaScript APIs that are used in user and site JS.** * think about narrower APIs for extensions/gadgets that would be more stable but more limited (compare to Firefox JS add-ons vs old xpcom extensions) * discover current users of deprecated APIs, post notifications / offer help * mwgrep - search through user scripts - useful for starting on deprecation decisions * add more automation & communication tools! * Feature removal. Pursuing simplicity via a process of "leaving or taking things out by constantly examining the difference between inherent and actual complexity, questioning and reducing the number of features, and questioning and reducing the amount of code." ([[http://www.artima.com/weblogs/viewpost.jsp?thread=362327)]] * Versioning APIs * Testability & Modularity * move towards "simple" dependency injection * promote use of "dumb" value objects / messages * promote interface segregation * Release & deployment * Dependency management * Extension installation & setup * "build" step in deployment * Multimedia * may want/need cleaner separation for embedding of fancy media objects (make it simpler for parsoid etc to treat them all alike) * have a stable name for resource, and API to fetch representations * (think about web vs print vs video editions variants?) * does this tie in with maps (and infoboxes?)? * (sending data into an embedded item?) ==== From the summit prep notes: Quick summary of topics that people are interested in: - [ ] Internationalization - [ ] Front-end standardization <- can vary from small to potentially huge changes to interaction - [x] Structured data <- data popular! content model may be less popular. but let's introduce it soon and get more feedback - [ ] Multimedia - [ ] Parsoid/content-api/etc - [ ] MediaWiki Vagrant/QA - [ ] Swift/thumbnails/tech debt - [ ] Browser testing - [ ] ElasticSearch (outside arch?) - [ ] Configuration - [ ] Flow - content model? -> lessons learned on content model issues should be shared! (roan & kunal will poke a doc up later) - [ ] Maps/OSM - [ ] A/B testing ==== Random notes: * web components -- useful model for transclusion and extensions? or keep to UI :) === Old notes [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015]] [[https://etherpad.wikimedia.org/p/ArchitectureCommittee-DevSummit2015-Topics]] [[https://etherpad.wikimedia.org/p/MWFixme]] [[https://etherpad.wikimedia.org/p/MWFixme2]] [[https://etherpad.wikimedia.org/p/ArchCom]] [[ https://www.mediawiki.org/wiki/Architecture_guidelines | mw.org Architecture guidelines]] - summarizable as "Incremental changes, Separation of concerns, and You aren't gonna need it" [[ https://www.mediawiki.org/wiki/Talk:Architecture_guidelines | mw.org Architecture guidelines discussion]]