Page MenuHomePhabricator

Services team roadmap July - September 2015 (Q1 2015/16)
Closed, ResolvedPublic0 Estimated Story Points

Description

Top priorities

Mobile app service deployment and API

Scale RESTBase, replicate across datacenters

Figure out a plan for change / event propagation

Ongoing

  • set up RESTBase cluster in codfw, replicate from eqiad
  • API support for mobile VE: citation and page metadata retrieval and edit APIs. See T102867, T55508.
  • REST API expansion to cover all content-related functionality: T103811
  • enable Varnish caching / purging for high-traffic end points
  • service development support: Graphoid, Citoid, Mathoid, possibly lead images (T88633)
  • Cassandra cluster scaling: T93790
  • Deployment system development (with release engineering)
  • Help with HTML dumps (T17017)

Related Objects

StatusSubtypeAssignedTask
Resolved GWicke
Resolved mobrovac
Resolved mobrovac
ResolvedNone
Resolved GWicke
Resolved GWicke
ResolvedNone
Resolved GWicke
Resolved mobrovac
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedCmjohnson
ResolvedCmjohnson
ResolvedJoe
Resolvedfgiunchedi
Resolved GWicke
Resolved Jdouglas
Resolved GWicke
Resolved GWicke
ResolvedArlolra
Resolved GWicke
Resolved mobrovac
Resolved mobrovac
Resolved mobrovac
Resolved mobrovac
Duplicate Jdouglas
ResolvedAndrew
Resolved GWicke
Resolvedfgiunchedi
Resolvedfgiunchedi
Resolvedfgiunchedi
ResolvedEevans
Resolvedfgiunchedi
Resolved GWicke
Resolved GWicke
Resolvedfgiunchedi
Resolved mobrovac
Resolved GWicke
InvalidNone
ResolvedPchelolo
ResolvedArlolra
Resolved mobrovac
Resolvedbd808

Event Timeline

GWicke raised the priority of this task from to Needs Triage.
GWicke updated the task description. (Show Details)
GWicke added a subscriber: GWicke.
GWicke added a project: Services.
GWicke set Security to None.
GWicke edited subscribers, added: damons, mobrovac, Eevans; removed: Aklapper.
GWicke updated the task description. (Show Details)
GWicke updated the task description. (Show Details)
GWicke closed this task as Resolved.EditedOct 5 2015, 11:49 PM
GWicke claimed this task.

The quarter is over, and it's time to wrap up. It looks like we did fairly well:

  • The mobile app service is deployed, and testing with the Android app has started. Basic Varnish caching support is ready to go once the app team is finalizing their responses. Pre-generating the content and storing it in RESTBase is something we'll take on along with basic change propagation this quarter.
  • RESTBase is up in codfw, with replication from eqiad. Cross-DC traffic is encrypted right from the start. The next step here is enabling multi-instance support, and once that's ready syncing the full dataset to codfw. After testing in staging, multi-instance support has been deployed to production earlier today, and Filippo is well on track to enable this in codfw soon.
  • We have a basic plan for the event bus and change propagation:
    1. Implement a basic Kafka-based event bus, storing JSON events conforming to JSON schemas.
    2. Emit basic edit events from MediaWiki, and change events from RESTBase.
    3. Use these events to orchestrate basic change propagation in RESTBase and other services.
    4. [Q3] Tackle the more complex fine-grained dependency tracking problem (T105766)