Change Details

## 1) Continue REST API build-out and service development mentoring # Focus on high-traffic end points. # Thumb API? # Develop a longer term layout for the REST API. {@eevans //upvote//} # Figure out a long-term plan for supporting existing services {@mobrovac //work-on//} # Service development mentoring {@mobrovac //go-to-so-work-on//} ## 2) Migrate several job queue use cases to the change propagation service; start dependency tracking # Build on work done in Q2 to replace job queue use cases. {@mobrovac: this should be merged with point 3 below imho} # Support dependency tracking (by integrating with link tables, tracking new dependencies), and start using this for change propagation. {@mobrovac //upvote//, //work-on//} # Develop long-term plan for replacement of the job queue {{@mobrovac //upvote//, //work-on//}} # Light-weight Kafka replacement for small-scale/dev deployments {@Pchelolo //work-on//} # Atomic event production from MediaWiki {@eevans //work-on,upvote//, @mobrovac //upvote//} ## 3) Storage: Separate hot from warm content; import full wikitext history # Store current versions separately, so that they are cheaper to access & can be replicated to PoPs with modest resources. {@Pchelolo //work-on//} # Improve compression ratios for old revisions, possibly at the cost of some access latency. {@eevans //work-on,upvote//} # Import the full wikitext history # Evaluate RB as an option for a longer-term ExternalStore replacement. {{@mobrovac //upvote//, possibly //work-on//} ## 4) Community engagement # Finish the RESTBase-The-Framework work {@Pchelolo //work-on//, @mobrovac //upvote//} # Consider using RB-the-FW as a replacement for #service-template-node {@mobrovac //upvote//} # Create a showcase on how to use it RESTBase-The-Framework (step-by-step tutorial) {@eevans && @mobrovac //work-on,upvote//} # Convert RESTBase to ES6 {@Pchelolo //work-on//} # Community engagement / marketing: tech talks, blog posts, updated docs & showcase {@eevans && @mobrovac //work-on,upvote//} # Push distribution for third party users forward (docker images, https://phabricator.wikimedia.org/T92826#1804775) {@mobrovac //upvote//, //work-on//} # Release RESTBase 1.0 (signaling a level of readiness, maturity). {@eevans //work-on,upvote//, @mobrovac //upvote//} ### Legend | //upvote// | indicates an opinion that the item should be included in the goals | | //work-on// | indicates a desire to work on the item | ## See also: - {T120171} - {T120242}

## 1) Continue RESTStrengthen: API build-out and service development mentoring& community build-out # Focus on high-traffic end points. # Thumb API? # DContinue API build-out & scale service develop a longer term layout for the REST API. {@eevans //upvote//}ment community resources. ### Key results # Figure out a long-term plan for support- Scale service development by refining existing services {@mobrovac //work-on//}guidelines and documentation. # Service development mentoring {@mobrovac //go-to-so-work-on//}- Support apps and the API-driven frontend effort with cacheable high-traffic entry points and clean content interfaces. ## 2) Migrate several job queue use cases to the change propagation service; start dependency tracking # Build on work done in Q2 to replace job queue use cases. {@mobrovac: this should be merged with point 3 below imho} # Support dependency tracking (by integrating with link tables, tracking new dependencies), and start using this for change propagation. {@mobrovac //upvote//, //work-on//}# Dependencies # Develop long-term plan for replacement of the job queue {{@mobrovac //upvote//, //work-on//}}- Operations # Light-weight Kafka replacement for small-scale/dev deployments {@Pchelolo //work-on//}- Reading # Atomic event production from MediaWiki {@eevans //work-on,upvote//- Editing, @mobrovac //upvote//}especially the Parsing team ## 3) Storage: Separate hot from warm content; import full wikitext history## 2) Focus: Storage scaling and selective replication for hot data. # Store current versions separately, so that they are cheaper to access & can be replicated to PoPs with modest resources. {@Pchelolo //work-on//}## Key results # Improve compression ratios for old revisions, possibly at the cost of some access latency. {@eevans //work-on,upvote//}- Prepare for cost-effective history storage by improving compression ratios for HTML content improved to < 10% of raw content size. # Im- Support the full wikitext historyselective replication of hot data to edge PoPs for low-latency API access. ### Dependencies # Evaluate RB as an option for a longer-term ExternalStore replacement. {{@mobrovac //upvote//, possibly //work-on//}- Operations ## 4) Community engagement## 3) Experiment: Reliable event production & change propagation # Finish the RESTBase-The-Framework work {@Pchelolo //work-on//, @mobrovac //upvote//} # Consider using RB-the-FW as a replacement for #service-template-node {@mobrovac //upvote//}## Key results # Create a showcase on how to use it RESTBase-The-Framework (step-by-step tutorial) {@eevans && @mobrovac //work-on,upvote//}- Implement reliable event production from MediaWiki to EventBus. # Convert RESTBase to ES6 {@Pchelolo //work-on//}- Migrate job queue use cases to EventBus consumers. # Community engagement / marketing: tech talks, blog posts, updated docs & showcase {@eevans && @mobrovac //work-on,upvote//}- Provide a light-weight EventBus implementation for third party users. # Push distribution for third party users forward (docker images, https://phabricator.wikimedia.org/T92826#1804775) {@mobrovac //upvote//, //work-on//} # Release RESTBase 1.0 (signaling a level of readiness, maturity). {@eevans //work-on,upvote//, @mobrovac //upvote//}- Prototype fine-grained dependency tracking for change propagation. ### Legend | //upvote// | indicates an opinion that the item should be included in the goals |Dependencies | //work-on// | indicates a desire to work on the item |- Performance for reliable event production. - Analytics for ongoing EventBus development. ## See also: - {T120171} - {T120242}