Page MenuHomePhabricator
Create Task
Maniphest T163706

[Spike] [4hrs] Investigate loading a random page from a zim file using an interceptor
Closed, DeclinedPublic
Actions

Description

  • Consider what it would take to create or modify an Interceptor to pull from an additional zim file source
  • Consider marking the source of the data with a custom header (cache, saved cache, or zim) from the cache Interceptor
  • Consider using URL rewriting to transform a MediaWiki request to a zim request
  • Consider the HTML blob to WebView to Java POJO path and what's necessary to do in Javaland vs WebView

Answer the question: can we use Interceptors for ZIM files? If so, make a task to refactor the initial implementation to use Interceptors and munge the data into the format expected.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Compilations: integrate with Random feed card.apps/android/wikipediamaster+23 -5
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 DeclinedDbrantT163706 [Spike] [4hrs] Investigate loading a random page from a zim file using an interceptor
 Resolved• MhollowayT163714 Create a page content pack info client
 Resolved• MhollowayT172135 Determine how to embed metadata into zim files
 InvalidNoneT172147 Expose endpoint for content packs API
 Resolved• MhollowayT170732 Create offline content packs endpoint
 Resolved• MhollowayT163586 Develop the content packs endpoint response schema
 InvalidNoneT172732 Create a production-ready zim content pack generation and upload service
 Invalid• FjalapenoT170841 Determine where to generate Zim content packs using Production WMF infrastructure
 Resolved• MhollowayT172296 Determine how to gather top-viewed article lists for use in generating ZIM files
 Invalid• FjalapenoT172734 Collect data on zim file generation and upload times
 InvalidNoneT172223 Investigate the status of the WMF-hosted openzim repo (and determine an update strategy)
 Resolved• MhollowayT172736 Determine how to extract article lists from their canonical sources
 Resolved• MhollowayT172797 Consolidate extraction steps in self-contained scripts
 InvalidNoneT172738 Update mwoffliner for WMF client-specific needs
 InvalidNoneT172739 Treat audio separately from video
 InvalidNoneT172764 Remove unnecessary/Kiwix-specific content massaging
 InvalidNoneT172768 Determine the lifecycle of a WMF zim file
 Invalid• MhollowayT172769 Finish productionizing/puppetizing the mwoffliner zim creation environment
 Resolved• FjalapenoT170843 Determine where to host zim files for the Android app
 Invalid• JMinorT169905 Determine the list of content packs to make available in V1
 InvalidNoneT172803 Write a script for getting article lists for other languages from enwiki lists for Offline Library articles
 InvalidNoneT171117 Track zim file downloads
 Resolved• TbayerT171764 Gather data to estimate Zim file downloads
 Resolved• FjalapenoT172137 Estimate both beta and production downloads of Zim files
 InvalidNoneT172123 Determine how to upload Zim files to Swift infrastructure
 ResolvedfgiunchediT172735 Create 'pagecompilation' Swift account(s) (beta + prod) for Readers offline article compilations project
 Invalid• FjalapenoT172148 Determine URL paths for Zim files

Event Timeline

• NHarateh_WMF created this task.Apr 24 2017, 3:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2017, 3:50 PM
• Mholloway renamed this task from [Spike] Investigate loading a random page from a sim file using an interceptor to [Spike] Investigate loading a random page from a ZIM file using an interceptor.Apr 24 2017, 3:54 PM
• Mholloway subscribed.
• Niedzielski renamed this task from [Spike] Investigate loading a random page from a ZIM file using an interceptor to [Spike] Investigate loading a random page from a zim file using an interceptor.Apr 24 2017, 4:19 PM
• Niedzielski updated the task description. (Show Details)
• NHarateh_WMF added a comment.May 8 2017, 4:02 PM

The goal of this task is to flush out enough information to do the actual implementation (another task).

• NHarateh_WMF renamed this task from [Spike] Investigate loading a random page from a zim file using an interceptor to [Spike] [4hrs] Investigate loading a random page from a zim file using an interceptor.May 8 2017, 4:04 PM
• NHarateh_WMF added a project: Spike.May 8 2017, 4:33 PM
RHo mentioned this in T166575: When offline, the Explore feed should show only content for whatever is available in an offline compilation.May 30 2017, 3:56 PM
Dbrant moved this task from Needs Triage to Android-app-release-v2.7.20x-Dookie💩 on the Wikipedia-Android-App-Backlog board.Jun 30 2017, 2:04 PM
Dbrant edited projects, added Wikipedia-Android-App-Backlog (Android-app-release-v2.7.20x-Dookie💩); removed Wikipedia-Android-App-Backlog.
• Niedzielski added a subtask: T163714: Create a page content pack info client.Jul 6 2017, 4:53 PM
• Niedzielski updated the task description. (Show Details)Jul 6 2017, 4:56 PM
• NHarateh_WMF moved this task from To Do to Ready for dev on the Wikipedia-Android-App-Backlog (Android-app-release-v2.7.20x-Dookie💩) board.Jul 6 2017, 5:00 PM
gerritbot subscribed.Jul 11 2017, 9:05 PM

Change 364577 had a related patch set uploaded (by Dbrant; owner: Dbrant):
[apps/android/wikipedia@master] Compilations: integrate with Random feed card.

https://gerrit.wikimedia.org/r/364577

gerritbot added a project: Patch-For-Review.Jul 11 2017, 9:05 PM
Dbrant claimed this task.Jul 11 2017, 9:09 PM
Dbrant moved this task from Ready for dev to Code Review on the Wikipedia-Android-App-Backlog (Android-app-release-v2.7.20x-Dookie💩) board.
gerritbot added a comment.Jul 14 2017, 11:16 PM

Change 364577 merged by jenkins-bot:
[apps/android/wikipedia@master] Compilations: integrate with Random feed card.

https://gerrit.wikimedia.org/r/364577

Dbrant moved this task from Code Review to Done on the Wikipedia-Android-App-Backlog (Android-app-release-v2.7.20x-Dookie💩) board.Jul 21 2017, 2:09 PM
Kelson subscribed.Aug 9 2017, 7:35 PM

I'm not sure exactly what should be done here... in particular what is an "interceptor". But the kiwix-lib already provides a method to load a random article.

Dbrant closed this task as Declined.Aug 9 2017, 7:47 PM

Whoops; this will not be done, and is already implemented in the app in a different way.

• Mholloway changed the status of subtask T163714: Create a page content pack info client from Open to Stalled.Aug 12 2017, 2:32 PM
Dbrant closed subtask T163714: Create a page content pack info client as Resolved.Aug 24 2017, 5:53 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits