Phase 1

Week 1: Dec 7 - Dec 13
Task: T119278: Identify the best strategy/APIs to find Commons categories that are within a certain radius of the specified GPS coordinates
Results: https://github.com/nicolas-raoul/apps-android-commons/wiki/Location-based-category-search

Day 1:

• Commenced testing WikiData API to see how it performs in categorizing our sample pictures. Used TABernacle to run queries.
• Filled out GitHub wiki with findings.
• Reading up on Commons:Commons API and Commons:API/MediaWiki.

Day 2:

• Trying to refine my analysis of test results according to feedback
• Talked to mentors about how to progress from here, as Commons:Commons API does not work for our purpose and Commons:API/MediaWiki appears to only return pages whereas we need Commons categories. Trying to find a way around this.
• Brainstormed with mentors at https://etherpad.wikimedia.org/p/commons-app-android-nearby-categories on how to solve this problem. After discussion, we concluded that I should try "Search for existing pics at that location" strategy.

Day 3:

• Commenced testing "Search for existing pics at that location" strategy (using Commons Mediawiki API Sandbox with geosearch generator, 100m ggsradius and 10 ggslimit)
• Filled out GitHub wiki with findings
• Modified 'manual categorization' according to mentor feedback (the benchmark has to be perfect)

Day 4:

• Identified false positives in results for WikiData and "Search for existing pics at that location" tests. Listed them in GitHub wiki.

Day 5:

• Calculated scores for WikiData API and "Search for existing pics at that location" strategy for each sample picture, and their total scores.
• Wrote small Python script to automate the calculations (not required, but it saved time and was fun :)).
• Results are listed at T119278 .

Day 6:

• Performed additional tests using "Method D" (described by mentor on task thread) to test effect of widening search radius. Added results to GitHub wiki.

Day 7:

• Wrote blog post for Outreachy Week 1.

Week 2: Dec 14 - Dec 20
Task: T119279: Find a library to extract GPS coordinates info from a picture, and implement its use
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/31 (merged)

Day 1:

• Searched for library to extract picture coords. After finding ExifInterface, searched for a method of converting the (degrees,minutes,seconds) format to decimal degrees.
• Read these tutorials (link 1, link 2)
• Combined the code in both of them in order to write a 'mini-app' that displays the latitude and longitude of a selected picture, and converts it to decimal degrees

Day 2:

• Trying to find out which part of the existing code would be a good place to put in the coord extraction method (from Day 1).
• Familiarizing myself with how to write and read the logcat, and tried (unsuccessfully?) to fix existing logcat errors in the code.
• Meeting with mentors. Decided on Method D as the answer to Week 1's task. Discussed how to implement the ExifInterface method in the existing codebase, and created a 'debugging' state in the app to simplify testing (we don't have to submit pics to Commons database during initial tests).

Day 3:

• Made a very silly mistake and wasted a few hours wondering why only some of my logs were showing up, before finding out that when filtering logcat messages in the DDMS, if you want to filter for the "image" tag, you should filter for "tag:image" and not just "image". :(
• Successfully got exifInterface methods working in ShareActivity.java, but stopped working after refactoring to put it in its own class. Issue described in more detail at T119279 .

Day 4:

• Fixed refactor problem with help of mentor and this tutorial. Code works in its own class now and logs the latitude/longitude of a picture.

Day 5:

• Implemented methods for conversion of coords from degrees,minutes,seconds to decimal degrees (required for next step)
• Submitted pull request to official repo.
• Modified submitted code according to mentor's feedback. Pull request merged.

Day 6:

• Wrote blog post for Outreachy Week 2.
• Learnt how to use 'git stash' to prevent committing code that is only used for debugging and that cannot be included in a release.

Day 7:

• Started on Week 3's task (T119280) ahead of time to make up for the extra day off that I intend to take during Week 4 (previously discussed with mentors and approved).
• Read up on MediaWiki API documentation, and methods of implementing the http requests to the API asynchronously.
• Meeting with Niedzielski (meeting with Nicolas will have to be held separately this week due to clashing schedules). Details of discussion described at T119280.

Week 3: Dec 21 - Dec 27
Task: T119280: Implement asynchronous calls to the APIs to match an uploaded picture's GPS coordinates with nearby Commons categories
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/33 (merged)

Day 1:

• Trying to build off of a dummy Volley app that @Niedzielski wrote, that sends requests to the MediaWiki API and receives responses as JSON objects.
• Managed to get dummy app to display categories alongside page ids, namespaces and filenames. However, having issues with constructing a Category model for responses. Talking to Niedzielski about how to solve these issues.
• Meeting with Nicolas scheduled for Wednesday.

Day 2:

• Implemented UrlBuilder helper class in ShareActivity.java to build URL for MediaWiki API calls based on coordinates of uploaded image
• Added incomplete APICalls class based on dummy app, that sends asynchronous calls to the API with the URL above. Imported necessary GSON libs via Maven.
• Still having problems with parsing and constructing a Category model for the responses received. Committed new (but incomplete) code to api-calls branch of my own repo.

Day 3:

• Waded through lots of confusing GSON/JSON tutorials before finding this helpful one.
• Finally got JSON unmarshalling to work for categories (I just found out that that means 'deserialization'...). Example output listed at T119280 and pull request submitted to official repo.
• Meeting with Nicolas. Discussed how to handle issues such as no GPS coordinates, no connection to server, etc. Pull request merged.

Day 4:

• Modified code according to mentor's feedback
• Implemented null checking to prevent crashes when invalid picture or picture without GPS coordinates is selected. Tested and found that app does not crash when MediaWiki server not accessible.
• Attempting to add categories to a Set in preparation for week 4's task ( T119281). Encountering problems which are described at https://github.com/nicolas-raoul/apps-android-commons/pull/34

Day 5:

• Wrote blog post for Outreachy Week 3. Reading up on static Java classes and methods.
• Managed to get categorySet working, now have a collection of unique categories for next step.
• Managed to get app to display categories found alongside previous categories (that was quicker than expected, amazing what the rare night of good sleep does!). Works but needs to be tidied up. Pull request submitted for feedback.

Day 6:

• Tested app with real uploads (instead of in debugging state previously). Used pictures sent by Nicolas. Made minor fixes to app (put nearby categories on top of recent categories instead of below, implemented checking for null pages).
• Used GeoSetter to add GPS info to my existing pictures. Tested with these pictures, and also with pictures with no GPS info. Some examples of test results shown at T119281.

Week 4: Dec 28 - Jan 3
Task: T119281: Display the resulting categories to the user alongside the "previous categories" that are currently already displayed
Task: T122568: Implement recursive calls to the MediaWiki API until a certain number of categories are obtained
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/34 (merged)

Note: Will be traveling for 3 days from Dec 30 - Jan 1

Day 1:

• Final tests, versioning and tagging, wrote changelog. Released updated app on Google Play Store with mentor's approval.
• Pull request waiting for merge. Reading up on next week's task (T119282) in the meantime.
• Learnt more about how Commons geocoding works, found an issue that I discussed at T119281#1905714. Opened GitHub issue for it.

Day 2:

• Reading up on dictionaries/maps in Java, thinking about how to map an 'area' (which involves a range as well as latitude and longitude values) to a 'category'. Going through existing codebase to try and figure out how 'recently used' categories are cached in the app
• Meeting with Nicolas (Niedzielski advised us that he would be traveling during this period). Discussed caching task and decided that we will need to use a 3rd party library to map a 2-dimensional area with radius to a category.
• Determined that an additional task (T122568) is required before starting on caching. Will spend the remainder of this week on that task after I return from traveling.

Day 3:

• Arrived back from traveling. Made some modifications to last week's pull request according to mentor's feedback.
• Worked on additional task in separate branch. Managed to get the functionality up, submitted separate pull request.
• Discovered bug with this functionality (described at T122568 ). Working on it...

Day 4:

• Read up on QuadTree library recommended by mentor for next week's task. Read up on how to send multiple requests with Volley in order to fix yesterday's bug. This and this tutorial seem helpful.
• Last week's pull request merged.
• Meeting with Niedzielski, worked through some problems that I was having. Conclusions: (1) API code needs to be refactored, ShareActivity.java should supply the Listener instead. (2) For sending multiple API requests, RequestQueue should be refactored into a singleton pattern. (3) Radius/caching logic for next week's task should be in a separate class. (4) We both face similar issues when trying to import Maven dependencies in Android Studio. I will import the 3rd party library for next week's task manually first, and we will sort the issue out later (5) The 'for' loop in my current code is running infinite loops because it isn't extracting the current request's radius. There is actually no need for a 'for' loop, we just need to check if there are enough categories and if the radius is within limits.

Day 5:

• Trying to do the refactoring mentioned above (Listener in separate class, singleton pattern, changing the 'for' loop). Encountering problems that are described at T122568. Current state of code can be found at pull request #36.

Week 5: Jan 4 - Jan 10
Task: T119282: Cache categories, establishing a short-lived area->categories dictionary
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/37 (merged)

Day 1:

• Still trying to fix yesterday's bug. Reading up on passing objects and nested classes in Java. Posted on StackOverflow about my issue.
• Wrote blog post for Outreachy Week 4.
• Reading javadocs for Quadtree library that Nicolas recommended using for caching task.

Day 2:

• Came up with a rough plan for using the Quadtree library to store points along with the associated categories and to query an area for previous points' categories. Details at T119282.
• Trying to get basic functionality up. Had to copy over Quadtree source files manually due to Maven/AS issues that were discussed previously with Niedzielski.
• Managed to get app running QuadTree and displaying dummy categories (that were set manually) for a given point. Code in caching branch. Need to figure out a way to store QuadTree information persistently, as well as how to structure categories for real queries.

Day 3:

• Talked to Nicolas - no need to store information persistently, a very short-lived cache is fine.
• Fleshed out CacheController class, trying to store and access real categories. Seems to work but is tied to activity lifecycle, which is far too short since each new upload recreates the activity. Found solution - using a global singleton in CommonsApplication (which extends Application).
• Implemented conversion of distance in metres (to point) to coordinate offset range for searchWithin() method
• Found bug where my cache currently only stores and retrieves ONE nearby point + assoc. categories. Working on it...

Day 4:

• Talked to mentors about the problematic repeated calls task, it seems there might not be a need for it due to recently finding out that MW API returns the closest n results. Awaiting conclusion.
• Logic for searching cache vs API request calls done. However the cache does not seem to pass on categories to CategorizationFragment for display properly. Found solution - needed to flatten the list in cache, and needed to change how the list of categories in MwVolleyApi was initialized.
• Caching seems to work okay in manual testing. :) Pull request submitted.

Day 5:

• Imported Quadtree JAR that Nicolas created, instead of having source code in app. Incidentally learnt quite a bit about creating and importing JARs.
• Versioning, tagging, changelog, tests - released new version (1.5) to Google Play
• Wrote blog post for Outreachy Week 5
• Found invalid URI issue when testing app on real device. More details on StackOverflow post...

Day 6:

• Might have solved invalid URI on real device bug. Submitted pull request.
• Filled GitHub wiki with test results for using a 10,000m radius as discussed at T122568.
• Talking to mentors about which JAR to use for submission of caching task.

Week 6: Jan 11 - Jan 17
Task: T119283: Manual testing of app, bugfixes, release to Google Play. Recruit testers.
Task: T123103: Mid-term evaluation for "Easier categorization of pictures in Upload to Commons Android app"
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/38 (merged)

Day 1:

• Modified code for caching task according to mentor's feedback (pull request #37).
• After further testing found out that my proposed fix for real device bug does not work for emulator (so it is still device-dependent) - see T119283. Trying to figure out how to solve this problem.
• Modified code based on tutorial - this appears to work on both my emulator and real device.

Day 2:

• Meeting with both mentors. Discussed how to handle URI->filepath conversion bug, decided to use aFileChooser library. Also decided that I should release as soon as possible to get community feedback, and start on Phase 2 task after.
• Tried to import aFileChooser as a library into Android Studio. Failed due to Grade/non-Gradle module incompatibility. Unfortunately this also messed up the AS project settings, and I had to rebase, re-clone, re-import etc to get it working again.
• Project is back in working order. Copied over relevant getFilePath() code. Changed target API to latest (was 17 previously).
• Released update (v1.6) on Google Play.

Day 3:

• Tested v1.6 on my phone and friend's phone, appears to work as intended.
• Found and fixed minor bug pertaining to outdated links on About screen.
• Posted update on wikitech-l, mobile-l, commons-l and wikivoyage-l to try and recruit testers. Also posted on Commons village pump and Commons mobile app discussion.

Day 4:

• Followed up on a crash report asking for more details. Found out what the error was caused by, talking to mentors about the best way to fix it.
• Filled GitHub wiki for samples for T119284 .

Day 5:

• Wrote blog post for Outreachy Week 6.
• Tested and released v1.7 containing bugfix for Marshmallow users.
• Filled in some test results of Method A and B on GitHub wiki.

Phase 2

Week 7: Jan 18 - Jan 24
Task: T119284: Identify common patterns of category search near-misses. Search for an API that would allow "fuzzy" search
Task: T119285: Test the APIs for fuzzy search, start implementing if time allows
Task: T119286: Implement the fuzzy search
Results: https://github.com/nicolas-raoul/apps-android-commons/wiki/Fuzzy-category-search

Day 1:

• Completed test results for Method A and B on GitHub wiki. Submitted my conclusions at T119284 .
• Looked (unsuccessfully) for other APIs that could be used for the same purpose.

Day 2:

• Identified improvements that could be made to Phase 1's implementation. Will start on them after getting minimum viable product up for Phase 2.
• Meeting with mentors. Agreed on using Method A for Phase 2 implementation. Discussed outline of implementation and libraries needed for it.
• Drafted plan for implementation at T119286.

Day 3:

• Wrote dummy version of method A by itself to test the API request and XPath expression - it works, searching for 'pork' returns 'grilled pork' etc.
• Read up on AsyncTask and attempting to write method A in parallel with prefix method. Encountering issues described at T119286.

Day 4:

• Refactored existing CategoriesUpdater class into smaller methods, and wrote parallel method A calls. Current code at method-a branch.
• Trying to figure out how to store the categories found by both of the parallel tasks in a LinkedHashSet and have it all displayed together. (Unsuccessful) code at linked-hash-set branch.

Day 5:

• Worked further on LinkedHashSet branch, still encountering issues. More details at T119286.

Day 6:

• Learnt about using anonymous inner classes to override methods of a class with an instance.
• Further discussion with Niedzielski about how to structure the parallel Method A/prefix calls. My last branch got too convoluted so I made a fresh start with subtask-async branch.
• Still wrestling bugs with parallel Method A/Prefix calls. More details at T119286.
• Wrote blog post for Outreachy Week 7.

Week 8: Jan 25 - Jan 31
Task: T119286: Implement the fuzzy search
Task: T119287: Manual testing of 'fuzzy search' functionality in app
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/49 (merged)

Day 1:

• Modified code according to Niedzielski's feedback.
• Fuzzy search appears to work as intended now, according to manual tests. Pull request here.

Day 2:

• Meeting with mentors. Discussed this week's task and decided that we should push the next release as soon as possible to get feedback early on the fuzzy search feature.
• Working on tidying up code according to mentor's feedback.

Day 3:

• Finished modifying code according to feedback. Pull request merged.
• Tested build on real device successfully. Did versioning, tagging, changelog, and pushed v1.8 release to Google Play.

Day 4:

• Tested v1.8 release on my phone and partner's phone, works as intended as far as I can see.
• Updated app credits.

Day 5:

• Wrote blog post for Outreachy Week #8.
• Talking to mentors about where to go from here - decided that next on the priority list is fixing bug with duplicate categories (T125195).

Day 6:

• Fixed repeat categories bug (T125195). Submitted pull request.

Week 9: Feb 1 - Feb 7
Task: T125195: Fix bug with duplicate categories being shown by GPS + recent cats
Task: T119288: If not disabled in preferences, use GPS to find nearby categories even when coordinates are not embedded in pictures (optional)
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/54 (merged)

Day 1:

• Reading up on optional task (T119288).

Day 2:

• Meeting with mentor. Decided that we should proceed with the optional GPS task, probably via the 'automatic but enabled/disabled in preferences' route. Niedzielski recommends passive location as a potential implementation.
• Also decided that we should send a final post to the mailing lists in the last couple of weeks of the program, to announce the new features.

Day 3:

• Read up on LocationManager and various methods of getting current location.
• Implemented basic functionality for optional GPS task - gets last known location if image does not have coordinates, and displays categories for that location. Pull request submitted for feedback, but not ready for merge yet.

Day 4:

• Feedback received - I went down the wrong track previously, oops. LocationListeners are required.
• Read up on LocationListeners, followed these tutorials - 1, 2. Re-wrote location code.

Day 5:

• Read up on shared preferences, notably these tutorials - 1, 2.
• Implemented checking for preferences, updated pull request.
• Wrote blog post for Outreachy Week 9.

Week 10: Feb 8 - Feb 14
Task: T119288: If not disabled in preferences, use GPS to find nearby categories even when coordinates are not embedded in pictures (optional)
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/55 (merged)

Day 1:

• Modified code on pull request according to mentor's feedback.
• Reading up on requestSingleUpdate() and how to appropriately wait for a stable location - link1, link2

Day 2:

• Further modification of code to give the location time to stabilize before reading it.
• Refactored ShareActivity and GPSExtractor in the process.

Day 3:

• Attempted fix for a crash report.
• Not feeling well. :(

Day 4:

• Made final modifications to location code. Pull request with location code and crash fix merged.
• Pushed v1.9 of app to Google Play.

Day 5:

• Wrote blog post for Outreachy Week 10.
• Tested v1.9 release on my phone and partner's phone. Appears to work as intended.

Week 11: Feb 15 - Feb 21
Task: T119289: Release to Google Play, final tests and bugfixes
Task: T119290: Update app documentation to reflect new enhancements
Task: T126848: Wrap-up report for "Easier categorization of pictures in Upload to Commons Android app"
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/59 (merged)
Pull request: https://github.com/nicolas-raoul/apps-android-commons/pull/71 (merged)

Day 1:

• Meeting with mentors. Discussed how to fix a bug (crash due to null location), when to release announcement of new features on mailing lists, and what sort of documentation to do.
• Fixed aforementioned bug (pull request here).
• Pushed v1.10 containing the bugfix to Google Play.

Day 2:

• Released v1.11 with Polish language fix (written by Nicolas) to Google Play. Notified person who filed this bug report, waiting for response.
• Announced new features on wikitech-l, mobile-l, commons-l, and wikivoyage-l mailing lists, as well as on the Commons mobile app discussion page and the Commons village pump.

Day 3:

• Started draft of wrap-up report and blog post according to recommendations at T126848.

Day 4:

• Added Javadocs as part of documentation task. Submitted pull request.
• Released v1.12 with Javadocs and Nicolas's Polish language patch.

Day 5:

• Writing documentation for category suggestions (rationale, API used, sample request & response) on GitHub wiki.

Week 12: Feb 22 - Feb 28
Task: T119290: Update app documentation to reflect new enhancements
Task: T126848: Wrap-up report for "Easier categorization of pictures in Upload to Commons Android app"
GitHub doc: https://github.com/nicolas-raoul/apps-android-commons/wiki/Category-suggestions-(readme)
MediaWiki doc: https://www.mediawiki.org/wiki/Wikimedia_Apps/Commons
Commons doc: https://commons.wikimedia.org/wiki/Commons:Mobile_app

Day 1:

• Finished writing documentation on GitHub wiki.
• Modified GitHub wiki according to mentor's feedback.
• Reading the mediawiki.org Commons app documentation and considering how to update it.

Day 2:

• Meeting with mentors, discussed how to modify the mediawiki.org page. Decided that outdated sections should be removed and links updated.
• Updated the mediawiki.org Commons app documentation.
• Also updated the Commons documentation.

Day 3:

• Working on wrap-up report (T126848).

Day 4:

• Completed wrap-up report and wrap-up blog post.
• I intend to apply for an IEG in order to do further work on this app. Spoke to Quim and Nicolas about it. Working on IEG proposal draft.

Day 5:

• Filled in Outreachy project page. Filled in relevant links for T128052: End term evaluation for 'Upload to Commons Android app' project