Page MenuHomePhabricator

Develop a web app for editing Toolhub records
Closed, ResolvedPublic

Description

Tracking task for Outreachy 25: Develop a web app for editing Toolhub records

TODO: Add description and subtasks

Related Objects

StatusSubtypeAssignedTask
ResolvedNicoleLBee
ResolvedNone
ResolvedNone
ResolvedNone
ResolvedNicoleLBee
ResolvedHannah_Njoroge
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedHannah_Njoroge
ResolvedHannah_Njoroge
ResolvedNicoleLBee
ResolvedDamilare
ResolvedHannah_Njoroge
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedHannah_Njoroge
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNone
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
InvalidNone
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedHannah_Njoroge
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedNone
ResolvedNicoleLBee
ResolvedBUG REPORTNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee
ResolvedHannah_Njoroge
ResolvedBUG REPORTNicoleLBee
DuplicateNicoleLBee
DuplicateNicoleLBee
ResolvedNicoleLBee
ResolvedNicoleLBee

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@NicoleLBee @Hannah_Njoroge Hello! As it's been a few weeks since the internship started, I am asking all interns to share a few updates (in 3-4 sentences) on their project progress in a comment on the relevant Phabricator task. I'd encourage you both to do the same. For other reminders, please see my message on Zulip. cc @Slst2020 @Damilare

A summary of the work so far:

In the first weeks of the internship, we settled on a tech stack (JavaScript/Vue in the front-end, Python/Flask in the backend) and got the project scaffolding for the front-end up and running using Vite. In order to maintain a standardized development environment, we containerized the front-end using Docker.

As we wanted to adhere as closely as possible to Toolhub's front end, we added Vuetify as a UI library, then began building out the pages (though each individual "page" is actually a component that's being loaded into the main Vue app). By the end of the week we hope to have the front-end more or less complete and from next week will be moving to the backend.

We'll be using our contribution period work and the work we've done in the front-end to inform the design of the API.

Each of us has a one-on-one bi-weekly meeting with each of our mentors @Slst2020 and @Damilare. We also have a bi-weekly team sync meeting with both of our mentors.

On 16 December we had a backend live coding session with @Slst2020 where we:
set up the app and got it running locally.
learned about Redis, Docker containers, and celery and how we would apply them in our project

On 21 December we had a live coding session with @Damilare where we:
set up the front end in docker and got to understand more about docker containers

Today we had a meeting with both of our mentors @Damilare and @Slst2020.
We have discussed how we can realign our projects and timelines.
We have agreed that we will be having a team sync meeting every week instead of bi-weekly.
We also discussed how we are going to work on the backend and @Slst2020 will be inviting us to a document where we will add all the questions we have concerning our expectations about the backend. This will be in preparation for the live coding session on 16/01/2023.

On the issue of our first deployment, we have agreed that our first frontend deployment on Toolforge will be in Week 7/week 8.
For the backend, we will deploy in week 10 also in Toolforge.

We had agreed in exploring cloud VPN for our backend deployment but due to various factors, we have agreed that we will also deploy the backend in Toolforge.

We are also to write an email with specifications of the members of the Wikimedia Foundation team that we would like to meet and have a chat with.

Today we had another team meeting to discuss how to get the backend side of the project underway. After addressing a few questions about the frequency of database updates and the interaction between Toolhunt's front-end, back-end, and database, and between Toolhunt and Toolhub, we outlined several key tasks that I have added to Phabricator:

T327077: Initial project setup, using Poetry as an installer and dependency management tool, and Docker for containerization.

T327101: Begin designing the database

T327103: Develop API specification

Following the team meeting on Monday and further discussion with @Slst2020, I've gotten to work on T327077 and its subtasks.

So far I've managed to add Poetry to the project and successfully containerize it; my next step is to refactor the existing code to bring it in line with an example application we worked on a few weeks ago, and then to get another container up and running for our database.

This week I've gotten the MariaDB container running and connected to the backend application (with much help from @Slst2020). See T327077 for details.

I've added Flask-Smorest and some test routes (see Pull Request #11 for details on the current status), and am working on documentation (T328002). My next step will be to tackle T327101 and T327103: developing the database tables and API endpoints.

Yesterday we had a workshop where we talked about Effective Time Management. We talked more about procrastination and how to avoid it. In particular, I was reminded about the Pomodoro Technique and always thinking about the outcome before responding to a distraction. We were also taught about the breathing technique where if a distraction calls you are required to pause and inhale on the count of 4, wait on the count of 4, and exhale on the count of 4. This as Wendy said would help one get out of the flight or fight mode and therefore make an informed response. It was a very informative talk and I really enjoyed it. Looking forward to the next Workshop.

Today we had our team sync and we discussed the progress of our project both on the frontend and the backend.
Nicole presented to us the endpoint that she has been working on.
We scheduled a live session to sync up on the API endpoint designs for the coming Monday and a Deployment with Toolforge live session to be on Friday(tomorrow)
We agreed to push the OAuth live session and discussion to be after we are done with API integration.
We also did some code reviews together on the frontend part. We discussed showcasing our project to the WMF team after we are done and also doing a Tech Engagement team presentation + small edit-a-thon.

Today we had another team meeting to discuss the API specification (see T327103, which has been updated as a result of the discussion). We also talked a little bit about data storage and the contents of some of the tables (see T327101), and touched on the top of task prioritization. I've opened a new task, T328335, to continue the discussion.

A few notes from today's team meeting:

We discussed what we had been working on during the past week.

I demoed the latest changes to our API and walked through the process of starting the database and inserting the mock data.

Hannah talked about how she had been reading and doing tutorials in order to increase her understanding of the backend code. She'd also spent time reviewing pull request 11, and asked for advice about Pinia use on the frontend.

Damilare recommended waiting on Pinia until the API calls for the frontend were in place; a discussion about state management would make more sense once we had a state to manage.

Damilare talked about CI through GitHub webhooks and walked us through the process of setting that up.

My primary goal for this week was to sketch out a workflow for getting data from Toolhub into our Toolhunt db. My resulting mental contortions are outlined in T329029 and were given expression here: https://github.com/wikimedia/toolhunt/blob/database_entry_testing/api/jobs/insertion.py

I also made some modifications to our API endpoints. See T327103 and pull request #18

At yesterday's team meeting we discussed the annotations fields and their expected input values (see T328335) and decided to implement only a subset of the available fields at the start. I expanded the database tables and the API endpoints accordingly. See pull request #20

Current tasks:

  • add CI to the backend (will wait on this until my current pull requests are resolved)
  • add a shell context processor to __init__.py
  • tidy up the project file structure (will wait on this for the same reason as the CI)
  • implement OAuth properly; I thought I had it working, but I was wrong

Backend status update:

OAuth is still a WIP but @Damilare estimates that we are 60% of the way there. We'll be discussing the next steps at our one-on-one tomorrow. At the moment, the front- and backend halves of the app are fully linked and basic authentication is working. (See https://github.com/wikimedia/toolhunt/pull/28 for the backend component and https://github.com/wikimedia/toolhunt-ui/pull/28 for the frontend; also T327856)

I've written a route to make /PUT requests which takes advantage of this; although the UI side isn't there yet, we now have an interface by which we can use the tool to make updates to Toolhub! (See https://github.com/wikimedia/toolhunt/pull/24 and T327103)

I've also added a date limiting parameter to the /high-scores endpoint which will allow us to populate the "Top Users in the Past 30 Days" table; it is currently under review. (See https://github.com/wikimedia/toolhunt/pull/27)

I'm also working on the backend refactoring tasks outlined by Slavina (T329621) and still need to get around to adding CI.

Here's this week's "State of the Backend" report:

We confirmed that the OAuth was working; Flask handled the transition of session tokens seamlessly.

API developments (T327103):

General improvements:

  • Completed a number of "housekeeping" tasks (see T329621 for details)
  • Implemented local CI (see T330128 and T330135)

Ongoing tasks:

  • Improve current code (see T329625)

My top priorities for next week:

  • Implement Redis and deploy to Toolforge!
NicoleLBee changed the status of subtask T331414: [backend][jobs] Improve populate_db.py from Open to In Progress.

The backend is now running with "production" values -- i.e., it's using data from our production database and getting information from the Toolhub servers.

On the frontend, there's some weirdness with the file structure that came to light while I was trying to pull the latest updates to live, which @Damilare and I will investigate tomorrow. But otherwise, I think we're done with the key parts.

NicoleLBee claimed this task.

We've been requested to close out pending Outreachy-specific tasks and now that the authentication issues have been resolved (and the other issues I was experiencing seem to have gone... dormant??), it seems like an appropriate time to do so.

Although there are still quite a few things left to polish, and yet other things on my own "bucket-list," it seems to me that we have successfully "develop[ed] a web app for editing Toolhub records." 🎉