Change Details

### Current Situation The tool uses a sqlite file committed to the git repository. To keep data persistence between redeployments, we actually moved to a copy of the database file outside of the repository hosted on Toolforge. At this point, the development is mature enough to move to a SQL database. ### Goal: Using Toolforge's database system, create a database, migrate structure and data of existing sqlite database file, point the application at the new database, and run database checks before redeploying. ### Unanswered Questions: [ ] Unclear what is needed or possible in terms of machine account for the application? ### Acceptance Criteria [ ] Create Database using Toolforge's database system [ ] Use alembic migrations to match the structure required by the application [ ] Migrate data from sqlite file hosted on toolforge to new database [ ] Update production settings to point to new database [ ] Update Procfile to run database checks on database before deploying application ### Technical Guidance * [[ https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases | Toolforge Database documentation ]]

### Current Situation The tool uses a sqlite file committed to the git repository. To keep data persistence between redeployments, we actually moved to a copy of the database file outside of the repository hosted on Toolforge. At this point, the development is mature enough to move to a SQL database. ### Goal: Using Toolforge's database system, create a database, migrate structure and data of existing sqlite database file, point the application at the new database, and run database checks before redeploying. ### Unanswered Questions: [ ] Unclear what is needed or possible in terms of machine account for the application? ### Acceptance Criteria [ ] Create Database using Toolforge's database system [ ] Use alembic migrations to match the structure required by the application [ ] Migrate data from sqlite file hosted on toolforge to new database [ ] Update production settings to point to new database [ ] Update Procfile to run database checks on database before deploying application ### Technical Guidance * [[ https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases | Toolforge Database documentation ]] * Alembic migrations should work without tweaking, but migrating //data// is likely to need a one-time specialized script. Suggest using [[ https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_sql.html | pandas' `to_sql` method ]], if done in Python.

### Current Situation The tool uses a sqlite file committed to the git repository. To keep data persistence between redeployments, we actually moved to a copy of the database file outside of the repository hosted on Toolforge. At this point, the development is mature enough to move to a SQL database. ### Goal: Using Toolforge's database system, create a database, migrate structure and data of existing sqlite database file, point the application at the new database, and run database checks before redeploying. ### Unanswered Questions: [ ] Unclear what is needed or possible in terms of machine account for the application? ### Acceptance Criteria [ ] Create Database using Toolforge's database system [ ] Use alembic migrations to match the structure required by the application [ ] Migrate data from sqlite file hosted on toolforge to new database [ ] Update production settings to point to new database [ ] Update Procfile to run database checks on database before deploying application ### Technical Guidance * [[ https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases | Toolforge Database documentation ]] * Alembic migrations should work without tweaking, but migrating //data// is likely to need a one-time specialized script. Suggest using [[ https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_sql.html | pandas' `to_sql` method ]], if done in Python.