Page MenuHomePhabricator

Automate project start through bots
Closed, ResolvedPublic

Description

Each year we spend a bunch of hours creating all the new project pages, subpages, categories etc. This is really a waste of time during the busiest part of the year and should be automatized.

We should create a bot that, based on a predefined list of project names in a spreadsheet, create all the pages with all the information, categories etc. This should ideally be done before December 2018.

This expanded to also include "year pages" i.e. pages that are created at the start of a new year.

Bot repo: https://github.com/Wikimedia-Sverige/project-start

Event Timeline

Jopparn updated the task description. (Show Details)
Jopparn set the point value for this task to 40.
Jopparn renamed this task from Automate project start to Automate project start through bots.Oct 29 2018, 9:28 AM

This page should be useful as guidance: https://se.wikimedia.org/wiki/Projektsidor

For creating Phab-projects, it looks like Conduit's project.edit could be used with transactions:

{
  "0": {
    "type": "parent",
    "value": "PHID-PROJ-ft7vbzykjs52i5vguajb"
  },
  "1": {
    "type": "name",
    "value": "WMSE-Project2"
  }
}

The PHID (for parent project) can be found using project.search with constraints:

{
  "ids": [
    11
  ]
}

The bot should update Mall:Aktuella projekt:

  1. Rename to keep as archive for the previous year (e.g. Mall:Projekt 2018)
  2. Change/add template to the project pages for the previous year
  3. Replace projects in Mall:Aktuella projekt with the projects for the current year

Added a few more columns to the spreadsheet for values that can be added to the project data template:

  • Project start
  • Project end
  • Financier
  • Budget

Initial version has been pushed to Github: https://github.com/Wikimedia-Sverige/project-start.

Currently, it creates the project page and subpages on the wiki and populates them with information from the project spreadsheet. I copied a few of the templates used to create the pages and added the new in parameters to them. To keep it consistent, all the templates now have the prefix "-sida". I didn't want to change the original templates as I didn't know if we still plan to use them and if I've broken some magic with my modifications. It should be easy to remove/merge if we want to.

@Jopparn, @AxelPettersson_WMSE: can you take a look at the test project page I've created with the bot and let me know if anything is missing? It uses the test project in the project spreadsheet as in data. The Phab link isn't working yet since I haven't implemented the project creation here yet. I also haven't done anything with the categories yet.

The bot can now add projects to Phab too, with English name and description. Projects will be added as subprojects to WMSE. Project name and id are added to the wiki page for the Phab link.

Before the bot will be able to do anything with Phab, an account will need to be created. A request for that has been made: T211351.

@Jopparn: I noticed that some projects have superprojects ("Överprojekt") in the spreadsheet. It looks like they are supposed to be subpages of the superprojects on the wiki. Can I get a confirmation that this is how it's intended to work? Do we want to have the same structure on Phabricator?

The bot should update Mall:Aktuella projekt:

  1. Rename to keep as archive for the previous year (e.g. Mall:Projekt 2018)
  2. Change/add template to the project pages for the previous year
  3. Replace projects in Mall:Aktuella projekt with the projects for the current year

I looked at how Mall:Aktuella projekt is used and it looks like it's only actually used on year specific pages. If that's the intent, we should be fine just using one template for each year. This would simplify the process to only one step: creating the new template.

Changing where the template is currently used (to use a 2018 template instead) would be easily done manually as it's only used on a handful of pages.

@Jopparn: I noticed that some projects have superprojects ("Överprojekt") in the spreadsheet. It looks like they are supposed to be subpages of the superprojects on the wiki. Can I get a confirmation that this is how it's intended to work? Do we want to have the same structure on Phabricator?

Yes, they are subpages of the superprojects on the wiki. They are the externally financed 'consultancy' projects we have with different GLAM partners.

The bot should update Mall:Aktuella projekt:

  1. Rename to keep as archive for the previous year (e.g. Mall:Projekt 2018)
  2. Change/add template to the project pages for the previous year
  3. Replace projects in Mall:Aktuella projekt with the projects for the current year

I looked at how Mall:Aktuella projekt is used and it looks like it's only actually used on year specific pages. If that's the intent, we should be fine just using one template for each year. This would simplify the process to only one step: creating the new template.

Changing where the template is currently used (to use a 2018 template instead) would be easily done manually as it's only used on a handful of pages.

That sounds like a reasonable change to me.

When I did a dry run I noted that line breaks are converted to spaces when downloading as TSV. There doesn't seem to be any way for Google Sheets to add the required quotation marks for multi line strings. A solution to this is to use LibreOffice Calc to convert to TSV, which does this by default. The chain is thus Google Sheets (download as .ods) -> LibreOffice Calc (save as CSV, but with tabs delimiters).

I just realized that you can't create/import workboards through the API, so that will have to be done manually.