Page MenuHomePhabricator

Create regularly updated maps of COVID-19 outbreak
Open, Needs TriagePublic

Description

Given the current challenges of using Graph-based maps (T248707, etc), let's build some simple SVG maps that can be updated on a daily basis and embedded within our COVID-19 articles. To do this we will need to have a bot that downloads the data and updates SVG maps on Commons on a daily basis. Here are some resources that can be used:

World blank SVG map: https://commons.wikimedia.org/wiki/File:BlankMap-World.svg
World data: https://github.com/CSSEGISandData/COVID-19

U.S. blank SVG map: https://commons.wikimedia.org/wiki/File:Blank_USA,_w_territories.svg
U.S. data: https://covidtracking.com/api/ or https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_daily_reports_us

Africa blank SVG map: https://commons.wikimedia.org/wiki/File:BlankMap-Africa.svg

To color-code the maps, you only need to add CSS code in the preambles, e.g.:

.ca, .ge, .gu, .mp, .uy, .za {fill:#0d00f2;}

We should re-upload new versions each day rather than creating new files for each day. That will allow the communities to just embed the maps in their articles and then leave them alone.

This could effectively replace all the maps that are currently being updated manually, such as:

Event Timeline

kaldari updated the task description. (Show Details)

@kaldari this seems like a solid plan. Twiddling the CSS seems much more feasible than generating SVG from scratch (scary!).

I think the main components of any program would be:

  • Download data for each locale
  • Build SVG files for each locale
  • Upload the SVG files for each locale

These seem like decomposable steps that multiple people could work on. I'm definitely into helping with it.

I've already started a bot for COVID-19 data on Commons, and I think this would fit under its mandate. I think the next step for this bot would be to do a practice run, so it's probably a good time to be writing code.

I can get started on this in NodeJS on Github, and we can move it to Gerrit if/when we're ready.

I think this is a lovely way to unblock the current situation, and sidestep problems with the graph extension and graphoid. For my part, I am writing an RFC for graphoid right now. I want to summarize here quickly because it may be relevant:

A new graph rendering service will take in data, the name of a rendering strategy (Vega 2, Vega 5, "bar-chart", Vega Lite, etc.) and output an image. We can use this more flexible service from bots and a re-imagined graph extension down the line. Details here: T249419

So, there's a first version of the bot here: https://github.com/evanp/covid19databot

The Commons bot is here: https://commons.wikimedia.org/wiki/User:Covid19DataBot

Right now, it's using Johns Hopkins data, which is probably fine for a map: https://commons.wikimedia.org/wiki/File:Covid19DataBot-Case-Data-World.svg

@eprodromou - Nice work! Has there been any interest on Commons for your bot to take over any of the existing SVG maps? I'd be happy to help with those discussions if you like.

Ideally, this would use the same data as the tables in the articles, but those are collected manually by editors and aren't machine-readable. I filed T250065: Store tabular data in a format that's machine-readable and can be shared between wikis for discussions about that.

@Tgr I wrote a jupyter notebook to take the JHU data and make the state maps. Would be easy to make different maps (#cases etc). My current challenge is that I can't mass upload/overwrite files easily (I don't have the time to dig into more advanced solutions for this right now, and all tools seem to require Java Runtime which I can't get to work on my mac), so the maps are not currently updated.

I ran into a data problem updating the Africa map (https://commons.wikimedia.org/wiki/File:COVID-19_Outbreak_Africa_Map.svg): Several of the territories in Africa aren't countries, but are overseas territories of European countries (thanks colonialism!). Reunion and Mayotte are included in the state level data for France, but Madeira, the Canary Islands, and Saint Helena, Ascension and Tristan da Cunha are not included in the data at all and have to be updated by hand :(

(quick note: I also figured out what i did wrong in the uploading part, and fixed some other stuff. I'm now uploading/updating US state maps daily.)

@Effeietsanders - That's awesome. Could you specify which maps exactly you are updating?

@kaldari it's the maps in this series: https://commons.wikimedia.org/wiki/File:COVID-19_Prevalence_in_Arkansas_by_county.svg .

It uses the JHU time series and census estimate data. Ideally I'd also make gif's for the prevalence development per state, but have too much on my plate for this week. Happy to share the jupyter notebook if it helps anyone.

Missouri is the only state missing, due to Kansas City being weird (I think I figured out a solution - to be implemented)

Removing "Product Doing" after 3 years of no updates from WMF. Should this realistically be declined?