Context:

Global Data and Insights plans to release the first public-facing dashboard to enable movement organizers and partners to map diversity, equity & inclusion among movement-wide programs and spaces. As part of our DaaS OKRs we need to enable a solution that would allow them to create and share public dashboards powered by both internal and external data.

Proposed Solution:

Integrate GDIs Data workflow into the Data and Analytics Platform and provide a path for them to upload, blend and share data and dashboards.

Scope:

The initial MVP project will focus on the workflows needed to integrate external data into our datastack and will be designed to be iterated upon on automation the second phase. The second Automation Phase (which is out of scope for now) will focus on the automating and handing over the pipelines delivered during the MVP to allow for self service updates and modifications to be made to the pipelines.

The MVP will deliver:

• A mechanism for Manual ingestion of input datasets (in expected format)
• Creation of tables in our data infrastructure based on input datasets
• Manual transformation jobs in spark
• Manual data sync of output to public access
• Data Visualisation Framework to deliver dashboard