Investigate adding user-friendly testing functionality to Reportupdater
Open, LowPublic

Description

To aid our migration to Reportupdater (T150915 / 325870), I developed an R-based utility (attached as test.R) which can be used to do a dry run (without affecting any of the datasets) of all the found config.yaml's and then outputs a user-friendly Markdown-formatted log file (attached along with the HTML version that includes a table of contents via pandoc). The generated log shows explicit commands and queries that are used, the output, and the time it took to execute report. At the end, there are some summary statistics.

We've been using this utility to check that the output is what we expect and to see which reports error and why. I don't know if this would be nice functionality for Reportupdater to have built-in but @Nuria suggested I make a ticket so yinz can take a proper look and maybe replicate some of it in Python :)

Note: Our nearly-50 reports (more when we enable forecasting modules) are split into modules and our config.yaml's include a description for each report:

  • modules
    • metrics
      • search
        • config.yaml
        • queries & scripts
      • portal
        • config.yaml
        • queries & scripts
      • etc.
    • forecasts
      • search
        • config.yaml
        • queries & scripts
      • wdqs
        • config.yaml
        • queries & scripts
      • etc.
      • shared forecasting & models R code

Latest attachments:

Previous attachments:

mpopov created this task.Jan 27 2017, 8:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 27 2017, 8:18 PM
Nuria moved this task from Incoming to Q4 (April 2017) on the Analytics board.Jan 30 2017, 5:16 PM
mpopov edited the task description. (Show Details)Feb 27 2017, 7:30 PM
Nuria triaged this task as "Low" priority.Mon, Mar 27, 3:56 PM