From the analytics perspective, I think there's three things we need to consider:

1. What's the basis of our analyses?
2. How often do we run them?
3. Do they involve complex processing in order to understand conditions?

Regarding the first point, the larger pieces of analyses that we've done so far are done on a per-wiki basis, with random treatment group assignment on a per-user basis (within each wiki), run for some specific period of time. We've also required wikis to meet certain requirements (e.g. translations of messages, definition of task categories) in order to get our features. The way I see the work we're doing around scaling means that it's going to be impossible to require wikis to meet certain requirements to be part of experiments (perhaps with the exception of five main wikis?) We'll instead have to either gather data to understand the conditions and control for those in our analyses, or see if there are ways to change our unit of analysis (for example we can see "number of edits made" as a second-order result and instead focus on users completing a recommended task if they clicked on it).

We currently don't run analyses very often. This means that having access to configuration data in the Data Lake isn't critical, and taking the time to write some Python to grab and parse JSON off wikis won't slow us down (it should also be a "write once, reuse" process, and the number of wikis is known and generally a low N). I also suspect that configuration changes won't happen very often, and lastly that the configuration itself isn't going to be complex. In other words, translating the configuration settings to some kind of "table structure" that we can join with other data is going to be relatively easy.

The third point is where my main concern is. I think one example here is the mentorship module, which as far as I know isn't shown to users unless there are mentors in the mentor list. This means that the "mentor module available" condition requires combining the JSON configuration and the mentor list to figure out. At the moment, I don't think we have many of these "system-derived configurations", but it's something to keep in mind together with the first point on the list, because if we get more of them we might switch our basis of analysis towards "what did users who saw intervention X do?" In the case of the mentorship module, that might mean that we rely on the HomepageModule data and analyze behavior of users who actually saw the mentorship module, taking away the need for understanding the configuration altogether. Or we might find that we then need to log configuration change events so we can keep track of them.

In summary: I think that we currently will be fine with grabbing the on-wiki JSON configuration when running analyses. If the conditions or our needs change, then we can revisit this decision because as mentioned above what we need can depend on how we do our analyses.

I'm tagging @MMiller_WMF as well, so he can review and follow up if needed.