Who can Ops spare for a 55 min meeting next week to help us flesh this out? @fgiunchedi or @akosiaris, are either of you available?

Another thing to throw into the mix is the non-production APIs that the dashboard interfaces with. The Wiki Ed version currently uses these three:

• http://tools.wmflabs.org/wikiedudashboard for pulling lots of data from the replica database
  • example: http://tools.wmflabs.org/wikiedudashboard/users.php?user_ids[0]=012345&user_ids[1]=678910&training_page_id=57
  • source: https://github.com/WikiEducationFoundation/WikiEduDashboardTools

• ores.wmflabs.org for wp10 scores (would need to be disabled for most languages where scores are not available, could probably be disabled altogether for WMF dashboard)
  • example: http://ores.wmflabs.org/scores/enwiki/wp10/263511535/

• http://tools.wmflabs.org/eranbot/plagiabot/api.py for plagiarism checking (could probably be disabled for WMF dashboard)
  • example: http://tools.wmflabs.org/eranbot/plagiabot/api.py?action=suspected_diffs
  • source: https://github.com/valhallasw/plagiabot

Minor note, we could get rid of the custom APIs with direct database access, T127404 (and another task for ORES), then disable the plagiabot as ragesoss suggested.

In T127390#2042557, @dduvall wrote:

Who can Ops spare for a 55 min meeting next week to help us flesh this out? @fgiunchedi or @akosiaris, are either of you available?

Sorry, I will not be around next week. Maybe the week after ?

Question, I assume the Program Dashboard Rails application is this http://wikiedu.org/. Am I correct ?

@dduvall the week after would work for me too, can you give us more context on what Program Dashboard is and what it will do, where's the source, etc? Personally it is the first time I've heard of it

In T127390#2043493, @fgiunchedi wrote:

@dduvall the week after would work for me too, can you give us more context on what Program Dashboard is and what it will do, where's the source, etc? Personally it is the first time I've heard of it

That works! I'll go ahead and schedule something.

I'll try to give you the gist of the project; I'm also pretty new to it. :) Check out the GitHub project README as well.

It's an upstream project from the Wiki Education Foundation that was developed to help educators integrate Wikimedia contribution into classroom curricula. We—a skunkworks-y team on temporary loan from our usual depts since CE lacks enough tech resources—are working in collaboration with WikiEd this month to help refactor the dashboard for more generalized use among other Wikimedia teams/projects, building in multi wiki and multi language support, and hosting an instance of the application under WMF resources.

The dashboard will serve as a replacement for the EducationProgram extension which has languished in security and support over the years and we're hoping that it will be more maintainable and secure over the long term since it will have both a highly invested/talented/dedicated upstream and hopefully some direct support from WMF in the near future.

I went ahead and scheduled a meeting for March 1st at 1900 UTC. @akosiaris, that seems especially late for you and we probably just need one of y'all opsen to help, so no worries if you can't make it. Alternatively, I could bump my meeting at 1600 UTC if that's not too early for @csteipp.

1600 UTC is fine for me.

Great. I've moved it to 1600 UTC to better accommodate ops.

Thanks again for meeting today, everyone. The general summary of this meeting is that in order to move forward with a full security review, we'll need a firm maintenance commitment from Ops and PC&L. Barring any changes in the annual plan that would grant the latter an actual tech budget, we should move forward with the assumption that the dashboard will continue to run in labs and that we should work to improve reliability/performance with a shared read/write application database and possibly a Redis instance for the Rails cache—ToolLabs was also mentioned as an option that could provide more scalability atop the Labs infrastructure.

Below are the notes from the etherpad:

Security review

• subscribe to gem CVE
• Rails CVE
• oauth login
• should run on separate domain
• would want a commitment from ops before going forward with security review

ToolLabs

• have we considered Tool Labs?
• is it stable? performant?
  • Filippo will look into questions of HA, replication, etc.
  • we should also talk to Yuvi

Ganeti

• • separate VM but still on prod network
  • means we'd still have to endure the same security review, etc.
• we don't need direct access to wiki DBs, etc.
• more isolation the better
• est 10,000 users in the first year

Ops concerns

• what are security concerns?
• where to put it?
• what kind of access does it need?
  • doesn't access MySQL replica, but it could/should to relieve performance bottleneck
  • would be read-only
• how often does it read/write to MySQL
  • lots of reads/writes
• current bottleneck is service that wrap DB replica

TODO

• compile list of gems that can be satisfied via Debian packages and which might require native compilation
• Filippo to see if we can have a read/write shared application database in Labs (not on our own instances)
• talk with Yuvi about whether ToolLabs might be a good fit
• Dan to profile some application requests to see what kind of read/write DB traffic we should expect

2c. For one reason or another, tools on toollabs may be occasionally down.
It's more or less OK when this means I won't be able to check my edit count for the day, it's definitely not a good circumstance for a tool which is meant to be uses in courses and classes where people can't just say "Well, the website doesn't work, I guess that means I'll see you all tomorrow".

In T127390#2076008, @Elitre wrote:

2c. For one reason or another, tools on toollabs may be occasionally down.
It's more or less OK when this means I won't be able to check my edit count for the day, it's definitely not a good circumstance for a tool which is meant to be uses in courses and classes where people can't just say "Well, the website doesn't work, I guess that means I'll see you all tomorrow".

Tool Labs stability is something we discussed briefly in the meeting but I'll make a point of asking for more information regarding uptime trends, etc., when talking with @yuvipanda. Now that we've some level of dashboard provisioning in ops/puppet, I'm leaning towards just going forward with bare Labs instances for now. Of course, that also hinges on the availability of a stable database platform. If we can't get that without Tool Labs, the latter may still be worth looking into barring uptime concerns.