Is there documentation for "enterprise mode"?

@Neil_P._Quinn_WMF

This task does not belong in Design Research's backlog. However, there is potentially valuable information in the task for you and other Product Analysts to implement A/B testing.

Could you please help assign a more appropriate project to this task?

Many thanks!

@Neil_P._Quinn_WMF many thanks for the quick response :)

@ggellerman, thanks for pulling me in! I find this particularly interesting because I've actually used Optimizely in a past job. One of its major strengths is its visual editor, which allows you to do quick-and-dirty tests of interface changes without the manual implementation that Facebook PlanOUT seems to require. Obviously, I'd prefer to use open source software, but this may be one of the cases where proprietary is the lesser of two evils. I'll put this project on my list.

@Neil_P._Quinn_WMF: Can I pass along some meeting minutes? Some Fundraising tech and analytics people went over to their office (a block away!) and chatted about quite a bit. I should probably just publish the notes here, I don't think there's anything private in there. The punchline, however, is that they can offer us an integration in which we control our own traffic and data.

@awight, please do! I agree that a comment here would be preferable, but if there is something private, Office wiki would work.

@awight, do you have those meeting notes? I'm interested to read them.

@Neil_P._Quinn_WMF
Sorry, I managed to locate the pen and paper meeting minutes last week, but won't be able to enter the contents here until the week of July 22nd, cos I'm currently out of town.

I do remember that the bottom line was, they would be happy to work with us, and they offer an enterprise model in which we host all components on our own servers, making it possible to comply with our privacy policy.

@awight, No worries, I probably won't have much time before then anyway.

Whew, finally reunited with my scraps of paper.

Notes from our 2014-12-5 meeting with Optimizely, sorry enough time has passed that I only partly understand what I've written here :-/

• Configurable rules to determine whether a person becomes part of a test (audience).
• Each test has a goal: clicks, pageviews, or a custom event. Goals can be split by segment.
• Campaign is an external URL parameter.
• They have an enterprise mode where we can host the data gathering snippet and (I believe) the data collection backend.
• Optimizely staff have published papers we should read, discussing ways to watch the result trends and make a conclusion before hitting rigid significance thresholds. For example, (client name redacted) shifts their p-value over the course of an experiment.
• Custom js can be used to isolate experiment and variations, and do scheduling. start and stop time, or repeated time of day experiments.
• Experiment API does all the UI magic.
• Experiments can be managed via code (was beta quality at the time): http://developers.optimizely.com/rest/introduction/index.html
• We can manage experiments to surface variations which are doing well.

@awight, thanks! I'll take a look at this. In general, what's Fundraising's take on Optimizely? Are y'all still evaluating it, or planning to evaluate it further at some point?

Great question. Adding our project to tag this for further investigation. @MeganHernandez_WMF has been in touch with Optimizely, and a group of Fundraising Tech devs plus @ellery were present at the meeting those notes are from. We're not actively evaluating it, but I'm certainly interested in either pursuing their SaaS or borrowing some of their better ideas.

Also CC'ing @AndyRussG here, he's been doing some thinking about how to isolate the A/B testing component from CentralNotice into a reusable library.