Page MenuHomePhabricator

Speed up the theory-testing pipeline
Closed, ResolvedPublic

Description

  • Background: There are many promising theories about how the zero-results rate can be reduced, and there will probably be additional theories as the project progresses.
  • Summary: Streamline the deployment processes for CirrusSearch/Elastic changes.
  • Benefit: Increase the quantity of theories that can be tested in Q1

Event Timeline

Jdouglas updated the task description. (Show Details)
Jdouglas raised the priority of this task from to High.
Jdouglas added a subscriber: Jdouglas.
Restricted Application added a project: Discovery. · View Herald TranscriptJul 8 2015, 4:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I would like @Manybubbles, @EBernhardson, @Jdouglas and @dcausse to get their heads together and make a list of the 10 things that impede them the most from moving rapidly on changes to influence the zero results rate. That list will form the basis of the subtasks to attack this problem.

Jdouglas added a comment.EditedJul 8 2015, 10:26 PM

get their heads together and make a list of the 10 things that impede them the most

A few quick ones for further discussion:

Impedance: Testing a new theory (i.e. deploying a change and scrutinizing it) requires real time to pass for logs to accumulate sufficient data to analyze.
Suggested solution: Build a mechanism for testing code against historical data, and finding out how it changes search results.

Impedance: Deployment of new code is tricky business.
Suggested solution: Automate things that are automateable. I'll need help coming up with specifics, but the ultimate goal (even if we can't get all the way there) is that when a change is +2'ed, the deployment happens automatically.

Impedance: Unclear prioritization of work causes things to sit around in code review indefintely.
Suggested solution: Within the team, the PO has final say on what is top priority. Between teams, the {TBD, maybe engineering director} has this responsibility.

dcausse added a subscriber: TJones.Jul 17 2015, 8:22 AM

After having a talk with @TJones we thought it could be interesting to see if some NLP algorithms could help to find results. Today it's not clear to me how and where we can do this kind of process.

Impedance: We need data to do this, some of this data like search request logs can't be downloaded locally.
Suggested solution: Allow developers to download some logs and experiment locally or have access to a machine in the cluster that we can stress (and install the tools we want) without perturbing production servers.

Impedance: Architecture, in case we conclude that NLP can help, where do we put this kind of process in the current architecture. It's unlikely that we'll use PHP to do that.

Deskana closed this task as Resolved.Jul 30 2015, 4:47 PM
Deskana claimed this task.

This was an epic we made when we started thinking about how to test theories. We've now got the tasks to test those theories in the sprint, so this is essentially resolved.