The tests and the configuration we run today with WebPageReplay lives on the server that runs the tests. The WebPageTest tests lives in Git but in the same project as the WebPageTest wrapper. When we start to add the mobile test, we need a new structure for that.
One of the goals we had as a team in 2015 was that we wanted to feel "We are not needed anymore". One way to fix that is to make it easier to add new tests.
We should make sure we follow the same structure on all tests we use. I think the idea of having one repo with all the tests (per type) is good. I like to have it in Git so we easily can track changes, then we can setup up our tests to run with descent defaults, so you as a developer needs to configure only the URL. I got a couple of different ideas I wanna try out and then get back to this issue.