We could use relforge elastic cluster for this purpose, I can load some data there, what wikis would you like to test first?
Concerning an instance to host a mediawiki installation we have rel-forge.search.eqiad.wmflabs but it'd be maybe simpler to have a dedicated instance.

Hi @dcausse - using relforge elastic cluster sounds great and I think it might be easier/simpler to have a dedicated instance of it for this data.

According to this query, I got back this for pages for enwiki and en projects:

I believe the content pages is the magical number of actual pages that are shown in search queries...but I'm not positive about that. My confusion is coming from enwikinews: there are 21K-ish content pages, but over 2.7 million "pages"...how does that happen?

Would we be able to add in all the indices for these projects, but not the actual data? Would that overload any of the servers if we have that much added to an instance? If so, we can cherry pick the projects we want to add in.

@debt ok I'll start to import all the en indices.
I'll prefix all the indice names with crosswiki to avoid confusion with other indices.
I think we can load all the data, if we do not query too often these indices the overhead should be ok.
Concerning the ratio between pages and content pages on wikinews apparently there are 2.6m pages in the User talk namespace.
If I run into space issues I'll probably load partial data on non content indices as you suggested.

All english wikis are available on relforge cluster:

green open crosswiki_enwikibooks_content                        1 0    76157      0      2gb      2gb 
green open crosswiki_enwikibooks_general                        1 0   147808      0 1011.8mb 1011.8mb 
green open crosswiki_enwiki_content                             7 0  6363500      0  131.7gb  131.7gb 
green open crosswiki_enwiki_general                             8 0 22896342      0  233.4gb  233.4gb 
green open crosswiki_enwikinews_content                         1 0    21105      0  518.2mb  518.2mb 
green open crosswiki_enwikinews_general                         4 0  2676473      0   12.5gb   12.5gb 
green open crosswiki_enwikiquote_content                        1 0    28571      0    1.1gb    1.1gb 
green open crosswiki_enwikiquote_general                        1 0    93231      0  574.6mb  574.6mb 
green open crosswiki_enwikisource_content                       7 0  1766590      0   30.5gb   30.5gb 
green open crosswiki_enwikisource_general                       1 0   161859      0    1.2gb    1.2gb 
green open crosswiki_enwikiversity_content                      1 0    33860      0  875.6mb  875.6mb 
green open crosswiki_enwikiversity_general                      1 0   113400      0    1.1gb    1.1gb 
green open crosswiki_enwikivoyage_content                       1 0   149916      0      1gb      1gb 
green open crosswiki_enwikivoyage_general                       1 0    84397      0  626.2mb  626.2mb 
green open crosswiki_enwiktionary_content                       5 0  4998355      0   14.1gb   14.1gb 
green open crosswiki_enwiktionary_general                       2 0   448670      0      5gb      5gb

We now need to set up a new lab vm to run mediawiki on top of these indices.

The labs instance with relforge indices has been setup and is available at: http://sistersearch.wmflabs.org/

In T151344#2952472, @Jdrewniak wrote:

The labs instance with relforge indices has been setup and is available at: http://sistersearch.wmflabs.org/

Excellent! :-D