Page MenuHomePhabricator

Request increased quota for etytree labs project
Closed, ResolvedPublic


Project Name: etytree
Type of quota increase requested: ram

I am going to install a triplestore (Virtuoso) to query RDF data equivalent to the server accessible at which is managed by a group from the University of Grenoble and uses a 32 Gb RAM. As we don't have need administration administration privileges on that resource and as we need admin privileges in order to develop the project we are trying to install the server in the Wikimedia Labs. The data consists of 80 million triples (statistics calculated with the following query at the Virtuoso server I have used so far

Select ?p count(?p) as ?count where {

Graph <> { ?s ?p ?o }

} group by ?p

order by  desc(?count)


Considering that we need to apply ontology inference rules to the database, the total number of triples could increase by a factor of 2 or 3, which makes for a total of possibly 240 million triples.
A standard setup for such data requires at least a RAM of 32Gb (see for example where they have 257 million triples and use a RAM of 32Gb), also given that we need to be able to perform complex queries to produce a complex graphical output.

The above calculations are estimates. If after the setup has been completed and queries have been defined we will see that less resources are used we will scale them down.

We got the impression that getting this amount of memory in the Wikimedia Labs can be fairly difficult. If it won't be possible we will likely install the server locally at the RECAS were we will be assigned a virtual machine with a 32 Gb RAM. We would prefer to have the resource installed on Wikimedia labs for two main reasons:

  1. the project uses Wiktionary dumps
  2. sharing resources with volunteers and maintainability would be much easier if this resource was hosted at the WMF.

Event Timeline

Epantaleo created this task.Dec 5 2016, 4:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 5 2016, 4:30 PM

I've added a new instance type ('flavor') to your project called 'bigram' with 36Gb of RAM. I've also increased the project quota to permit creation of one bigram instance in addition to the instance you already have there. Please delete the old instance once you've migrated your configuration.

By the way -- is this project intended to run indefinitely, or does it have an expected lifespan? (If the latter, I'll make a note to clean up after the use period ends.)


That's great, thank you! I just deleted the old instance.

I hope the project will run indefinitely. As mentioned before, if I manage to reduce the amount of memory needed I'll write a note.

Andrew closed this task as Resolved.Dec 19 2016, 2:14 AM