Do we really want to give more resources to a team-specific VPS project?

@Krinkle, are we so tight on our quota that we can't afford a few (seven) more instances? The tooling is really great and we'd love to make better use of it! A few more instances would really help developers deploy changes for testing and have more options for Vagrant-based workflows which are often flaky and very fragile. If we are so limited, how many instances can we add?

Edit: sorry @Krinkle, I meant @Krenair above ^^^.

Well I'm not @Krinkle but for me the problem is less about affordability and more about trying to remove the anti-pattern of team-specific projects. Raising quotas on request for such projects is likely moving in the wrong direction. Instead we should probably be decreasing quotas on a regular basis or at least every time an existing instance disappears.

See also: T177616: Request creation of reading-demos VPS project T177632: Document policy on Cloud VPS project scope

@Krenair, sorry about the name mix up.

You're welcome of course to close this ticket but I wanted to share my opinion and reasoning that allowing a small pool of instance resources to be assigned to a small group can be a good practice. I'll start first by discussing the points in T177632:

There are a few reasons for discouraging 'team' aligned projects. One is that teams can and do change while their software projects tend to stick around. Another is that team alignment can restrict or discourage community involvement in the Cloud VPS projects. A third is that it can become very difficult for the Cloud VPS admins to find canonical points of contact for grab bag projects. When we need to determine who to work with to correct issues with VMs in a project its not fun to wait a couple of weeks for people to "ask around" about who may still be using a particular VM.

Our number of instances is small and we've exhausted them which is why we're making this request for a few more. If someone departs the team and no longer participates, we will reclaim.

I disagree that sharing instances among teammates discourages community involvement. I apologize in advance if that's ignorance. All our code is open and HTTP access to the servers is not restricted. We also make an effort to document long term instances if other contributors want to mimic the setup but no doubt there's more we could do. Would it help if we assigned these resources to each individual on the team instead of "the team"?

With no intent to oversimplify, we do have a dedicated mailing list for ownership / point of contact issues. I would argue that it's actually easier if a team of individuals is responsible for a group of instances since someone usually knows what's going on and if no one does, then you have reason to free the instance(s). I disagree with the idea that it's easier to chase individuals down since there's no backup person but I respect that your experience in actually working with teams may have shown differently in the past.

I wanted to add some of my own reasons as well for requesting these additional resources:

• I recognize that a ticket is a small thing. Nevertheless, requiring formal requests to establish often temporary instances can inhibit the spirit and practice of innovation. If a trusted developer decides they want to create a prototype and share it quickly, or deploy requested changes for testing by others, it now involves paperwork of a sort and signoff by others. Developers should be free within reason to use the tools to collaborate and build.
• It's a great freedom to be able to tear down an entire VM and start over as opposed to shared instances where configuration is never really known since multiple developers are working in the same space and starting over affects multiple users.

Thanks for reading and your consideration.

In T198135#4313662, @Niedzielski wrote:

@Krenair, sorry about the name mix up.

We're used to it. :)

In T198135#4313662, @Niedzielski wrote:

You're welcome of course to close this ticket

I'm not in a position to really close it but I'm not convinced it's a good idea to accept it.

In T198135#4313662, @Niedzielski wrote:

Our number of instances is small

7 instances puts you in the top 20 labs projects, so certainly not large but not exactly small either.

In T198135#4313662, @Niedzielski wrote:

I disagree that sharing instances among teammates discourages community involvement. I apologize in advance if that's ignorance. All our code is open and HTTP access to the servers is not restricted.

How widely do you hand out SSH access to people outside your team?

In T198135#4313662, @Niedzielski wrote:

• I recognize that a ticket is a small thing. Nevertheless, requiring formal requests to establish often temporary instances can inhibit the spirit and practice of innovation. If a trusted developer decides they want to create a prototype and share it quickly, or deploy requested changes for testing by others, it now involves paperwork of a sort and signoff by others. Developers should be free within reason to use the tools to collaborate and build.

It's not like we're talking about what developers may run on their laptops - these are finite shared resources.

In T198135#4313662, @Niedzielski wrote:

• It's a great freedom to be able to tear down an entire VM and start over as opposed to shared instances where configuration is never really known since multiple developers are working in the same space and starting over affects multiple users.

Yes, though this in itself is only really enough to justify an instance or two's breathing room rather than doubling the quota.

Given the amount of users in this instance and our history of requests (see also T144623) can I ask that we at least bump this quota up by 2-3 requests to give us some breathing room since this is a recurring but irregular problem.

We often have multiple large projects running at a single time - right now our instances are used for testing page previews; skin changes; page images; proton (pdf printing); supporting our IFTTT wikipedia channel and prototypes such as trending.wmflabs.org.

I've been working hard to merge projects onto a single machine where possible (focusing on prototypes) but ive exhausted those options now.

Right now I need to build a new instance for testing some accessibility changes to users which will happen during the month of July (so im running out of time) and i cant recycle any of existing machines given our current commitments.

Personally Id be happy with a moderate increase of 2-3 (one large instance) to allow me to do this. We can continue talking about doubling if necessary but that would at least unblock me :)

@Niedzielski and @Jdlrobson, please update this request to follow the template described at https://phabricator.wikimedia.org/project/profile/2880/ with specific instance count, cpu core, and ram increases that are being requested. If you can adjust your ask to be less than double normal quota that would be appreciated.

It would also be helpful to see some documentation at https://wikitech.wikimedia.org/wiki/Nova_Resource:Reading-web-staging (or linked to from there) that describes the projects being maintained in your Cloud VPS project. This will help everyone better understand the work that these resources are advancing and make it easier to notice when an experiment has advanced to the point where it needs additional resources or retirement.

I've created T198267 to request breathing room. I'll document the existing services this afternoon so keep an eye on those wiki pages!

(We should keep this ticket open to explore the need and justification for more separately.)

I'm removing this from the Cloud-VPS (Quota-requests) workboard now that T198267: Request increased quota for reading-web-staging Cloud VPS project exists.