Maniphest T332040

Shut down our previous Cloud VPS project and create a new one
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	awight
	Mar 14 2023, 4:14 PM

Description

We regrettably named our WMCS project "wmde-templates-alpha" which no longer reflects the purpose of these machines. Destroy the instances and request the project be closed. Start a new one named something like "wmde-technical-wishes" with similar--this probably requires a separate Phabricator request.

Requirements:

16 VCPUs
16GB RAM
NFS access to dumps (currently dumpsdata1003.eqiad.wmnet)

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T345411 Scraper: destroy Cloud VPS runner instance
Resolved	None	T341751 Publish dump scraper reports
Resolved	None	T335411 Scraper: produce spreadsheet of scraped statistics for comparing wikis
Resolved	awight	T332032 Create baseline statistics for reference usage (2023)
Resolved	None	T332162 Run scraper on samples from several wikis
Resolved	None	T332045 Stream input file from a tarball
Resolved	None	T332047 Script to fetch the latest HTML dump for a given wiki
Resolved	awight	T332051 Alternative output formats
Resolved	awight	T332053 Basic aggregation from intermediate format
Resolved	None	T332056 Include additional page metadata
Resolved	awight	T332159 New cloud instance and attached volume for dumps processing
Resolved	awight	T332040 Shut down our previous Cloud VPS project and create a new one
Resolved	taavi	T332773 Destroy "wmde-templates-alpha" Cloud VPS project
Resolved	Andrew	T332777 Request creation of "dump-references-processor" VPS project
Resolved	None	T332165 Implement subdirectories for different wiki outputs
Resolved	None	T332873 Decide on file naming and organization structure
Resolved	None	T334591 Collect stats on templates inside of ref tags

Event Timeline

awight created this task.Mar 14 2023, 4:14 PM

awight updated the task description. (Show Details)Mar 15 2023, 9:28 AM

awight mentioned this in T332159: New cloud instance and attached volume for dumps processing.Mar 15 2023, 12:29 PM

awight added a parent task: T332159: New cloud instance and attached volume for dumps processing.

lilients_WMDE moved this task from Incoming to In progress on the WMDE-References-FocusArea board.Mar 15 2023, 2:15 PM

awight moved this task from Sprint Backlog to Doing on the WMDE-TechWish-Sprint-2023-03-14 board.Mar 22 2023, 12:21 PM

taavi closed subtask T332773: Destroy "wmde-templates-alpha" Cloud VPS project as Resolved.Mar 22 2023, 12:41 PM

awight added a subtask: T332777: Request creation of "dump-references-processor" VPS project.Mar 22 2023, 12:57 PM

awight updated the task description. (Show Details)

I kind-of arbitrarily increased our vCPU request, after getting hints from initial performance. We can process 30-40 simple pages per second per core, and there are roughly 250M pages total. This gives roughly 231 core-hours, or a day on 8 cores and half a day on 16 cores.

awight moved this task from Doing to Watching / Epic / Stalled on the WMDE-TechWish-Sprint-2023-03-14 board.Mar 22 2023, 1:17 PM

taavi removed a project: Cloud-Services.Mar 23 2023, 1:40 PM

aborrero changed the status of subtask T332777: Request creation of "dump-references-processor" VPS project from Open to In Progress.Mar 29 2023, 3:41 PM

Andrew closed subtask T332777: Request creation of "dump-references-processor" VPS project as Resolved.Mar 29 2023, 3:47 PM

awight closed this task as Resolved.Mar 30 2023, 12:15 PM

awight claimed this task.

awight removed a subtask: T333549: Requesting Cloud VPS access to NFS mount /public/dumps.

awight moved this task from Watching / Epic / Stalled to Done on the WMDE-TechWish-Sprint-2023-03-14 board.

awight moved this task from In progress to Done on the WMDE-References-FocusArea board.Oct 23 2024, 7:06 AM