Page MenuHomePhabricator
Create Task
Maniphest T238652

Hardware request for Postgres database for censorship monitoring scripts
Closed, ResolvedPublic
Actions

Description

Hi,

I am looking to deploy our censorship monitoring scripts, one of which uses OONI's metadb [https://github.com/ooni/sysadmin/blob/master/docs/metadb-sharing.md], a PostgreSQL database with an approximate current size of 719GB.

I understand that Postgres is not the preferred database of choice but OONI uses that and so we are stuck with it. The database also has a lot of historic information so we need to query it occasionally and thus having it deployed makes it easier.

Note that this is not critical as we are making read-only queries to the database and so even if it fails, we can deploy it again.

Other requirements:

OS: Linux
Architecture: x86_64
Postgres version: 9.6

Please let me know if you have any questions about this. Thank you.

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedRobHT238652 Hardware request for Postgres database for censorship monitoring scripts
Unknown Object (Task)
Unknown Object (Task)
 ResolvedssinghT239250 setup/install cescout1001.eqiad.wmnet

Event Timeline

ssingh created this task.Nov 19 2019, 12:23 PM
Restricted Application added a project: SRE. · View Herald TranscriptNov 19 2019, 12:23 PM
akosiaris triaged this task as Medium priority.Nov 19 2019, 12:31 PM
akosiaris subscribed.

Adding some extra info as we 've discussed with @ssingh before this task was filed. After the discussion it became obvious that a ganeti VM would not be a good candidate for this, mainly because of the disk size requirement.

Site: EQIAD
Server Type: Traditional, a spare system would probably do just fine.
CPU/RAM/Storage: Aside from the need for ~1TB of disk space, the rest seem, assuming the usual reasonable ratios of CPU/RAM we have across the fleet seem fine.
Network: Internal IP
SSL Requirements: It does not need to terminate it's own SSL
Duration: Permanent
OS: 9.6 postgres means stretch. For now it should be fine, but looking into whether the software supported postgres 11 (aka Debian buster) might be worth it.

RobH moved this task from Backlog to Allocation/Ordering/Implementation on the hardware-requests board.Nov 25 2019, 7:25 PM

Ok, this is an odd one and doesn't meet any of our current configuration requirements. The only reason this isn't a VM is due to storage requirements being < 1TB. Our single (and dual CPU) misc class systems would work for this, except they have (2) 480GB SSDs.

We will likely need to order a new single cpu misc class system, but swap the (2) 480GB SSD for (2) 2TB SATA SFF disks in raid1 to meet the requirement.

I'll create a procurement task and link it to this. The procurement task will include the pricing and approvals.

RobH mentioned this in Unknown Object (Task).Nov 25 2019, 7:29 PM
RobH added a subtask: Unknown Object (Task).
RobH added a comment.Nov 25 2019, 7:35 PM

Alternatively, we use existing spare pool system wmf5174 or wmf5175 which have dual SSDs, and then swap in dual 2TB SFF disks if they have any spare in eqiad.

RobH mentioned this in Unknown Object (Task).Nov 25 2019, 7:57 PM
RobH added a subtask: Unknown Object (Task).
RobH reassigned this task from RobH to faidon.Nov 25 2019, 8:06 PM
RobH added subscribers: faidon, RobH.

Please note this task is now pending the approval of @faidon in conjunction with associated HDD purchase task T238652.

@faidon:

We would like to allocate single CPU misc system WMF5174 for use as a postgres db system for censorship monitoring scripts. It will have to have its dual 480GB SSDs swapped out with the dual 2TB SATA disks being ordered via T238652.

Basically both need approval together, or we'll need to go with a different solution. Thanks!

faidon reassigned this task from faidon to RobH.Nov 26 2019, 3:00 PM

Approved.

RobH mentioned this in T239250: setup/install cescout1001.eqiad.wmnet.Nov 26 2019, 5:15 PM
RobH closed subtask Unknown Object (Task) as Declined.Nov 26 2019, 10:06 PM
• ema subscribed.Dec 2 2019, 4:13 PM
Jclark-ctr closed subtask Unknown Object (Task) as Resolved.Dec 3 2019, 8:28 PM
RobH closed this task as Resolved.Dec 12 2019, 8:58 PM

Setup is being done via T239250, resolving task.

ssingh closed subtask T239250: setup/install cescout1001.eqiad.wmnet as Resolved.Apr 6 2020, 4:29 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL