Page MenuHomePhabricator

Prepare and check storage layer for zghwiki
Closed, ResolvedPublic

Description

The new wiki's visibility will be: Public.

Related Objects

Event Timeline

Marostegui triaged this task as Medium priority.Nov 2 2023, 6:52 AM
Marostegui moved this task from Triage to Blocked on the DBA board.
Marostegui subscribed.

Let us know when the wiki is created so we can sanitize it

Marostegui moved this task from Blocked to Done on the DBA board.
Marostegui added a subscriber: BTullis.

All done, ready for the views creation.

The views were generated, but the cookbook failed when attempting to run the DNS step.

----- OUTPUT of 'source /root/nov...ca-dns --aliases' -----                                                                                                                                                        
2023-11-06T16:29:56Z __main__     WARNING : Ensuring db.svc.eqiad.wmflabs.                                                                                                                                         
2023-11-06T16:29:57Z root         WARNING : Error response from https://openstack.eqiad1.wikimediacloud.org:29001/v2/zones:                                                                                        
{"code": 500, "type": "unknown", "request_id": "req-eb301ab4-53e3-40ee-a9d2-9a0a631eaeb3"}                                                                                                                         
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/local/sbin/wmcs-wikireplica-dns", line 190, in <module>                                                                                                                                               
    main()                                                                                                                                                                                                         
  File "/usr/local/sbin/wmcs-wikireplica-dns", line 128, in main                                                                                                                                                   
    r = dns.zones(name=zone)                                                                                                                                                                                       
        ^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                       
  File "/usr/lib/python3/dist-packages/mwopenstackclients.py", line 518, in zones                                                                                                                                  
    r = self._get("/v2/zones", params=params)                                                                                                                                                                      
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                      
  File "/usr/lib/python3/dist-packages/mwopenstackclients.py", line 500, in _get                                                                                                                                   
    return self._req("GET", *args, **kwargs)                                                                                                                                                                       
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                       
  File "/usr/lib/python3/dist-packages/mwopenstackclients.py", line 496, in _req                                                                                                                                   
    r.raise_for_status()                                                                                                                                                                                           
  File "/usr/lib/python3/dist-packages/requests/models.py", line 1021, in raise_for_status                                                                                                                         
    raise HTTPError(http_error_msg, response=self)                                                                                                                                                                 
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://openstack.eqiad1.wikimediacloud.org:29001/v2/zones?name=db.svc.eqiad.wmflabs.                                              
================                                                                                                                                                                                                   
PASS |                                                                                                                                                                             |   0% (0/1) [00:02<?, ?hosts/s]
FAIL |█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:02<00:00,  2.25s/hosts]
100.0% (1/1) of nodes failed to execute command 'source /root/nov...ca-dns --aliases': cloudcontrol1005.eqiad.wmnet
0.0% (0/1) success ratio (< 100.0% threshold) for command: 'source /root/nov...ca-dns --aliases'. Aborting.
0.0% (0/1) success ratio (< 100.0% threshold) of nodes successfully executed all commands. Aborting.

I will check whether there are any known issues with Openstack at the moment.

Confirmed that I can query the view just fine and see all the rows. So it might be indeed just related to the DNS and not affecting the data underneath. Once it is fixed the data should be there ready for the tools.

Mentioned in SAL (#wikimedia-operations) [2023-11-07T11:33:47Z] <btullis@cumin1001> Added views for new wiki: zghwiki T350240

BTullis moved this task from In Progress to Done on the Data-Platform-SRE board.

I've run the cookbook again and the DNS step has now completed, so it must have been a transient failure. Resolving this ticket.