User Details
- User Since
- Jun 7 2021, 2:32 AM (159 w, 2 d)
- Availability
- Available
- LDAP User
- Vivian Rook
- MediaWiki User
- VRook (WMF) [ Global Accounts ]
Sun, Jun 23
Thu, Jun 20
Wed, Jun 19
Ok, it's back online on version 3.1.1 with a restored db from June 18.
This went down as part of a shift to g4 vm flavors. It's back, but it looks like 4.0.0 was deployed, so the query history doesn't work. I'm unsure if that will cause other problems, and we may have to wipe everything and reload fresh. There are more details in T364022 about the query history.
Tue, Jun 18
Sat, Jun 15
Thu, Jun 13
Wed, Jun 12
Tue, Jun 11
Thu, Jun 6
Installing as root seems to get R working, and seems to still allow package install. Though bash kernel does not seem to work, pip shows it as installed, but the link does not appear in jupyterlab
https://github.com/OpenRefine/CommonsExtension/issues/101 is closed and seems to fix the issue. When a new release is made we can try this with an updated plugin.
Fri, May 31
+1
Rstudio is installing, but not R notebook/console
Thu, May 30
Changing project name to lowercase only.
root@cloudcontrol1005:~# openstack project delete TfInfraTest
root@cloudcontrol1005:~# openstack project create --description 'tofu infra tests' tofuinfratest --domain default
+-------------+------------------+
| Field | Value |
+-------------+------------------+
| description | tofu infra tests |
| domain_id | default |
| enabled | True |
| id | tofuinfratest |
| is_domain | False |
| name | tofuinfratest |
| options | {} |
| parent_id | default |
| tags | [] |
+-------------+------------------+
root@cloudcontrol1005:~# openstack role add --project tofuinfratest --user rook member
root@cloudcontrol1005:~# openstack role add --project tofuinfratest --user rook readerhttps://github.com/toolforge/paws/pull/429 seems to work
Wed, May 29
I agree, that the checksums and secrets could be stored as annotations. Though the image puller job would be more difficult to manage in a similar fashion. And in this case, the thing being deployed is the main part of the project, so not reporting changes isn't ideal. Leaving it as always showing changed, may be the better option.
This appears to be the same issue with prometheus. Removing:
set_values:
- value: prometheus.retention=30d
value_type: stringgets it working as expected.
This would appear to be a limitation of the helm ansible module. It seems to be seeing the set_values as a difference:
set_values:
- value: controller.service.type=NodePort
value_type: string
- value: controller.service.enableHttps=false
value_type: string
- value: controller.service.nodePorts.http=30001
value_type: string
- value: controller.config.proxy-body-size=4m
value_type: string
- value: controller.config.allow-snippet-annotations=true
value_type: stringRegardless of if they were already applied. Removing them makes the process deploy without changes, but of course also causes it to not work.
Tue, May 28
Seems to work. Generated:
https://public-paws.wmcloud.org/User:VRook_(WMF)/quarto.html
with quarto render quarto.ipynb --to html
May 24 2024
I keep getting kicked off ssh mid tofu test. Though it seems to mostly be working. I'll leave it off for the weekend and test it more in the week.
root@cloudcontrol1005:~# openstack project create --description 'TfInfraTest' TfInfraTest --domain default
+-------------+-------------+
| Field | Value |
+-------------+-------------+
| description | TfInfraTest |
| domain_id | default |
| enabled | True |
| id | TfInfraTest |
| is_domain | False |
| name | TfInfraTest |
| options | {} |
| parent_id | default |
| tags | [] |
+-------------+-------------+
root@cloudcontrol1005:~# openstack role add --project TfInfraTest --user rook member
root@cloudcontrol1005:~# openstack role add --project TfInfraTest --user rook readerMay 23 2024
May 22 2024
May 21 2024
Restarting the services seems to have things connecting again.
kubectl rollout restart deployment.apps/redis deployment.apps/web deployment.apps/worker
Still appears to be down. Web page is loading, but queries are giving:
Access denied for user 'quarry'@'172.16.2.72' (using password: NO)
Looks like none of the web pods are running. Logs give
[2024-05-21 15:28:04 +0000] [11] [ERROR] Error handling request /
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 178, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/app/quarry/web/app.py", line 82, in index
stats_count_users=global_conn.session.query(User).count(),
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3091, in count
return self._from_self(col).enable_eagerloads(False).scalar()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2832, in scalar
ret = self.one()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2809, in one
return self._iter().one()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2850, in _iter
execution_options={"_sa_orm_load_options": self.load_options},
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1689, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1583, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 324, in _execute_on_connection
self, multiparams, params, execution_options
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1462, in _execute_clauseelement
cache_hit=cache_hit,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1669, in _execute_context
conn = self._revalidate_connection()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 560, in _revalidate_connection
self._invalid_transaction()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 540, in _invalid_transaction
code="8s2b",
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. (Background on this error at: https://sqlalche.me/e/14/8s2b)