I need to ask a stupid question, aren't we making code more vulnerable to possible attacks by listing all dependencies we have, including with versions as a web resource?

Most of this information is already possible to extract by checking our git repo, but allowing any bot to retrieve current information by fetching a URL resource seems a bit scary.

In T363589#9771077, @Tgr wrote:

Tested on core: P61866
It passes JSON Schema validation at least.

I generally like this, just it would be great to have PURL too. If possible.

In T363589#9771428, @pmiazga wrote:

I need to ask a stupid question, aren't we making code more vulnerable to possible attacks by listing all dependencies we have, including with versions as a web resource?

Most of this information is already possible to extract by checking our git repo, but allowing any bot to retrieve current information by fetching a URL resource seems a bit scary.

STO is highly discouraged in cybersecurity: https://en.wikipedia.org/wiki/Kerckhoffs's_principle

In T363589#9771582, @Ladsgroup wrote:

I generally like this, just it would be great to have PURL too. If possible.

We'd either have to add a new field to foreign-resources.yaml then, or try to parse file URLs and figure out what kind of package manager they correlate with (probably not worth it).

In T363589#9771428, @pmiazga wrote:

Most of this information is already possible to extract by checking our git repo, but allowing any bot to retrieve current information by fetching a URL resource seems a bit scary.

Eh, I think it's pretty trivial for most would-be attackers to pull any of the lockfiles from public wikimedia repos, and that's all they would ever really need.

In T363589#9771582, @Ladsgroup wrote:

STO is highly discouraged in cybersecurity: https://en.wikipedia.org/wiki/Kerckhoffs's_principle

STO can be a marginally helpful tool in a limited number of scenarios. But no, it should never be the predominant or singular mitigation.

Change #1027588 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/core@master] foreign-resources: Add purl field

https://gerrit.wikimedia.org/r/1027588

Change #1027190 merged by jenkins-bot:

[mediawiki/core@master] foreign-resources: Add CycloneDX export support

https://gerrit.wikimedia.org/r/1027190

Change #1027588 merged by jenkins-bot:

[mediawiki/core@master] foreign-resources: Add purl field

https://gerrit.wikimedia.org/r/1027588

Change #1034991 had a related patch set uploaded (by Sportzpikachu; author: Sportzpikachu):

[mediawiki/core@master] foreign-resources: Add purl field to vue-demi

https://gerrit.wikimedia.org/r/1034991

Change #1034991 merged by jenkins-bot:

[mediawiki/core@master] foreign-resources: Add purl field to vue-demi

https://gerrit.wikimedia.org/r/1034991

Change #1060826 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/core@master] ForeignResourceManager: Write CDX file to disk

https://gerrit.wikimedia.org/r/1060826

Change #1060826 merged by jenkins-bot:

[mediawiki/core@master] ForeignResourceManager: Write CDX file to disk

https://gerrit.wikimedia.org/r/1060826

Change #1087577 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/CodeEditor@master] foreign-resources: Add purl for ace editor

https://gerrit.wikimedia.org/r/1087577

Change #1087578 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] foreign-resources: Add purl for ace editor

https://gerrit.wikimedia.org/r/1087578

Change #1087599 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/extensions/Chart@master] Add SBOM for foreign-resources

https://gerrit.wikimedia.org/r/1087599

Change #1087599 merged by jenkins-bot:

[mediawiki/extensions/Chart@master] Add SBOM for foreign-resources

https://gerrit.wikimedia.org/r/1087599

Change #1087577 merged by jenkins-bot:

[mediawiki/extensions/CodeEditor@master] foreign-resources: Add purl for ace editor

https://gerrit.wikimedia.org/r/1087577

In T363589#9770861, @Tgr wrote:

foreign-resources.yaml contains the following information for each file:

• the URL to download from (or the URL of a tarball and a path within, possibly as a path pattern with a * for variable parts like version numbers)
• the destination within the MediaWiki repository
• a hash

CycloneDX contains the following information:

• a single top-level hash for the entire library

After exploring CycloneDX (I’m novice in this area, take it with precautions), I see there is a type "file", so theoretically I guess it is possible to write something like the following for the "multi-file" type given above as example in T363589#9769177:

{
	"components": [
		{
			"type": "file",
			"name": "qunitjs/qunit.js",
			"hashes": [
				{
					"alg": "SHA-256",
					"content": "a833db4fcbe2683ea009a4eb51dec5384e655394bd1f95396dbb903000e7ad16"
				}
			],
			"externalReferences": [
				{
					"url": "https://code.jquery.com/qunit/qunit-2.20.0.js",
					"type": "source-distribution"
				}
			]
		},
		{
			"type": "file",
			"name": "qunitjs/qunit.css",
			"hashes": [
				{
					"alg": "SHA-256",
					"content": "34dda8ae8eebf1a61c4580387e705b8a374b452ed17383f3f6291c9a5c16be5e"
				}
			],
			"externalReferences": [
				{
					"url": "https://code.jquery.com/qunit/qunit-2.20.0.css",
					"type": "source-distribution"
				}
			]
		}
	]
}

I’m not sure if it is possible to map files from a tarball to extracted files in the source tree though, at least I didn’t find.

Change #1106928 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/extensions/ProofreadPage@master] Add CDX SBOM and purl for foreign-resources.yaml

https://gerrit.wikimedia.org/r/1106928

In T363589#10369049, @Seb35 wrote:

In T363589#9770861, @Tgr wrote:

foreign-resources.yaml contains the following information for each file:

• the URL to download from (or the URL of a tarball and a path within, possibly as a path pattern with a * for variable parts like version numbers)
• the destination within the MediaWiki repository
• a hash

CycloneDX contains the following information:

• a single top-level hash for the entire library

After exploring CycloneDX (I’m novice in this area, take it with precautions), I see there is a type "file", so theoretically I guess it is possible to write something like the following for the "multi-file" type given above as example in T363589#9769177:

{
	"components": [
		{
			"type": "file",
			"name": "qunitjs/qunit.js",
			"hashes": [
				{
					"alg": "SHA-256",
					"content": "a833db4fcbe2683ea009a4eb51dec5384e655394bd1f95396dbb903000e7ad16"
				}
			],
			"externalReferences": [
				{
					"url": "https://code.jquery.com/qunit/qunit-2.20.0.js",
					"type": "source-distribution"
				}
			]
		},
		{
			"type": "file",
			"name": "qunitjs/qunit.css",
			"hashes": [
				{
					"alg": "SHA-256",
					"content": "34dda8ae8eebf1a61c4580387e705b8a374b452ed17383f3f6291c9a5c16be5e"
				}
			],
			"externalReferences": [
				{
					"url": "https://code.jquery.com/qunit/qunit-2.20.0.css",
					"type": "source-distribution"
				}
			]
		}
	]
}

I like the idea. What do you think @Tgr ?

I’m not sure if it is possible to map files from a tarball to extracted files in the source tree though, at least I didn’t find.

We can look for the source instead? I honestly think that's better.

Change #1106928 merged by jenkins-bot:

[mediawiki/extensions/ProofreadPage@master] Add CDX SBOM and purl for foreign-resources.yaml

https://gerrit.wikimedia.org/r/1106928

Change #1087578 abandoned by Jforrester:

[mediawiki/extensions/WikiLambda@master] foreign-resources: Add purl for ace editor

Reason:

Was done in 15b23fa4ac8de7f04db874a3ae6db5ff38c55c0d.

https://gerrit.wikimedia.org/r/1087578

Change #1190105 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/CheckUser@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190105

Change #1190106 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/GrowthExperiments@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190106

Change #1190107 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/VisualEditor@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190107

Change #1190192 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/TimedMediaHandler@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190192

Change #1190193 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/Math@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190193

Change #1190194 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/EventLogging@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190194

Change #1190196 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/CodeMirror@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1190196

Change #1182449 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/TemplateData@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1182449

Change #1190255 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/CodeEditor@master] Add foreign-resources.cdx.json and fix error in config

https://gerrit.wikimedia.org/r/1190255

Change #1182449 merged by jenkins-bot:

[mediawiki/extensions/TemplateData@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1182449

Change #1190192 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190192

Change #1190105 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190105

Change #1190196 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1190196

Change #1190106 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190106

Change #1190800 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/TemplateData@master] Add foreign-resources file to linting

https://gerrit.wikimedia.org/r/1190800

Change #1190802 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/CodeMirror@master] Add foreign-resources files to linting

https://gerrit.wikimedia.org/r/1190802

Change #1190107 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190107

Change #1190802 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Add foreign-resources files to linting

https://gerrit.wikimedia.org/r/1190802

Change #1190194 merged by jenkins-bot:

[mediawiki/extensions/EventLogging@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190194

Change #1190800 merged by jenkins-bot:

[mediawiki/extensions/TemplateData@master] Add foreign-resources file to linting

https://gerrit.wikimedia.org/r/1190800

Change #1190255 merged by jenkins-bot:

[mediawiki/extensions/CodeEditor@master] Add foreign-resources.cdx.json and fix error in config

https://gerrit.wikimedia.org/r/1190255

Change #1190193 merged by jenkins-bot:

[mediawiki/extensions/Math@master] Add foreign-resouces.cdx.json

https://gerrit.wikimedia.org/r/1190193

Change #1191007 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/EventLogging@master] Update version number in foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1191007

Change #1191007 merged by jenkins-bot:

[mediawiki/extensions/EventLogging@master] Update version number in foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1191007

Seeing this moving forward is like seeing a dream come true. Thank you!

Change #1191644 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/Citoid@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191644

Change #1191645 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/DiscussionTools@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191645

Change #1191649 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/ReaderExperiments@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191649

Change #1191651 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/3D@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1191651

Change #1191644 merged by jenkins-bot:

[mediawiki/extensions/Citoid@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191644

Change #1191645 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191645

Change #1191649 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] Add foreign-resources.cdx.json and linting

https://gerrit.wikimedia.org/r/1191649

Change #1191651 merged by jenkins-bot:

[mediawiki/extensions/3D@master] Add foreign-resources.cdx.json

https://gerrit.wikimedia.org/r/1191651

Change #1197259 had a related patch set uploaded (by Samwilson; author: Samwilson):

[mediawiki/extensions/DonationInterface@master] Fix validation errors in foreign-resources.yaml

https://gerrit.wikimedia.org/r/1197259

The last part of this is DonationInterface, but it's hitting an error: T407812.

Change #1197259 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Foreign resources: Add CDX file and fix validation errors

https://gerrit.wikimedia.org/r/1197259

All of the above extensions are done now. (The DonationInterface error is unrelated.)