This is not really a task or a bug, but more of a question from a toy maker (gadget author) to MediaWiki developers. The question is, would is make sense to filter gadgets server-side? Many gadgets are only applicable on a certain page, or in a certain namespace, or during a certain action. However, they are currently loaded everywhere anyway.
To show you what I mean, here is an example of what MediaWiki:Gadgets-definition could look like:
WikidataSearchResults[default|ResourceLoader|page=Special:Search]|WikidataSearchResults.js CommonsUploads[default|ResourceLoader|ns=2,3]|CommonsUploads.js Edittools[default|ResourceLoader|action=edit,submit]|Edittools.js
(I know that work is being done on a visual gadgets-definition, making this exact syntax obsolete, but that's beside the point)
The crucial question, of course, is whether this would be a performace improvement.