There are also core-specific modules, like the bulid and mobile groups, and libraries we already have in mw (jquery etc)

Fair point, it's a bit complicated.

I'd suggest creating a extension-ve-mw.json with ve-mw specific modules, then creating a grunt task to stitch together that file with bits of modules.json. We could specify which core modules to merge in with a "@coreModules" list on extension-ve-mw.json.

Unfortunately we can't add json comments to flag modules.json as a generated file :( (we can add a "@comment" property at the top though)