Copied here from GitHub, to keep all issues together.
Look for duplicate files with linksearch
To avoid #5 it would be useful to look for the IA URL in the linksearch API and perhaps confirm with the CommonsMetadata API. I'll send a patch at some point.
And we can check Wikidata for matches between Commons files and IA identifiers too (although that's not all that widely used yet, I think).