Is this still a problem?

(In reply to comment #1)

Is this still a problem?

Seems there're still a bunch.

$ echo 'select el_from, el_to, count(*) c from externallinks group by el_from, el_to having c > 1;' | sql itwiki_p > bug9900

http://toolserver.org/~liangent/-/dbq/bug9900

545907 rows in set (2 min 45.47 sec)

Looking at tables.sql on Gerrit (https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob;f=maintenance/tables.sql;h=a848bf5eb469ce63b2693b4a392241c5eab76dd1;hb=HEAD), we can see the pagelinks, templatelinks, categorylinks, imagelinks, langlinks, and iwlinks all have a unique index on them. externallinks, however, has the following indices:

CREATE INDEX /*i*/el_from ON /*_*/externallinks (el_from, el_to(40));
CREATE INDEX /*i*/el_to ON /*_*/externallinks (el_to(60), el_from);

CREATE INDEX /*i*/el_index ON /*_*/externallinks (el_index(60));

Try the query

select * from externallinks where el_to like "//%" limit 10;

and you will see the so called "duplicates". el_to is duplicate, but el_index (and el_index_60) is different.

I think this can get closed.