HomePhabricator

Transactions T110964 Change Details

Change Details

Old
New
Diff

So I think it would be useful if we generated something like 1000-2500 stop words using TF-ISF per wiki, then run a cross-TF-IDF on all wikis and take the top 500 to 1000 words common in all wikis. I would expect these to include interwiki language ISO codes and words like http, com, net etc from urls.

So I think it would be useful if we generated something like 1000-2500 stop words using TF-ISF per wiki, then run a cross-TF-IDF on all wikis and take the top 500 to 1000 words common in all wikis. I would expect these to include interwiki language ISO codes and words like http, com, net etc from urls.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL