We talked back at Wikimania about the possibility of moving the interface for this tool from on-wiki to Tool Labs (and also giving it a database back-end in the process). This would remove the need for users to install custom JavaScript and would also make it easier to build a UI for doing things like filtering and searching. @eranroz, @Doc_James, @Ragesoss: Does that still sound like a good idea? Are there any issues that would make that unfeasible?

@kaldari There is already a database backend, with an API for fetching data (although it needs some improvements). The API does not include a write component for recording which hits are false positive, which are true positives, and which are still not checked. Here's the basic API: http://tools.wmflabs.org/eranbot/plagiabot/api.py?action=suspected_diffs

Some specific suggestions for improving the API:

• Allow it to return all suspected diffs from a certain time range (so that, for example, a client could poll it once per day to get the new copyvio diffs).
• Implement unicode support (for article titles) and standard JSON output ( here's how my system works around it: https://github.com/WikiEducationFoundation/WikiEduDashboard/blob/master/lib/importers/plagiabot_importer.rb#L70 )
• Support for writing back to the database to update the status (for example: false positive, true positive, checked but unclear, unchecked)

Agree we need to figure out how to get data from human follow up entered on Wikipedia into the database.

When create project specific boxes of concern, many edits of concern will be within a number of WikiProjects. If one WikiProject deals with it we need that info to feedback to the database and than out to all the project boxes.

@Ragesoss: Right you are. My memories from Wikimania are a bit fuzzy now :)

Earn added that database after Wikimania.

Any thoughts on moving the interface to Tool Labs? In theory, we could implement it as a special page, but it would be a bigger project and also a bit awkward as MediaWiki software is generally supposed to be service agnostic (at least for services that aren't part of MediaWiki itself), but this would depend on a specific API service running on Tool Labs via ithenticate/turnitin. Personally, I think it would make the most sense to host the entire thing on Tool Labs, but perhaps also have a simple report bot that posts small reports for WikiProjects which link back to the main interface on Tool Labs (possibly pre-filtered for the WikiProject).

I was hoping to have the interfaces built into each Wikiproject. We want to have it in the places people most often go. We still need to bring on board more volunteers to follow up.

How https://en.wikipedia.org/wiki/Special:NewPagesFeed works is excellent IMO.

Re: the API:

In Python, you can use ast.literal_eval to turn it back into a dictionary. Still not a great solution--I'd love to have a proper JSON API, and preferably to store the report data in a more structured form so you don't need to parse wikitext to get the values out.

@Niharikakohli and I are discussing this at the #DevSummit16. Questions:

• Who has the most expertise and who should Community Tech talk with?
• What are the desired outcomes? Currently noted:
  • A way to get the false/true positive checks into the DB
  • A way to get turnitin to check a revision on demand
  • UI/UX improvements (extensionise?)
  • WikiProject integration (how? A fairly enwiki-specific usecase--how broadly requested is this)
  • API improvements
• What is the minimum viable product for this? What minimal changes will improve the flow/experience?

I'm going to participate in Wikimedia Hackathon 2016 and would like to work on the copy&paste detection bot.

1. If there are other developers who are interested in working on this please let me know, so we can plan and what sub-tasks to work here.
2. As the main developer of the current bot, I will be happy to give intro to the current implementation for anyone who is interested.

@eranroz Excellent, @Fhocutt and @NiharikaKohli will be at the Hackathon too. We should talk before then...

Re the query from @Fhocutt on WikiProject integration, it comes down to the question of alerting the relevant concerned humans to an edit that needs human attention. The viable options may include:

1. a bot remark (with difflinks) on the affected article's talkpage, perhaps with an associated hidden maintenance category
2. a structured bot-post to a centralized noticeboard (similar to WP:ANI) served by a trusted team, substantially larger than the current team
3. a structured bot-post to a WikiProject's-central noticeboard, such as WT:WikiProject Medicine, where editors have somewhat more interest in the affected article, as cued by the article talkpage header {{WikiProject Medicine}}

So one hope, now that we have the data in a database, is that we can have boxes that go on each WikiProject of copyright concerns that relate to that WikiProject.

People can than click true or false in those boxes and the data will automatically feed back to the database and then feed out to all the WikiProjects boxes that contain that entry as a single article can be part of many projects.

By the way we have a talk accepted for Wikimania on "Detecting Copyright Concerns in Near Real Time" :-)

By the way we have a talk accepted for Wikimania on "Detecting Copyright Concerns in Near Real Time" :-)

In T120435#1998715, @Doc_James wrote:

By the way we have a talk accepted for Wikimania on "Detecting Copyright Concerns in Near Real Time" :-)

Awesome!

@eranroz, I'd be happy to work with you on this task at the Hackathon if you're still looking for collaborators. :)

@Niharika , yes I would like to. See you in the hackthon.

@Niharika, @eranroz: If you decide to take the MediaWiki extension route, you should take a look at https://en.wikipedia.org/wiki/Special:NewPagesFeed. This page is created by the PageTriage extension. Specifically, you'll want to look at:
PageTriage/SpecialNewPagesFeed.php
PageTriage/modules/ext.pageTriage.views.list/*
PageTriage/api/ApiPageTriageList.php
Note that it uses Backbone and Underscore extensively on the client-side. If you don't want to use those, you could use Mustache for the templating part (MediaWiki core has built in support for Mustache: https://www.mediawiki.org/wiki/Manual:HTML_templates).

@kaldari, from my chat with Eran, he doesn't think it is a good idea to have a Special page to specifically provide support to a non-official/external bot. I agree with him because labs is not 100% reliable. He suggests developing a Tool Labs interface for the bot. This should be not very difficult to achieve given that there's a database that his tool maintains with information for every copyright violation the bot found and logged on https://en.wikipedia.org/wiki/User:EranBot/Copyright/rc. Then we can add filters for Wikiprojects and date-time range searches etc.

Thoughts?

@Niharika: Sounds reasonable to me. If you want the interface to use infinite scrolling, you may want to use the jQuery waypoints (http://imakewebthings.com/waypoints/), BTW. I would be happy to help if you guys have a repo set up for it.

In the Hackathon notes I see you are considering integrating ORES into the system. I would love to hear @Halfak's thoughts on this. Do you think it would be possible to train ORES to help detect plaigarism?

Here's some notes based on discussions we're having at the Hackathon:

https://meta.wikimedia.org/wiki/Community_Tech/Improve_the_plagiarism_detection_bot/Notes#Hackathon.2C_Apr_1

How working in other wiki?

@Shizhao It's only working on English Wikipedia now... Once we get it working on Tool Labs for English WP, we'll see what needs to get done to make it available to other languages.

Updates: We've started putting together a prototype tool here: http://tools.wmflabs.org/plagiabot/

And here are some new draft wireframes. I'd love to know what you all think -- these are still developing.

Item structure:

Opening the compare pane:

Clicking on a review setting:

Clicking on the review again, to change your mind:

Filters, with autocomplete for WikiProjects:

Filters, with WikiProjects selected:

Also, so no one interest misses it: changes on http://tools.wmflabs.org/plagiabot/ are not being logged right now, so don't be afraid to play around.

The bot provides "hints" on the source (see in the /rc page) - as part of the post processing the bot visits the possible source and checks if the source cites Wikipedia (e.g "Mirror"), or have a clearly defined CC copyright. I think the current web based tool doesn't show it yet (I couldn't at least find it in "Meena", revision time: 2016-05-13T16:14:17Z), so this should be considered.

I really like the design (or the last updates in the design :) ) of plagiabot tool and in particular the "Edit count" near the editor.

Will try to have a look over the weekend.

Besides the obvious need to keep problematic edits out of articles, this tool is also an opportunity to educate editors who make these mistakes. Many problematic edits are done in good faith. Somewhere this tool should mention the need to put notes on user talk pages. Maybe an automatic link to the user's page with a couple templates (warnings) to choose from.

In T120435#2293881, @Lucas559 wrote:

[...] Somewhere this tool should mention the need to put notes on user talk pages. Maybe an automatic link to the user's page with a couple templates (warnings) to choose from.

In my opinion, the tool should allow the operator to perform all operations without leaving the page. So in addition to a button to enter a message in the user's talk page (like Lucas559 in T120435#2293881), also a link to undo or rollback the edits are useful.
Even the information that the editor has already had other reports or blocks is very important.

I like the layout of page, is clear and simple; good!

The wire frames above look good. Currently missing from the web version is the percentage of plagiarism detected and the word count for the edit. These would definitely be helpful on smaller wikis where there are fewer users to do plagiarism checks.

As you develop this tool, please consider how it can be used on multiple projects, not just Wikipedia. This would be very useful on en.wikiversity, for example.

In T120435#2293298, @eranroz wrote:

The bot provides "hints" on the source (see in the /rc page) - as part of the post processing the bot visits the possible source and checks if the source cites Wikipedia (e.g "Mirror"), or have a clearly defined CC copyright. I think the current web based tool doesn't show it yet (I couldn't at least find it in "Meena", revision time: 2016-05-13T16:14:17Z), so this should be considered.

I really like the design (or the last updates in the design :) ) of plagiabot tool and in particular the "Edit count" near the editor.

Ah, there's where the hints come from! Do you think it makes more sense to store that in the database instead of both the tools checking the page individually?

In T120435#2293881, @Lucas559 wrote:

Besides the obvious need to keep problematic edits out of articles, this tool is also an opportunity to educate editors who make these mistakes. Many problematic edits are done in good faith. Somewhere this tool should mention the need to put notes on user talk pages. Maybe an automatic link to the user's page with a couple templates (warnings) to choose from.

That's a good idea, Lucas. I'll create a ticket for it.

In T120435#2294363, @Beta16 wrote:

also a link to undo or rollback the edits are useful.

This has come up a few times now and I must raise my concern. In a lot of cases that I went through, the edit was either 1) already reverted or 2) there were more edits on top of it.
Unless you look at the edit immediately after it comes in, such a rollback button is not very useful.
What we can do is: have a rollback button in cases where it is possible to rollback and when you click the rollback button, we perform a check to see if it is still possible to rollback the edit (that is, nobody edited the page in the meantime) and if so, we roll it back. Sounds useful? More ideas welcome!

Even the information that the editor has already had other reports or blocks is very important.

It should be simple to keep a count of how many times an editor's edit has been marked as a Copyvio in the tool. What kind of other blocks do you have in mind?

I like the layout of page, is clear and simple; good!

Thanks!

In T120435#2294641, @Niharika wrote:
This has come up a few times now and I must raise my concern. In a lot of cases that I went through, the edit was either 1) already reverted or 2) there were more edits on top of it.
Unless you look at the edit immediately after it comes in, such a rollback button is not very useful.
What we can do is: have a rollback button in cases where it is possible to rollback and when you click the rollback button, we perform a check to see if it is still possible to rollback the edit (that is, nobody edited the page in the meantime) and if so, we roll it back. Sounds useful? More ideas welcome!

You're right, this scenario is very common. So maybe these links can be useful to add near 'Diff':

• 'Diff to the current' (&diff=cur&oldid=nnnnnnn) for check if the edit is already reverted
• 'hist' (&action=history) for check if there are other edits

What kind of other blocks do you have in mind?

Local or global wiki blocks. I know that a block is not only for copyrights violations, but it's a good indicator if the user is known, and for a good o bad faith.

/w/api.php?action=query&format=json&list=blocks&bkstart=(check_date_time)&bkusers=+Foo&meta=globaluserinfo&guiuser=Foo

Blocking a user direct from the tool is bound to backfire on w:en and likely elsewhere. At most the tool should create a noticeboard post to draw local admin attention to the offending editor's repeated actions. The bulk of Plagiabot reporty gnoming should not require admin priviledges, but blocking users obviously would. A standardized action request to (e.g.) [[w:en:wp:AIN]] can bridge the gap.

In T120435#2297541, @LeadSongDog wrote:

Blocking a user direct from the tool is bound to backfire on w:en and likely elsewhere. At most the tool should create a noticeboard post to draw local admin attention to the offending editor's repeated actions. The bulk of Plagiabot reporty gnoming should not require admin priviledges, but blocking users obviously would. A standardized action request to (e.g.) [[w:en:wp:AIN]] can bridge the gap.

Hi, I think you misunderstood the comment. We're not going to provide a way to block someone from the tool but instead we can provide them the information that the user has had X bans on Y wiki etc.

Everyone, https://tools.wmflabs.org/copypatrol/ is ready for you to look at and use. We're still doing some feature development but all of the basic things are in place. You can start reviewing edits after you log in. Keen to hear your feedback!

For now it is only usable by English wikipedia; it's right? In the future, it will also be activated for other wikis?
It also expected to translate the interface?

In T120435#2387989, @Beta16 wrote:

For now it is only usable by English wikipedia; it's right? In the future, it will also be activated for other wikis?
It also expected to translate the interface?

We use Ithenticate to detect plagiarism on english wikipedia. It's database is rich in English content but not so much in other languages. Eran tested running the tool with Hebrew wikipedia but the results were not successful.
In future, if this tool proves to be useful, we would like to try using the Google API service to see if it works better for other languages.

They have a fair bit of coverage in other languages. We could definitely
look at trying to get this running in other languages. Coverage was week in
Hebrew but I am sure good in French, German, and Spanish.

J

One strange question: Where is source code for CopyPatrol tool to contribute to? There is no link in tool page and no link in the discussion page nor here

@Ladsgroup currently at https://github.com/Niharika29/PlagiabotWeb. Contributions welcome :) I'll look into adding a "View source" link to the interface.