Create a method for users to call for page checks on demand
Open, NormalPublic

Description

When the bot code is in it's final stages of development, and I'm thinking we're slowly approaching that area, we should think of ways of implementing a way for users to request an on-demand check of specific pages.

One idea I have is to create an interface for the user to OAuth into an interface, where they can input a page, or list of pages, to lookup and/or run checks on.

Working on the bot code to make it more modular, the interface can utilize the bot's internals as it's engine. More details to come.

This is low priority for now until the bot is nearly finished in development.

Restricted Application added subscribers: JEumerus, Aklapper. · View Herald TranscriptJan 29 2016, 5:15 AM
Addshore added a subscriber: Addshore.EditedJan 29 2016, 6:00 PM

This is starting to sound like it could be an extension.
I'm sure some third party wikis would find an extension that would identify 404 etc links as usefull and provide an option to use the wayback machine where possible.
Also looking at how this is currently done on various wikis it looks like a centralized effort might make sense

There many even be some code that could be beaten into some sort of shape https://www.mediawiki.org/wiki/Extension:BrokenLinks
Or borrowed from other places http://plugins.svn.wordpress.org/broken-link-checker/trunk/

Some other ideas to consider:

  • A gadget that adds a "Fix dead links" link to the Tools menu. Clicking the link either triggers Cyberbot to immediately fix the dead links in the page or it takes the user to a Tool Labs interface similar to what is described in the task description above.
  • A gadget that adds a "Fix dead links" button to the Wikitext editing interface. Clicking the button sends the Wikitext of the page to a Cyberbot API which then returns the Wikitext with all the dead links fixed. The new text is inserted into the editing interface and the user can continue to edit before saving.

Some other ideas to consider:

  • A gadget that adds a "Fix dead links" link to the Tools menu. Clicking the link either triggers Cyberbot to immediately fix the dead links in the page or it takes the user to a Tool Labs interface similar to what is described in the task description above.
  • A gadget that adds a "Fix dead links" button to the Wikitext editing interface. Clicking the button sends the Wikitext of the page to a Cyberbot API which then returns the Wikitext with all the dead links fixed. The new text is inserted into the editing interface and the user can continue to edit before saving.

I love the idea, but how would I inject the wikitext into the edit window? I don't think that's doable. I was thinking that the interface edits on behalf of the user with the click of a button, using OAuth. The bot itself uses OAuth to edit so the underlying communication system is in place to place an interface over.

kaldari added a comment.EditedJan 29 2016, 8:06 PM

It would all be handled by the gadget code on the client-side. Basically, clicking on the "Fix dead link" button would execute something like:

  $.ajax( {
  	type: 'POST',
  	url: 'https://tools.wmflabs.org/cyberbot/gadgetapi.php',
  	data: {
  		text: $( '#wpTextbox1' ).val()
  	},
  	success: function( data ) {
  		var textWithFixedLinks = data.fixedtext;
  		if ( textWithFixedLinks ) {
  			$( '#wpTextbox1' ).val( textWithFixedLinks );
  			$( '#wpDiff' ).click();
		}
  	}
  } );

That would insert the new Wikitext into the edit window and also give you a preview of the changes. (Just to clarify, this is for the 2nd API-based gadget idea.)

I'm not convinced about a gadget doing this. Some pages can take a bit of processing if that page hasn't been cached in the bot's DB. This can be unintentionally abused when impatient users keeping clicking the button, and end up inadvertently calling the bot several dozen times. I'm still more fond of the interface idea and using Oauth. This is exactly why xTools always went down, because of excessive requests. Besides, I have other ideas for an interface that would make the most use of the gadget seem more tedious.

When the bot code is in it's final stages of development, and I'm thinking we're slowly approaching that area, we should think of ways of implementing a way for users to request an on-demand check of specific pages.

Sorry, but I don't really understand the need for this task. What makes you think people would want to go querying for "whether page X has dead links"?

One idea I have is to create an interface for the user to OAuth into an interface, where they can input a page, or list of pages, to lookup and/or run checks on.

Why do we need OAuth? People can get access to the tool interface without authentication, like most other similar tools.

Some other ideas to consider:

A gadget that adds a "Fix dead links" link to the Tools menu. Clicking the link either triggers Cyberbot to immediately fix the dead links in the page or it takes the user to a Tool Labs interface similar to what is described in the task description above.
A gadget that adds a "Fix dead links" button to the Wikitext editing interface. Clicking the button sends the Wikitext of the page to a Cyberbot API which then returns the Wikitext with all the dead links fixed. The new text is inserted into the editing interface and the user can continue to edit before saving.

@kaldari, when the bot does run automatically, why do we need a gadget for it? Why not just let people add the category for external dead links (something like that?) and the bot will fix it when it updates the page next?

DannyH moved this task from Untriaged to Analysis on the Community-Tech board.Feb 1 2016, 6:11 PM
In T125188#1985659, @NiharikaKohli wrote:

When the bot code is in it's final stages of development, and I'm thinking we're slowly approaching that area, we should think of ways of implementing a way for users to request an on-demand check of specific pages.

Sorry, but I don't really understand the need for this task. What makes you think people would want to go querying for "whether page X has dead links"?

One idea I have is to create an interface for the user to OAuth into an interface, where they can input a page, or list of pages, to lookup and/or run checks on.

Why do we need OAuth? People can get access to the tool interface without authentication, like most other similar tools.

Some other ideas to consider:

A gadget that adds a "Fix dead links" link to the Tools menu. Clicking the link either triggers Cyberbot to immediately fix the dead links in the page or it takes the user to a Tool Labs interface similar to what is described in the task description above.
A gadget that adds a "Fix dead links" button to the Wikitext editing interface. Clicking the button sends the Wikitext of the page to a Cyberbot API which then returns the Wikitext with all the dead links fixed. The new text is inserted into the editing interface and the user can continue to edit before saving.

@kaldari, when the bot does run automatically, why do we need a gadget for it? Why not just let people add the category for external dead links (something like that?) and the bot will fix it when it updates the page next?

Let me try to explain. Some users who like statistics it might prove useful. With the newest DB feature I am integrating into the bot, building an interface isn't too much out of the way/difficult to implement. Also the interface serves to let user input a list of articles/categories.

The OAuth serves to prevent bots from hammering the interface so only legitimate users can make use of this bot.

To answer the last part, since this is an interface which spawns the bot with the selected articles, it is a preferable method over spamming articles with categories to simply summon the bot. This however is low priority and won't gain focus until the bot code is complete for use.

In T125188#1985659, @NiharikaKohli wrote:

Let me try to explain. Some users who like statistics it might prove useful. With the newest DB feature I am integrating into the bot, building an interface isn't too much out of the way/difficult to implement. Also the interface serves to let user input a list of articles/categories.

The OAuth serves to prevent bots from hammering the interface so only legitimate users can make use of this bot.

To answer the last part, since this is an interface which spawns the bot with the selected articles, it is a preferable method over spamming articles with categories to simply summon the bot. This however is low priority and won't gain focus until the bot code is complete for use.

Thanks for the explanation, @Cyberpower678. It might prove useful for some people but I would definitely like to see a little more formal use-case analysis and maybe talking to some community members who'd possibly use this to gain their input. As you said, this can wait until the bot is up and perfect.

Cyberpower678 added a comment.EditedFeb 1 2016, 6:31 PM
In T125188#1986970, @NiharikaKohli wrote:
In T125188#1985659, @NiharikaKohli wrote:

Let me try to explain. Some users who like statistics it might prove useful. With the newest DB feature I am integrating into the bot, building an interface isn't too much out of the way/difficult to implement. Also the interface serves to let user input a list of articles/categories.

The OAuth serves to prevent bots from hammering the interface so only legitimate users can make use of this bot.

To answer the last part, since this is an interface which spawns the bot with the selected articles, it is a preferable method over spamming articles with categories to simply summon the bot. This however is low priority and won't gain focus until the bot code is complete for use.

Thanks for the explanation, @Cyberpower678. It might prove useful for some people but I would definitely like to see a little more formal use-case analysis and maybe talking to some community members who'd possibly use this to gain their input. As you said, this can wait until the bot is up and perfect.

I've got numerous on and off wiki praises for the bot followed with a question if there is a way to summon the bot to hand picked articles for the user, on demand. I came up with the interface idea because it can easily be built on the new code structure I designed for the bot, and even multi-thread.

To follow up, this interface can be made to run globally as the bot supports more wikis to run on.

This seems like a reasonable feature to add once the bot is complete. I imagine the most common use cases would be:

  • People needing to fix specific articles that were undergoing Good Article Review or Featured Article Review (which has a deadline and thus might not want to wait for the bot to get around to it).
  • People wanting to immediately fix dead links for specific articles where source verification is important, for example, on medical topics.

I agree that it is low priority, though, and I would suggest taking an agile approach to development of this feature. For example, building a very simple version with throttling instead of OAuth to start with. Then if it's apparent that throttling isn't adequate, an OAuth layer could be added afterwards.

Cyberpower678 raised the priority of this task from "Low" to "Normal".Mar 25 2016, 11:15 PM

With Cyberbot approaching completion for the English Wikipedia, I'd say we can get started on this now.

Cyberpower678 removed Cyberpower678 as the assignee of this task.Jun 23 2016, 2:55 AM

@kaldari is this something you would want to take on?

Idea: Maybe a VE extension for this makes sense.