One of the things I would really like to see is the suite broken up in to multiple separate tool accounts. This should make it easier to keep a particular tool in the suite running. It will also promote cleaner coding practices in the shared parts.

Hello, @kaldari! Thank you for creating this task.

The current components of xTools are as follows (I've already checked off the ones I've converted the code for):

• Edit Counter
• Article Info ("Page History")
• Pages Created
• Top Edits
• Range Contributions
• Article Blamer
• Autoblock (It checks for autoblocked users - Maybe we should rename that tool)
• AdminStats
• RfX Analysis
• RfX Vote Counter
• Quick, Dirty, Simple Edit Counter

In additions, there are three tools currently disabled but have the code still in the codebase:

• Automated Edits
• Bash Quotes
• Scottywong's AdminScore

Right now, rXT has two directories. "Modules" contains the shared code as necessary, including the heart of the edit counter and API query code. "public_html" contains the individual code for the tools.

The current problem, the biggest one we're attempting to solve is to make xTools more maintainable. The current setup moves the view and controller logic in a single file, by using overloaded methods. Troubleshooting an issue often takes hours as it's impossible to locate the specific area where a problem occurs. This is also tied to uptime, as fixing an issue or finding an issue can break other components of xTools without any rhyme or reason. Code is done on a live toolset, which is not sustainable.

The rebirth code is written using the Symfony framework: http://symfony.com/. I opted to use the framework because it is very similar to Ruby on Rails, with regard to the Model-View-Controller scheme. Also, it includes caching and profiling out of the box.

My measure of completeness (We can debate this if you want) is that the rewrite acts the same as the currently running xTools. Or, to put it in different terms: Each one of the tools above returns valid results as expected based on its counterpart in the current live version of the tools.

@bd808: @MusikAnimal and I have been discussing moving xTools to its own Labs instance, or multiple labs instances behind a load balancer. The code right now is too tied to Tool Labs, and one of our goals of the rewrite is to make it platform-independent. However, I see your sentiment and agree with it wholeheartedly.

Update: 01/28 - updated list

In T154551#2917678, @Matthewrbowker wrote:

@bd808: @MusikAnimal and I have been discussing moving xTools to its own Labs instance, or multiple labs instances behind a load balancer. The code right now is too tied to Tool Labs, and one of our goals of the rewrite is to make it platform-independent. However, I see your sentiment and agree with it wholeheartedly.

Moving to your own Labs project is a big step in terms of resource commitment. Tool Labs provides a managed service where system administration is handled by the Labs staff. A standalone Labs project is more like leasing a VPS from a hosting provider. The Labs team will still provide a very basic level of support in keeping the OpenStack hosting platform operational, but nearly all system administration and management for your VMs will fall to the maintainer team.

I know that xTools has suffered from uptime issues in the past, but I really don't think you can fairly lay that on the Tool Labs infrastructure. Running a very large collection of separate tools under a single tool account is not a core use case for Tool Labs. By putting everything in one process you have made it difficult for yourselves and others to diagnose and correct problems. I certainly won't block you from requesting and migrating to a stand alone project, but you should weigh the costs and benefits fairly.

@bd808 I understand. My comment with divorcing the code from Tool Labs more had to do with fun like this fragment of code (Which makes it impossible to install a local copy, for example). I have not found tool labs unstable in the least and I know the problem is xTools not the Tool Labs infrastructure. I am working on making xTools completely platform independent, which means it could be installed on either without a problem. This also means, in theory, that we could create separate tool accounts for each tool with no ill effect.

Also worth noting: we do have a separate project for xTools, which means we can experiment with both if need be. I've actually created a VM and am running an experimental version of my code in there: http://xtools-dev.wmflabs.org . This is something that doesn't need to be decided now, thankfully.

In T154551#2917678, @Matthewrbowker wrote:

The rebirth code is written using the Symfony framework: http://symfony.com/. I opted to use the framework because it is very similar to Ruby on Rails, with regard to the Model-View-Controller scheme. Also, it includes caching and profiling out of the box.

In the past when I have tried to use Symfony on Tool Labs it has been very difficult because the timezone has not been set in php.ini. The Tool Labs team said that this was the case because of historical reasons, and people in the Symfony channel on freenode simply advised me to not use shared hosting. The documentation on setting a per-webservice php.ini does not seem to work with the timezone setting (it didn't when I tried it).

Other than the timezone issue, Symfony is a great framework to use.

@tom29739 Thank you for that information. I'll take a look at that when I have a few moments, and see if there's a workaround. It may require adding an ini_set to the Symfony core somewhere, though I'd rather not do that.

In T154551#2917712, @Matthewrbowker wrote:

@bd808 I understand. My comment with divorcing the code from Tool Labs more had to do with fun like this fragment of code (Which makes it impossible to install a local copy, for example).

@bd808 I wonder if toollabs-vagrant is worth considering? It'll make many tools that are hardwired to tool labs (directory paths, service dns aliases, etc.) much more easier to test locally.

In T154551#2919064, @zhuyifei1999 wrote:

@bd808 I wonder if toollabs-vagrant is worth considering? It'll make many tools that are hardwired to tool labs (directory paths, service dns aliases, etc.) much more easier to test locally.

It's a threadjack here, but I'd be willing to discuss it. It is completely possible to develop many tools using the existing MediaWiki-Vagrant project. I would be happy to review and merge patches for new roles that help folks out with their projects.

Generally the problem however is not that developers are forced to hardcode configuration. Instead it is a problem of education that they choose to hardcode configuration. This to my mind is the thing to attack with better tutorials that show how to setup a basic tool in $LANG that is designed to be portable between multiple developer environments and Labs/Tools.

There will always be challenges with configuring tools that use the replica DBs or dumps to run locally. Using ssh tunnels to expose these Labs resources to a local host is certainly possible but not something that most find trivially easy. There have been some other ideas about how to make that easier such as adding support for some sort of VPN that can be easily configured on Windows/OS X hosts.

My preference would be for any server-specific aspects of XTools to be configurable or optional. If we want more people to help maintain it, it needs to be easy to set up and test locally.

@kaldari I would go so far as to say it's required with this rewrite. In fact, I have spent quite a bit of time over the last two weeks ensuring that the labs-specific code is behind a configuration option.

Here's what I've got, mostly copied from @Matthewrbowker's comment:

What are the current components of XTools? (striked have already been rewritten)

• Edit Counter
• Article Info ("Page History")
• Pages Created
• Top Edits
• Range Contributions
• Article Blamer
• Autoblock (It checks for autoblocked users - Maybe we should rename that tool)
• AdminStats
• RfX Analysis
• RfX Vote Counter
• Quick, Dirty, Simple Edit Counter
• XTools API

There is also an API to get some of the data provided by the tools, which doesn't seem to work very well.

Do they share a lot of code or are they mostly separate?
Right now, rXT has two directories. "Modules" contains the shared code as necessary, including the heart of the edit counter and API query code. "public_html" contains the individual code for the tools. The views in the public_html may contain significant backend logic intertwined with the HTML. For the rewrite, we're using the Symfony framework and Twig, to separate out backend logic from the frontend.

Should they be split off into separate Tool Labs projects or kept together (keep in mind uptime issues)?
Per above, I would say yes. Tool Labs is very stable, comes with free sysadmin support, it'd make it easier to add new maintainers, and so forth. I believe Matthew is agreeance if it works well for us. There are some concerns regarding how compatible Symfony will work with Tool Labs (T154551#2917738), but I think we can figure it out.

What are the current major problems with the tools? Is it mainly just uptime or are there other problems?
Unmaintainable codebase, memory leaks, deprecated functions, etc., all leading to frequent downtime. A complete rewrite seems most favourable.

Would some sort of model/view framework benefit the codebase?
We think so. Symfony is what is being used for the rewrite, and makes setting up the suite on your local machine really, really easy. I personally am hugely impressed. It also provides a caching mechanism and cool performance diagnostics out of the box, among other things.

Are there any components that should be combined or abandoned?
We probably don't need to do "Range Contributions" given we'll have T145912 hopefully within a few months. "Autoblock" should also be deprioritized in favour of T146414.

Here are the other tools (roughly) in order of popularity:

1. Article Info ("Page History")
2. Edit Counter
3. Pages Created
4. Article Blamer (would be more popular if it worked well)
5. Top Edits
6. AdminStats
7. RfX Analysis
8. RfX Vote Counter
9. XTools API
10. Quick, Dirty, Simple Edit Counter (on-wiki gadgets and scripts are more often used)

I also personally feel "Automated Edits" should be revived, which would be a PHP-based, Xtools-branded version of https://tools.wmflabs.org/musikanimal/nonautomated_edits. Very low priority considering there is a working alternative.

How do we know when the rewrite is complete?
When all in-demand tools (numbered above) have been rewritten, are stable, and redirects put it place for the old tools. E.g. /xtools/topedits would go to /xtools-topedits, and so forth.

What about the Xtools API?
This was only implemented in part for the old Xtools, giving you basic edit data about a page, that was used in some gadgets such as User:Hedonil/XTools.js. This was (is) a very popular gadget, and hopefully we can continue to offer it, perhaps also providing other functionality for API consumers. The thing is that this gadget fed off of a single API endpoint that provided info you'd otherwise have to get from multiple tools. That being said, having a /api for each corresponding tool may not make sense.

So my thoughts are to repurpose the base /xtools Tool Labs project as sort of a "home page" for the XTools, with links to each tool, but on the backend there'd be /xtools/api.php (just as there is now). We can work on that API separately from the other tools. I don't think any of the actual tools (/xtools-ec, /xtools-articleinfo, etc.) should rely on an API that exists as part of another Tool Labs project, it will be there purely for consumers and to revive the popular gadget.