Currently, there is no information recorded when Mitmachen is used. (Unless Toolforge/Labs does some monitoring beyond server logs in the background.)
For the campaign, it might be valuable to have some information on how many users visited and actually used the tool to go search for articles to edit.
I'm not sure how much effort this requires and whether we can actually gather insightful data that way.
Notes from call with Cirdan:
Track how many people clicked the link to the tool in the tutorial
and then count how many search requests the tool got.
Find out what tracking already exists in toolforge
What is meaningful to track depends on our concerns. I assume for now, it is
We could also check for the ranking quality of the algorithm etc. but I assume the above are more important and are relatively easy to gather.
Thanks Jan! The next time we use this tool in a campaign, we will consider these points for tracking/ analysis:
How many people use the tool
how many people choose a tutorial link
how many use the search or choose a suggestion (so they actually get to see possible articles)?
how many people click an article link (so they might actually improve the article)?
I cc @Stefan_Schneider_WMDE because he is our tracking pro.
From my perspective I have the following datapoint, that would be interesting:
Maybe they can be implemented too?
Adding the protocol from a meeting with Software engineering on August 15th:
Tracking whether edits are done/ how many:
We need to use the foundation analytics tool “Eventlog”, which is run by the Foundation.
It demands programmer and database skills. It’s not just data analyst skills. This takes time, need to talk to another team (WMF)
Jan D. from UX Team has done this successfully once with Verena from IF Team at WMDE.
Tracking on the page:
Can we use Piwik / Matemo? => can be answered by Kai or by Gabriel from Software Engineering at WMDE.
Could we write manually to the Eventlog thing? => Gabriel probably knows. If Gabriel can’t answer the question, Verena can set us up with someone from the WMF analytics team.
Tracking from where users come:
This is possible with a parameter/id/extension that will be just stored everytime the link is clicked.
This link must be defined when it is inserted in a page or other training tool.
Link example with parameter: https://foobartool.de/?comingfrom=wikihelp
Update: @Stefan_Schneider_WMDE will follow up with this task, to gauge options of how we're going to proceed regarding this matter.
I've had now a Meeting with SWE and we figured out some ways to track the things we want to know about the users. The most questions are directed to you. Mostly according the Logs that are already running on the web-server (or not).
How many people use the tools
Please check the Web-Server-Access-Log for the data and feedback, what logs and how long the data is stored.
How many people choose a link to other onboarding material
This I will clarify with Goran: Check Web-Request-Logs of Wikipedia if there are referrer from the Tool-URL.
ToDo: Check with Goran, if this data can be easily gatherd and analysed and probably witch timeframe would be sufficient. Input Data would be: Wiki-pages with the tutorials and the time-frame to analyse the referrer of the web request logs.
tasks the users enter in the search field and finally click on
The data is sent as a service request to the server. So it could be stored in some log-files.
ToDo: Which data is in the log-files and what is possible here? How long the data is stored there?
clicks on specific search items after entering the topic of interest
In the meeting we found: This is not possible to track within the tool. Only the potential clickable pages would need to be checked. This could be very sophisticated.
ToDo: Cirdan, does the tool really cannot track these clicks?
IF NOT
I'll ask Goran if this is trackable with his tools.
URL they came from
This information could be in the Web-Access-Logs/Web-Server-Logs.
ToDo: Could you check this, too? And also what time-frames are logged?
All relevant Information to this task you also find in this doc. I would appreciate your answers and also comments there.
I'm not sure whether Toolforge logs anything. You would need to discuss this with someone from the Cloud Services Team.
tasks the users enter in the search field and finally click on
The data is sent as a service request to the server. So it could be stored in some log-files.
ToDo: Which data is in the log-files and what is possible here? How long the data is stored there?
I don't think we want to write to a file, as there will be concurrent requests. As we discussed at WikiCon, we could set up a small database for this purpose, or log to an external service by sending a request to some API. The latter would be my preferred solution, as we could avoid adding a lot of campaign-specific code to Mitmachen.
clicks on specific search items after entering the topic of interest
In the meeting we found: This is not possible to track within the tool. Only the potential clickable pages would need to be checked. This could be very sophisticated.
ToDo: Cirdan, does the tool really cannot track these clicks?
IF NOT
I'll ask Goran if this is trackable with his tools.
The tool currently does not track anything. I also don't understand why this would be interesting, as the list of search results is inherently random. It is possible to trigger a log entry every time someone clicks on a link to in the search results, but as with the search keywords, these logs need to be stored somewhere.
URL they came from
This information could be in the Web-Access-Logs/Web-Server-Logs.
ToDo: Could you check this, too? And also what time-frames are logged?
As stated above, I don't know about any logs for Toolforge, so you would need to contact the Cloud Services team.
@Cirdan Thank you for your quick reply. I will check with the Cloud Services team on the topics above: Could you help me with a contact? How do I contact them best to get the questions answered?
Regarding an API or a possibility to store data I will talk to our software department and see if this is not to complicated.
@Cirdan I recently got the following feedback from a cloud services admin:
You should find an access.log file in the tool home dir that contains all web access logs for the tool.
Could you have a look at this directory and send me a copy of this logs? I would like to check wether there is information we can use.
Another option – so I don't have to bother you again with this issues – would be to provide me access to the directory. I'm not confident in accessing it myself, but I would find someone from our software-team to help me here.
Thx in advance for your support here.
Kindly,
Stefan
Sorry that it took me so long to get back to you!
I don't see any such file in the home directory.
Another option – so I don't have to bother you again with this issues – would be to provide me access to the directory. I'm not confident in accessing it myself, but I would find someone from our software-team to help me here.
I added you as a maintainer of the tool, you should be able to access it by SSHing into Toolforge and executing become mitmachen.
Thank you for your reply - I‘ll check it out :)
Cirdan <no-reply@phabricator.wikimedia.org> schrieb am Fr. 29. März 2019 um
07:41:
Cirdan added a comment.
Sorry that it took me so long to get back to you!
In T198938#4869512 https://phabricator.wikimedia.org/T198938#4869512,
@Stefan_Schneider_WMDE
https://phabricator.wikimedia.org/p/Stefan_Schneider_WMDE/ wrote:You should find an access.log file in the tool home dir that contains all
web access logs for the tool.I don't see any such file in the home directory.
Another option – so I don't have to bother you again with this issues –
would be to provide me access to the directory. I'm not confident in
accessing it myself, but I would find someone from our software-team to
help me here.I added you as a maintainer of the tool, you should be able to access it
by SSHing into Toolforge and executing become mitmachen.*TASK DETAIL*
https://phabricator.wikimedia.org/T198938*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/*To: *Stefan_Schneider_WMDE, Cirdan
*Cc: *Stefan_Schneider_WMDE, Jan_Dittrich, Verena, sonkiki, Cirdan,
AndyTan, Zylc, 1978Gage2001, Jayprakash12345, Chicocvenancio, Tbscho,
JJMC89, srodlund, Gryllida, scfc, Mbch331, Krenair, chasemp