Page MenuHomePhabricator

Measure traffic for new wikimedia foundation site
Closed, ResolvedPublic3 Estimated Story Points

Event Timeline

Nuria created this task.Feb 27 2018, 6:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 27 2018, 6:14 PM
Nuria added a comment.Feb 27 2018, 6:25 PM

I *think* traffic is about 150k pageviews per day last month: https://tinyurl.com/ydcgapbx

Seems quite a bit for piwik actually.

Nuria renamed this task from Piwiki for new wikimedia foundation site to Piwik to measure traffic for new wikimedia foundation site.Feb 27 2018, 6:27 PM
Nuria added a subscriber: faidon.

Turns out documentation pages will not be included in this site, from my brief inspection terms_of_use and privacy_policy are the most visited pages in wikimediafoundation.org so it might be that number of pages is quite smaller

Q: This is to be a WordPress site, right? Looks like WordPress comes with built in pageview and unique stats? https://en.support.wordpress.com/stats/#views-and-visitors

Is there something else piwik would provide?

elukey added a subscriber: elukey.Mar 1 2018, 5:47 PM
Milimetric moved this task from Incoming to Blocked on the Analytics board.Mar 1 2018, 6:13 PM
Nuria added a comment.Apr 17 2018, 5:20 PM

Update on conversation. Given that this new foundation site is a third party hosted on wordpress. site our proposal is to check whether these wordpress plugins for stats do work to gather the data comns is interested in:

Nuria renamed this task from Piwik to measure traffic for new wikimedia foundation site to Measure traffic for new wikimedia foundation site.Apr 17 2018, 5:20 PM

@Nuria - our vendors reviewed those two modules and are not able to recommend them:

"As a summary, both plugins fall short in some key areas but the biggest blockers include:

  • Writes on page load which will cause scaling problems
  • Incompatibility with caching which will result in inaccurate data
  • Potential security concerns

One of the biggest issues is the caching, so it will be important that any solution uses javascript to handle the analytic reports. Ideally it should handle this on an external server to avoid scan problems because the requests writing to the server database on each page load will ultimately not scale."

I believe we remain with Piwik as the most preferable solution. We can investigate a third-party hosting solution for Piwik installation, but would obviously prefer to keep it in-house if possible. Please let me know how you suggest proceeding.

Hm, likely I just don't understand how the plugins work. Is it that the plugins only work record data on the backend when PHP is executed? I'd be surprised if that were the case, but if so, the concern about caching makes sense. I'd expect that these work more like google analytics (and piwik), but perhaps not?

Nuria added a comment.Jun 12 2018, 4:52 PM

Please @varment be so kind as to share the details about the technical objections.

@Ottomata and @Nuria - I have requested the technical information from Reaktiv and will share as soon as I can. However, they have reviewed these extensions and done two broad reviews of multiple WordPress plugins at our request as well. So far, each process has resulted in the elimination of WordPress plugin based options, leaving us with Piwik as the best option. So, I think at this point, the question is more are we using our installation or a third-party installation.

@Ottomata and @Nuria, here is the writeup they provided:

Alright, thanks. I've also just looked a bit into wp-statistics code, and it seems they do not use client side logging to grab the data. https://github.com/wp-statistics/wp-statistics/pull/109 would do that, but that has been sitting since January.

Thanks for doing the research, I concur that neither of these plugins will do the job. :)

@Ottomata and @Nuria, with that in mind - is internal hosting possible or should we look at third-party hosting for Piwik? We are looking to finalize that in the next few days in case we need to negotiate a third-party contract (which can take some time).

Nuria added a comment.Jun 15 2018, 8:57 PM

@Varnent we can probably accommodate the site but not immediately. What are the target dates you are looking for launch?

Site launch is scheduled for 30 July. Should we utilize third-party until internal is ready? Any sense of if we are talking about months or a fiscal year? :)

Nuria added a comment.Jun 19 2018, 5:08 AM

We can probably accomodate a 30 july launch date if by then we are getting less traffic on our current instance.

Nuria added a comment.Jun 28 2018, 4:43 PM

@Varnent confirming that we can accommodate site. Let us know when you are ready to include the piwik snipet

Nuria added a comment.Jun 28 2018, 6:07 PM

We will need to know the url of the new site

@Nuria - awesome! When it launches on 30 July 2018, the URL for the new site will be https://wikimediafoundation.org/

Nuria added a comment.Jul 2 2018, 8:16 PM

You will need to have your developer team include the piwik snippet, let us know when you are ready for that.

@Nuria - Great! They have an open ticket in their system ready for us to give them the code. Feel free to send it to me, or post it here and I'll pass it along. :)

Nuria added a comment.Jul 2 2018, 9:28 PM

<!-- Matomo -->
<script type="text/javascript">

var _paq = _paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
  var u="//piwik.wikimedia.org/";
  _paq.push(['setTrackerUrl', u+'piwik.php']);
  _paq.push(['setSiteId', '17']);
  var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
  g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s);
})();

</script>
<!-- End Matomo Code -->

I have passed it along - thank you!

Milimetric triaged this task as High priority.Jul 5 2018, 4:40 PM
Milimetric added a project: Analytics-Kanban.
Milimetric set the point value for this task to 3.
Milimetric moved this task from Blocked to Operational Excellence on the Analytics board.
Nuria added a comment.Jul 12 2018, 5:28 PM

You can ping us on irc once you are close to launch, your piwik user 'wikipedia15" should have access to the data for this site, i woudld check that is the case now (even if data reports are empty). I will be closing this ticket shortly, please reopen if you have an aissue.

Nuria closed this task as Resolved.Jul 12 2018, 5:29 PM

Just wanted to verify that you are receiving data from the site. :)

Varnent reopened this task as Open.Aug 1 2018, 9:50 PM

The login information we have for wikipedia15 does not appear to be working. Can you email me updated info or set up a general comms account? Thank you!

Nuria added a comment.Aug 6 2018, 2:07 PM

In order to access piwik (and any internal tool) you need to have ldap credentials (a user and a password) once you enter those the wikimedia15 credentials can be entered in the piwik login screen that should appear. Let us know what steps are not working.

Nuria added a comment.Aug 6 2018, 2:59 PM

Piwik is registering about 5000 visits per day

In order to access piwik (and any internal tool) you need to have ldap credentials (a user and a password) once you enter those the wikimedia15 credentials can be entered in the piwik login screen that should appear. Let us know what steps are not working.

I seem to be getting stuck at the LDAP login. I am using the same LDAP info I do elsewhere (I just verified on the OIT LDAP password screen that my info is working), but not letting me to Piwik login screen. Does my account have access?

Nuria added a comment.Aug 7 2018, 5:53 PM

Do file a ticket, it might be you are not on the ldap group needed: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Piwik#Access

Do file a ticket, it might be you are not on the ldap group needed: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Piwik#Access

Done - thank you!

@Nuria - good news - we now have access to piwik site via LDAP logins - bad news - wiki15 does not have access to the new website's data:

Varnent closed this task as Resolved.Aug 13 2018, 4:13 PM
Varnent assigned this task to Nuria.

This appears to be working - thank you @Nuria!