Looks like jenkins is giving the patch a +2 now

@GoranSMilovanovic see https://wikitech.wikimedia.org/wiki/Production_shell_access#Additional_permissions_for_existing_users

In T171049#3480783, @GoranSMilovanovic wrote:

@Addshore Depends. If mwlog1001.eqiad.wmnet is a production web-server, I then *I don't want to have access to it* - and it probably is, since people need special access rights there.

In T171049#3478889, @GoranSMilovanovic wrote:

@Addshore Adam, could you please send that hack generated data on guided tours for this campaign to @Verena and @Stefan_Schneider_WMDE - they manage a concise table with all the data on the Summer Banner Campaign that were collected thus far? Thank you.

This is blocked by T165316 which is the issue we see in CI with the above patch.

Ahh, if you simply mean the details of the users that were shown a guided tour vs not shown a guided tour then that comes from the log created by the hack as discussed in the other ticket.

@bmansurov this makes sense now that I have read the description of Spike! thanks.

I'm curious, this task specifies a 4 hour window, which 4 hours in particular is the task talking about?

So, of course all requests are in the webrequest table but I don't think it would be possible to figure out which requests to the create account page are for either campaign and which are not.

That seems realistic, we had a similar difference in numbers between this code log and the records recorded in the event logging tables for the last campaign.

Those numbers are for all requests made in the following log files / the folliwng days:

go for the webrequest table on Hadoop and get the user registration data for Summer Banner Campaign 2017 from there; compare with what is found in log.ServerSideAccountCreation_5487345.

So, this was part of the LARGE refactoring of the serialization stuff / removing lib serializers for those that remember that mess (it spans about 30 patches or something like that).

So this is the same issue that we had for the spring campaign.

I wouldn't say it makes dispatching slower, just there is more to dispatch, and currently the dispatch system can only handle so many changes per second / minute / hour.

Devs that might be able to help can be found @ https://github.com/wikimedia/mediawiki-extensions-GuidedTour/blob/master/extension.json#L4

Would it be possible to link to the script and or command that was run in this ticket for future reference?

So this is something that we never ended up looking at for the last campaign.

I believe https://github.com/wikimedia/mediawiki-extensions-Campaigns/blob/master/Campaigns.hooks.php#L88 should help you find it.

I won't be able to help out / look at this for the next days (vacation, woo) but will be able to check back in this ticket on the 27th.

I won't be able to help out / look at this for the next days (vacation, woo) but will be able to check back in this ticket on the 27th.

Which is provided by the Campaigns mediawiki extension.

I dont see why not!

The table need to be replicated to the labs DB replicas in order to be accessible to Quarry.

In T95663#3444616, @Legoktm wrote:

I think we should start this by doing it one extension/library at a time, starting with those that aren't modified as often. Libraries should go in mediawiki/vendor (though there's still a dependency listed in Wikibase's composer.json) instead of the "Wikidata" extension, and sub-extensions should be loaded like a normal extension through mediawiki-config, etc.

So although I got the scripts into working order the crons were broken due to the server move.
See T170472#3439400 for the details.
Should have 2 clear runs over the weekend now, but I'll leave this ticket open and confirm on monday.

So it looks like all of the scripts seem to be working now in my testing, however all of the crons are failing....

Reopened until the above patch is merged

So it looks like this is unrelated to the other issue and actually this broke after the issue above was fixed (i believe)

Marking as stalled until there is an NDA

In T170551#3435086, @MoritzMuehlenhoff wrote:

I've removed her from the cn=wmde LDAP group (removals don't need manager approval, especially if confirmed by https://wikimedia.de/wiki/Mitarbeitende)

Relating to the main issue, my searching lead me to https://stackoverflow.com/questions/38643944/sqoop-import-as-parquetfile-from-oracle-gets-wrong-file-types#comment64792030_38643944

The SQLException: Access denied issue has been resolved outside of this ticket.

After giving this a quick go it has something to do with passing the password for mysql in through --password. If you use -P and enter the password this works just fine.

So, I checked a few scripts and they all seem fine (after we fixed some permissions)
Also checked the Java stuff and that seems to be running just fine too.

Ooooh, PHP7 instead of PHP5.
Please just go ahead and flip the switch. I'm around to fix anything that breaks but i doubt things should... Famous last words.

In T110170#3427264, @Qgil wrote:

As per T116271, the plan is to start deploying in mediawiki.org only. In that task there is a link to the community check.

I'm going to go ahead and mark this as low was we are trying to get rid of the build anyway.

Script for today is re running now and the mysql errors are gone.
Data should appear soon.

Yep, I can confirm this is to do with the change @ T170066, I'll try and get this fixed today.

The data for the wikidata-api dashboard is always a couple of days behind and it looks like that is running just fine.

I'm pretty sure there is another ticket discussing the issues around the build somewhere but I can't seem to find it.
Perhaps it has only been talked about in hallway discussions & emails.

Fixed? :)

In T168632#3420776, @kai.nissen wrote:

After a short talk, we decided to remove the parameters from the link.

In T110170#3423295, @greg wrote:

Looks like next step is to get it on Beta Cluster?

Now on commons and added to https://commons.wikimedia.org/wiki/Category:Wikidata_geocoding

Please take a look at the latest maps generated @ https://tools.wmflabs.org/wikidata-analysis/20170703/geo2png/

The first run failed as the JAVA proc ran out of memory.
I have started another one tonight with twice the memory allocation.

In T168628#3421477, @GoranSMilovanovic wrote:

@kai.nissen @Verena @Addshore

Please [CHECK THIS]: Pivot reports back 5 Request Counts and 5 Normalized Request Counts for:
WMDE_editor_campaign_summer_2017_a

approx. 18:33 CEST

Please let me know whether this numbers make sense.

In T170072#3418732, @Esc3300 wrote:

If it's scripted, is there a way to do one for every country ?

In T169066#3420853, @WMDE-Fisch wrote:

Maybe lost data during transfer?

@kai.nissen It might be possible to use the JS console to trigger a pretend request to the beacon?
In Fact, if you deploy the banner on beta-meta and trigger it using a regular page view you should be able to copy the request nice and easily with all the URL params.
Copying that and changing the servername from the beta server to de.wikipedia.org should then all work.
Back in 1.5 hours.