Page MenuHomePhabricator
Create Task
Maniphest T100781

Wikimetrics crashes when cohort description has special characters {dove} [5pts]
Closed, ResolvedPublic
Actions

Description

When uploading a cohort with special characters in its description, the upload form works fine.
But from there on, all cohort/list requests fail because of encoding conversion problems. Ha!

Details

SubjectRepoBranchLines +/-
Fix cohort description utf8 bug (3)analytics/wikimetricsmaster+29 -2
Fix cohort description utf8 bug (2)analytics/wikimetricsmaster+51 -1
Fix cohort description utf8 buganalytics/wikimetricsmaster+45 -2
Customize query in gerrit

Related Objects

Event Timeline

mforns created this task.May 29 2015, 12:08 PM
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added projects: Analytics-Wikimetrics, Analytics-Kanban.
mforns subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 29 2015, 12:08 PM
JAnstee_WMF awarded a token.May 29 2015, 5:21 PM
• kevinator triaged this task as Unbreak Now! priority.May 29 2015, 5:34 PM
• kevinator set Security to None.
• Halibutt subscribed.May 29 2015, 8:20 PM

Just to make it clear, "special characters" means most likely any non-Latin 1 character.

mforns renamed this task from Wikimetrics crashes when cohort description has special characters to Wikimetrics crashes when cohort description has special characters [5pts].Jun 1 2015, 3:21 PM
mforns claimed this task.
mforns moved this task from Next Up to Tasked_Hidden on the Analytics-Kanban board.
mforns moved this task from Tasked_Hidden to In Progress on the Analytics-Kanban board.Jun 1 2015, 3:23 PM
mforns renamed this task from Wikimetrics crashes when cohort description has special characters [5pts] to Wikimetrics crashes when cohort description has special characters {dove} [5pts].Jun 1 2015, 3:42 PM
gerritbot subscribed.Jun 1 2015, 9:01 PM

Change 215200 had a related patch set uploaded (by Mforns):
Fix cohort description utf8 bug

https://gerrit.wikimedia.org/r/215200

gerritbot added a project: Patch-For-Review.Jun 1 2015, 9:01 PM
mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 1 2015, 9:01 PM
gerritbot added a comment.Jun 2 2015, 4:29 PM

Change 215200 merged by Milimetric:
Fix cohort description utf8 bug

https://gerrit.wikimedia.org/r/215200

mforns mentioned this in rANWM54266113694c: Fix cohort description utf8 bug.Jun 2 2015, 5:15 PM
mforns moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Jun 2 2015, 7:01 PM
mforns moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Jun 2 2015, 7:11 PM
mforns moved this task from Done to In Progress on the Analytics-Kanban board.Jun 5 2015, 6:19 PM
mforns added a comment.Jun 5 2015, 6:21 PM

So Kevin discovered that the cohorts that have special characters in its descriptions are still crashing Wikimetrics.
Namely, all reports on those kind of cohorts are not working. When launching them, Wikimetrics alerts with the following error:

Error! Wikimetrics is experiencing problems. Visit the Support page for help if this persists. You can also check the console for details.
Milimetric moved this task from In Progress to Paused on the Analytics-Kanban board.Jun 8 2015, 4:16 PM
Milimetric moved this task from Paused to In Progress on the Analytics-Kanban board.
gerritbot added a comment.Jun 10 2015, 6:08 PM

Change 217315 had a related patch set uploaded (by Mforns):
Fix cohort description utf8 bug (2)

https://gerrit.wikimedia.org/r/217315

• ggellerman moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 11 2015, 3:35 PM
gerritbot added a comment.Jun 12 2015, 3:32 PM

Change 217315 merged by Milimetric:
Fix cohort description utf8 bug (2)

https://gerrit.wikimedia.org/r/217315

• ggellerman moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Jun 12 2015, 3:36 PM
mforns mentioned this in rANWM57c371b7992a: Fix cohort description utf8 bug (2).Jun 12 2015, 4:26 PM
mforns added a comment.Jun 16 2015, 10:10 AM

I tested it it staging and then deployed to production... and boom! It crashed in report_list.

The problem is that db collations are different in vagrant/staging vs. production.
The swedish_latin1 collation in vagrant/staging was masking a problem and made it difficult to identify in our tests.
When deployed to production (utf8_general_ci) the issue becomes evident.

Next step: Change the column type of report.parameters to varbinary, via a migration.
This should fix the issue.

mforns moved this task from Ready to Deploy to In Progress on the Analytics-Kanban board.Jun 16 2015, 3:31 PM
gerritbot added a comment.Jun 19 2015, 2:19 PM

Change 219373 had a related patch set uploaded (by Mforns):
Fix cohort description utf8 bug (3)

https://gerrit.wikimedia.org/r/219373

Milimetric moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 19 2015, 3:42 PM
gerritbot added a comment.Jun 22 2015, 8:56 PM

Change 219373 merged by Madhuvishy:
Fix cohort description utf8 bug (3)

https://gerrit.wikimedia.org/r/219373

mforns mentioned this in rANWM5172697fc89f: Fix cohort description utf8 bug (3).Jun 22 2015, 9:23 PM
mforns moved this task from In Code Review to Done on the Analytics-Kanban board.Jun 23 2015, 3:40 PM
• kevinator closed this task as Resolved.Jun 25 2015, 3:18 PM
• kevinator subscribed.

verified it works in production :-)

mforns added a comment.Jun 25 2015, 3:19 PM

o/

mforns mentioned this in T114881: Wikimetrics' cohort page is returning 500 in production {dove} [2 pts].Oct 7 2015, 11:48 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits