awight (Adam Roses Wight)
User

Projects (24)
View All

Active Tasks
Milestone
AICaptcha
Component
Book-Referencing
Component
Diffusion-Repository-Administrators
Group
Dumps-Rewrite
Component

Calendar

User Details

User Since: Oct 12 2014, 9:02 PM (333 w, 1 d)
Availability: Available
IRC Nick: awight
LDAP User: Awight
MediaWiki User: Adamw [ Global Accounts ]

Recent Activity
View All

Yesterday

awight moved T259347: Accesskey "," doesn't work when syntax highlighting is toggled on from Review to Watching (Stalled) on the WMDE-TechWish-Sprint-2021-02-17 board.

Mon, Mar 1, 4:01 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, MediaWiki-extensions-CodeMirror, Community-Tech

awight added a comment to T274353: Check for the ReferenceTooltips gadget having a non-canonical name.

I believe it's still a god idea to list all exceptions in https://gerrit.wikimedia.org/r/663185, even for wikis where it currently can't have an effect.

Mon, Mar 1, 10:46 AM · WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Reference Previews

Fri, Feb 26

awight placed T193169: [reportupdater] Add a configurable hive client up for grabs.

Fri, Feb 26, 6:07 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight committed rARPQ27f2cc2067bf: Rewrite date match to avoid buggy UDF (authored by awight).

Rewrite date match to avoid buggy UDF

Fri, Feb 26, 5:10 PM

awight claimed T193169: [reportupdater] Add a configurable hive client.

Fri, Feb 26, 4:59 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight added projects to T193169: [reportupdater] Add a configurable hive client: WMDE-TechWish-Sprint-2021-02-17, Unplanned-Sprint-Work.

Fri, Feb 26, 4:58 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight moved T193170: [reportupdater] eliminate the funnel parameter from Sprint Backlog to Watching (Stalled) on the WMDE-TechWish-Sprint-2021-02-17 board.

Fri, Feb 26, 3:36 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Analytics

awight added projects to T193170: [reportupdater] eliminate the funnel parameter: WMDE-TechWish-Sprint-2021-02-17, Unplanned-Sprint-Work.

Fri, Feb 26, 3:36 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Analytics

awight moved T193174: [reportupdater] consider not requiring date as a first colum of query/script results from Sprint Backlog to Watching (Stalled) on the WMDE-TechWish-Sprint-2021-02-17 board.

Fri, Feb 26, 3:35 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight added a project to T193174: [reportupdater] consider not requiring date as a first colum of query/script results: WMDE-TechWish-Sprint-2021-02-17.

Fri, Feb 26, 3:35 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight added a comment to T193174: [reportupdater] consider not requiring date as a first colum of query/script results.

Another detail to mention: the output writer currently includes a date column, and I believe that removing it would cause the header change detection to invalidate old results. Maybe we add a date column back into the results before writing?

Fri, Feb 26, 12:36 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight added a comment to T193174: [reportupdater] consider not requiring date as a first colum of query/script results.

Since we have control over all jobs using this tool, I think we can move quickly with the migration. It's still nice to include a soft cutover, in case of rollback, etc. My thought is:

Phase 1: RU checks whether the first output column header is date. If so, logic is enabled to strip the value during import.
Phase 2: Once RU with soft migration has been deployed for a few days without errors, then we remove the date column from all reports.
Phase 3: At our leisure, remove the soft migration code from RU.

Fri, Feb 26, 12:28 PM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, Patch-For-Review, Analytics

awight added projects to T270239: Grafana board for TemplateWizard: WMDE-TechWish-Sprint-2021-02-17, Unplanned-Sprint-Work.

This is unblocked now. Moving into the sprint because it's related to the other Grafana board changes.

Fri, Feb 26, 10:19 AM · Unplanned-Sprint-Work, WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-01-20), WMDE-QWERTY-Sprint-2021-01-06, WMDE-QWERTY-Sprint-2020-12-16, WMDE-Templates-FocusArea

awight added a comment to T273475: Adjust edit count bucketing for TemplateWizard, segment all metrics.

The metrics are landing in Graphite, but we're still waiting for the updated bucket labels.

Fri, Feb 26, 10:18 AM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, Analytics-Radar, WMDE-Templates-FocusArea

awight updated the task description for T273475: Adjust edit count bucketing for TemplateWizard, segment all metrics.

Fri, Feb 26, 10:14 AM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, Analytics-Radar, WMDE-Templates-FocusArea

Thu, Feb 25

awight added a comment to T210106: Provide a reusable getEditCountBucket function for analytics purposes.

Sorry I wasn't aware of this task until now, and may have duplicated some work. In patch aca489f139cd13df, I added code to the EventLogging extension which makes wgUserEditCountBucket available from JS, and EventLoggingServices::getInstance()->getUserBucketProvider()->getUserEditCountBucket( $user ) from the backend.

Thu, Feb 25, 8:53 PM · Readers-Web-Backlog (Kanbanana-FY-2020-21), Analytics-Radar, Patch-For-Review, Front-end-Standards-Group, Analytics-EventLogging, MediaWiki-extensions-WikimediaEvents

awight updated subscribers of T275757: Reportupdater output can be corrupted by hive logging.

Thu, Feb 25, 4:02 PM · Patch-For-Review, Analytics-Radar

awight updated subscribers of T275757: Reportupdater output can be corrupted by hive logging.

Playing with logging is uncovering more errors. When run like hive --hiveconf hive.root.logger=ERROR,console, you can see each of our boilerplate cast(split('2021-02-11', '-')[2] as int) calls included in every reportupdater query actually cause a shower of these internal errors:

2021-02-25T14:29:03,877 ERROR [e2497dad-77df-493c-ad49-f3ea5cc83a38 main] optimizer.ConstantPropagateProcFactory: Unable to evaluate org.apache.hadoop.hive.ql.udf.generic.GenericUDFSplit@3f81621c. Return value unrecoginizable.

Thu, Feb 25, 3:31 PM · Patch-For-Review, Analytics-Radar

awight added a comment to T275757: Reportupdater output can be corrupted by hive logging.

A related issue is that SL4J is complaining, and I believe this topic is unmaskable without changing config. Can we fix the root cause?

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Thu, Feb 25, 2:09 PM · Patch-For-Review, Analytics-Radar

awight updated the task description for T274544: Support multiple transclusions (i.e. templates) in the template dialog sidebar.

Thu, Feb 25, 1:29 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight updated the task description for T274543: Create a new checkbox list view for the VisualEditor template dialog.

Thu, Feb 25, 1:23 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight updated the task description for T274544: Support multiple transclusions (i.e. templates) in the template dialog sidebar.

Thu, Feb 25, 1:22 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight updated the task description for T274543: Create a new checkbox list view for the VisualEditor template dialog.

Thu, Feb 25, 1:20 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight set the point value for T274542: Fork / subclass / include variations in BookletLayout for new VisualEditor template dialog to 3.

Thu, Feb 25, 1:18 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight created T275761: Remove the trash icons which show next to the parameter name when it's "in focus".

Thu, Feb 25, 1:16 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight updated the task description for T274542: Fork / subclass / include variations in BookletLayout for new VisualEditor template dialog.

Thu, Feb 25, 1:15 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight renamed T274542: Fork / subclass / include variations in BookletLayout for new VisualEditor template dialog from Fork or include variations in BookletLayout for new VisualEditor template dialog to Fork / subclass / include variations in BookletLayout for new VisualEditor template dialog.

Thu, Feb 25, 1:13 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight renamed T274542: Fork / subclass / include variations in BookletLayout for new VisualEditor template dialog from Fork BookletLayout for new VisualEditor template dialog to Fork or include variations in BookletLayout for new VisualEditor template dialog.

Thu, Feb 25, 1:13 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight set the point value for T274541: Adapt VisualEditor template dialog code to reduce duplication after fork to 5.

Thu, Feb 25, 1:11 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight updated the task description for T274541: Adapt VisualEditor template dialog code to reduce duplication after fork.

Thu, Feb 25, 1:05 PM · VisualEditor, VisualEditor-MediaWiki-Templates, WMDE-TechWish, WMDE-Templates-FocusArea

awight created T275757: Reportupdater output can be corrupted by hive logging.

Thu, Feb 25, 12:29 PM · Patch-For-Review, Analytics-Radar

awight added a project to T272569: Adjust edit count bucketing for TemplateData: Analytics-Radar.

Thu, Feb 25, 11:35 AM · Patch-For-Review, Analytics-Radar, WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), WMDE-Templates-FocusArea

awight added a comment to T273474: Adjust edit count bucketing for VisualEditor, segment all metrics.

Lots of errors in the logs:

Feb 25 11:00:01 an-launcher1002 systemd[1]: Started Report Updater job for visualeditor.
Feb 25 11:00:01 an-launcher1002 kerberos-run-command[20004]: User analytics executes as user analytics the command ['/usr/bin/python3', '/srv/reportupdater/reportupdater/update_reports.py', '-l', 'info', '/srv/r
Feb 25 11:00:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:00:01,595 - INFO - Starting execution.
Feb 25 11:00:02 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:00:02,701 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:01:33,558 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be written because of error: 'NoneType' object h
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 45, in run
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]:     self.write_results(header, updated_data, report, self.get_output_folder())
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 149, in write_results
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]:     row[0] = row[0].strftime(DATE_FORMAT)
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: AttributeError: 'NoneType' object has no attribute 'strftime'
Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:01:33,562 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:03:01,144 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data ' RecordReader initialized will read a total of 4740 records.' does not match format '%Y-%m-%d'
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:03:01,146 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:04:27,310 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'eader: RecordReader initialized will read a total of 2246 records.' does not match format '%Y-%m-%d'
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:04:27,311 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:06:15,394 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'rdReader: RecordReader initialized will read a total of 2501 records.' does not match format '%Y-%m-%d'
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:06:15,395 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:07:45,591 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'dReader: RecordReader initialized will read a total of 2226 records.' does not match format '%Y-%m-%d'
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:07:45,593 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:09:08,824 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'ader: RecordReader initialized will read a total of 5703 records.' does not match format '%Y-%m-%d'
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:09:08,825 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:11:31,594 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data ': RecordReader initialized will read a total of 5218 records.' does not match format '%Y-%m-%d'
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:11:31,595 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:13:00,614 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'RecordReader initialized will read a total of 2477 records.' does not match format '%Y-%m-%d'
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:13:00,615 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:14:32,318 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'eader: RecordReader initialized will read a total of 3800 records.' does not match format '%Y-%m-%d'
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:14:32,319 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:16:07,164 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:     normalized_date = datetime.strptime(raw_date, DATE_FORMAT)
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:     tt, fraction, gmtoff_fraction = _strptime(data_string, format)
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:   File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:     (data_string, format))
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'RecordReader initialized will read a total of 5762 records.' does not match format '%Y-%m-%d'
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred:
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last):
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]:     raise ValueError('Could not parse date from results.')
Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.

Thu, Feb 25, 11:26 AM · WMDE-TechWish-Sprint-2021-02-17, Unplanned-Sprint-Work, Patch-For-Review, MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), Analytics-Radar, WMDE-TechWish (Sprint-2021-01-20), WMDE-Templates-FocusArea

awight added a comment to T272569: Adjust edit count bucketing for TemplateData.

Error logs:

Feb 25 10:00:00 an-launcher1002 systemd[1]: Started Report Updater job for templatedata.
Feb 25 10:00:00 an-launcher1002 kerberos-run-command[11705]: User analytics executes as user analytics the command ['/usr/bin/python3', '/srv/reportupdater/reportupdater/update_reports.py', '-l', 'info', '/srv/r
Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,851 - INFO - Starting execution.
Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,908 - INFO - Executing "<Report key=dialog type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 start
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,701 - ERROR - Report "dialog" could not be executed because of error: object of type 'NoneType' has no len()
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last):
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]:     empty_row = [report.start] + [None] * (len(normalized_header) - 1)
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len()
Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,729 - INFO - Executing "<Report key=actions type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 star
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,827 - ERROR - Report "actions" could not be executed because of error: object of type 'NoneType' has no len()
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last):
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:     report.results = self.normalize_results(report, None, tsv_reader)
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:   File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]:     empty_row = [report.start] + [None] * (len(normalized_header) - 1)
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len()
Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,829 - INFO - Execution complete.
Feb 25 10:00:27 an-launcher1002 systemd[1]: reportupdater-templatedata.service: Succeeded.

Thu, Feb 25, 11:23 AM · Patch-For-Review, Analytics-Radar, WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), WMDE-Templates-FocusArea

awight updated subscribers of T273471: Adjust edit count bucketing for CodeMirror.

@JAllemandou has kindly copied the logged failures:

Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: 2021-02-25 10:56:50,391 - ERROR - Report "users_codemirror_and_wikitext" could not be written because of error: 'NoneType' object has no attribute
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: Traceback (most recent call last):
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]:   File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 45, in run
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]:     self.write_results(header, updated_data, report, self.get_output_folder())
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]:   File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 149, in write_results
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]:     row[0] = row[0].strftime(DATE_FORMAT)
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: AttributeError: 'NoneType' object has no attribute 'strftime'
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: 2021-02-25 10:56:50,395 - INFO - Executing "<Report key=users_codemirror_and_wikitext type=sql granularity=days lag=0 is_funnel=False first_date=2

Thu, Feb 25, 11:19 AM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), Analytics-Radar, WMDE-Templates-FocusArea

awight added a comment to T157145: Twitter cards don't work for any projects besides Wikidata.

Can we experiment more with og:type? Has anyone else done the reverse-engineering to find how these map to twitter:card?

Thu, Feb 25, 9:29 AM · MediaWiki-Core-Skin-Architecture, Readers-Web-Backlog (Kanbanana-FY-2020-21), MW-1.36-notes (1.36.0-wmf.32; 2021-02-23), MW-1.35-notes (1.35.0-wmf.21; 2020-02-25), Patch-For-Review, User-notice, User-Nikerabbit

Wed, Feb 24

awight committed rARPQ27875550b319: Update event bucketing for visualeditor events (authored by awight).

Update event bucketing for visualeditor events

Wed, Feb 24, 5:08 PM

awight committed rARPQ3dcf6a81dd50: Fix bad table name in query (authored by awight).

Fix bad table name in query

Wed, Feb 24, 5:08 PM

awight committed rARPQ532e92a54e42: Use the edit count bucket sent by TemplateData (authored by Andrew-WMDE).

Use the edit count bucket sent by TemplateData

Wed, Feb 24, 5:08 PM

awight claimed T273471: Adjust edit count bucketing for CodeMirror.

Wed, Feb 24, 12:33 PM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), Analytics-Radar, WMDE-Templates-FocusArea

awight moved T273471: Adjust edit count bucketing for CodeMirror from Sprint Backlog to Doing on the WMDE-TechWish-Sprint-2021-02-17 board.

Wed, Feb 24, 12:33 PM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), Analytics-Radar, WMDE-Templates-FocusArea

awight added a comment to T222200: Run browser tests as part of "npm test" for wikidata/query/gui in CI.

The latest patchset drops selenium-standalone.

Wed, Feb 24, 12:23 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), wdwb-tech-focus (backlog), Patch-For-Review, Wikimedia-Hackathon-2019, User-zeljkofilipin, Wikidata, Continuous-Integration-Infrastructure

Tue, Feb 23

awight moved T273471: Adjust edit count bucketing for CodeMirror from Watching (Stalled) to Sprint Backlog on the WMDE-TechWish-Sprint-2021-02-17 board.

All new segmentation is broken in various ways:

New preferences are not written to Graphite yet.
sessions byEditCount is all "anonymous" and "unknown".
New toggles are not written yet.

Tue, Feb 23, 10:38 PM · WMDE-TechWish-Sprint-2021-02-17, WMDE-TechWish (Sprint-2021-02-03), Patch-For-Review, MW-1.36-notes (1.36.0-wmf.29; 2021-02-02), Analytics-Radar, WMDE-Templates-FocusArea

awight added a comment to T275013: TwoColConflictConflict Event Platform Migration.