User Details
- User Since
- Oct 12 2014, 9:02 PM (333 w, 1 d)
- Availability
- Available
- IRC Nick
- awight
- LDAP User
- Awight
- MediaWiki User
- Adamw [ Global Accounts ]
Yesterday
I believe it's still a god idea to list all exceptions in https://gerrit.wikimedia.org/r/663185, even for wikis where it currently can't have an effect.
Fri, Feb 26
Another detail to mention: the output writer currently includes a date column, and I believe that removing it would cause the header change detection to invalidate old results. Maybe we add a date column back into the results before writing?
Since we have control over all jobs using this tool, I think we can move quickly with the migration. It's still nice to include a soft cutover, in case of rollback, etc. My thought is:
- Phase 1: RU checks whether the first output column header is date. If so, logic is enabled to strip the value during import.
- Phase 2: Once RU with soft migration has been deployed for a few days without errors, then we remove the date column from all reports.
- Phase 3: At our leisure, remove the soft migration code from RU.
This is unblocked now. Moving into the sprint because it's related to the other Grafana board changes.
The metrics are landing in Graphite, but we're still waiting for the updated bucket labels.
Thu, Feb 25
Sorry I wasn't aware of this task until now, and may have duplicated some work. In patch aca489f139cd13df, I added code to the EventLogging extension which makes wgUserEditCountBucket available from JS, and EventLoggingServices::getInstance()->getUserBucketProvider()->getUserEditCountBucket( $user ) from the backend.
Playing with logging is uncovering more errors. When run like hive --hiveconf hive.root.logger=ERROR,console, you can see each of our boilerplate cast(split('2021-02-11', '-')[2] as int) calls included in every reportupdater query actually cause a shower of these internal errors:
2021-02-25T14:29:03,877 ERROR [e2497dad-77df-493c-ad49-f3ea5cc83a38 main] optimizer.ConstantPropagateProcFactory: Unable to evaluate org.apache.hadoop.hive.ql.udf.generic.GenericUDFSplit@3f81621c. Return value unrecoginizable.
A related issue is that SL4J is complaining, and I believe this topic is unmaskable without changing config. Can we fix the root cause?
SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Lots of errors in the logs:
Feb 25 11:00:01 an-launcher1002 systemd[1]: Started Report Updater job for visualeditor. Feb 25 11:00:01 an-launcher1002 kerberos-run-command[20004]: User analytics executes as user analytics the command ['/usr/bin/python3', '/srv/reportupdater/reportupdater/update_reports.py', '-l', 'info', '/srv/r Feb 25 11:00:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:00:01,595 - INFO - Starting execution. Feb 25 11:00:02 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:00:02,701 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:01:33,558 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be written because of error: 'NoneType' object h Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 45, in run Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: self.write_results(header, updated_data, report, self.get_output_folder()) Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 149, in write_results Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: row[0] = row[0].strftime(DATE_FORMAT) Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: AttributeError: 'NoneType' object has no attribute 'strftime' Feb 25 11:01:33 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:01:33,562 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:03:01,144 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data ' RecordReader initialized will read a total of 4740 records.' does not match format '%Y-%m-%d' Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:03:01 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:03:01,146 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:04:27,310 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'eader: RecordReader initialized will read a total of 2246 records.' does not match format '%Y-%m-%d' Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:04:27 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:04:27,311 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:06:15,394 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'rdReader: RecordReader initialized will read a total of 2501 records.' does not match format '%Y-%m-%d' Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:06:15 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:06:15,395 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:07:45,591 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'dReader: RecordReader initialized will read a total of 2226 records.' does not match format '%Y-%m-%d' Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:07:45 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:07:45,593 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:09:08,824 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'ader: RecordReader initialized will read a total of 5703 records.' does not match format '%Y-%m-%d' Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:09:08 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:09:08,825 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:11:31,594 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data ': RecordReader initialized will read a total of 5218 records.' does not match format '%Y-%m-%d' Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:11:31 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:11:31,595 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:13:00,614 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'RecordReader initialized will read a total of 2477 records.' does not match format '%Y-%m-%d' Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:13:00 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:13:00,615 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:14:32,318 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'eader: RecordReader initialized will read a total of 3800 records.' does not match format '%Y-%m-%d' Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results. Feb 25 11:14:32 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:14:32,319 - INFO - Executing "<Report key=template_dialog_parameters_by_edit_success type=script granularity=days lag=18000 is_fun Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: 2021-02-25 11:16:07,164 - ERROR - Report "template_dialog_parameters_by_edit_success" could not be executed because of error: Could not parse da Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 156, in normalize_results Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: normalized_date = datetime.strptime(raw_date, DATE_FORMAT) Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 577, in _strptime_datetime Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: tt, fraction, gmtoff_fraction = _strptime(data_string, format) Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: File "/usr/lib/python3.7/_strptime.py", line 359, in _strptime Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: (data_string, format)) Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: time data 'RecordReader initialized will read a total of 5762 records.' does not match format '%Y-%m-%d' Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: During handling of the above exception, another exception occurred: Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: Traceback (most recent call last): Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 158, in normalize_results Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: raise ValueError('Could not parse date from results.') Feb 25 11:16:07 an-launcher1002 reportupdater-visualeditor[20004]: ValueError: Could not parse date from results.
Error logs:
Feb 25 10:00:00 an-launcher1002 systemd[1]: Started Report Updater job for templatedata. Feb 25 10:00:00 an-launcher1002 kerberos-run-command[11705]: User analytics executes as user analytics the command ['/usr/bin/python3', '/srv/reportupdater/reportupdater/update_reports.py', '-l', 'info', '/srv/r Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,851 - INFO - Starting execution. Feb 25 10:00:00 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:00,908 - INFO - Executing "<Report key=dialog type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 start Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,701 - ERROR - Report "dialog" could not be executed because of error: object of type 'NoneType' has no len() Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last): Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: empty_row = [report.start] + [None] * (len(normalized_header) - 1) Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len() Feb 25 10:00:15 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:15,729 - INFO - Executing "<Report key=actions type=script granularity=days lag=18000 is_funnel=True first_date=2020-12-14 star Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,827 - ERROR - Report "actions" could not be executed because of error: object of type 'NoneType' has no len() Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: Traceback (most recent call last): Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 131, in execute_script_report Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: report.results = self.normalize_results(report, None, tsv_reader) Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: File "/srv/reportupdater/reportupdater/reportupdater/executor.py", line 171, in normalize_results Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: empty_row = [report.start] + [None] * (len(normalized_header) - 1) Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: TypeError: object of type 'NoneType' has no len() Feb 25 10:00:27 an-launcher1002 reportupdater-templatedata[11705]: 2021-02-25 10:00:27,829 - INFO - Execution complete. Feb 25 10:00:27 an-launcher1002 systemd[1]: reportupdater-templatedata.service: Succeeded.
@JAllemandou has kindly copied the logged failures:
Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: 2021-02-25 10:56:50,391 - ERROR - Report "users_codemirror_and_wikitext" could not be written because of error: 'NoneType' object has no attribute Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: Traceback (most recent call last): Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 45, in run Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: self.write_results(header, updated_data, report, self.get_output_folder()) Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: File "/srv/reportupdater/reportupdater/reportupdater/writer.py", line 149, in write_results Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: row[0] = row[0].strftime(DATE_FORMAT) Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: AttributeError: 'NoneType' object has no attribute 'strftime' Feb 25 10:56:50 an-launcher1002 reportupdater-codemirror[30844]: 2021-02-25 10:56:50,395 - INFO - Executing "<Report key=users_codemirror_and_wikitext type=sql granularity=days lag=0 is_funnel=False first_date=2
Can we experiment more with og:type? Has anyone else done the reverse-engineering to find how these map to twitter:card?
Wed, Feb 24
The latest patchset drops selenium-standalone.
Tue, Feb 23
All new segmentation is broken in various ways:
- New preferences are not written to Graphite yet.
- sessions byEditCount is all "anonymous" and "unknown".
- New toggles are not written yet.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Events successfully received from testwiki.
Was blocking a sprint task, so I've pulled in and finished the patch.
Notes:
- The wdio-chromedriver-service seems better from a distance, but the antagonist seems to be in the details. See this commit for some notes about why the approach was abandoned in mw-core.
- The container already includes a /run-with-xvfb.sh, ideally we can drop ./selenium.sh and set the container entrypoint to start the services, then run npm run browser_test directly.
- I guess the simplest fix for the grunt-merge-i18n dependency is to upload it to npm?
I mashed together some bits laying around and got the above patch to pass. The only big issue is the grunt-merge-i18n package coming from github, which breaks due to missing ssh. Otherwise, I think this is usable.
Good news, we made the branch cut! Thiemo's patch will automatically go out with MW-1.36-notes (1.36.0-wmf.32; 2021-02-23), this week.
Mon, Feb 22
Quibble runs some services in the background, which you would also need in order to run browser tests. Specifically I think you're missing:
- xvfb in order to run headless, like xvfb :94 -screen 0 1280x1024x24 -nolisten tcp -nolisten unix
- chromedriver, either the npm driver or standalone: chromedriver --port=4444 --url-base=/wd/hub
Thu, Feb 18
Wed, Feb 17
Turns out these all have a browser-family "Web-Crawler", nothing we need to care about.