scripts
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Addshore
	Jan 29 2020, 12:18 PM

Description

https://www.php.net/manual/en/function.set-time-limit.php

The time limits should have sensible settings, probably based on the current execution times that can be found in the logs.

Details

	Subject	Repo	Branch	Lines +/-
	Add Time limit to scripts executed on stat1007	analytics/wmde/scripts	production	+15 -0
	Add Time limit to scripts executed on stat1007	analytics/wmde/scripts	master	+15 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		elukey	T243871 Long query running on dbstore1005:3318
		Resolved		Rosalie_WMDE	T243894 Add time limits to scripts executed on stat1007 as part of analytics/wmde/scripts

Event Timeline

Addshore triaged this task as Low priority.Jan 29 2020, 12:18 PM

Addshore created this task.

Addshore mentioned this in T243871: Long query running on dbstore1005:3318.

Addshore moved this task from Incoming to Prioritized Wikidata Tech Backlog (prioritised from top to bottom) on the Wikidata-Campsite board.

Addshore moved this task from incoming to consider for next sprint on the Wikidata board.Jan 29 2020, 12:20 PM

Addshore moved this task from Prioritized Wikidata Tech Backlog (prioritised from top to bottom) to Wikidata-Campsite-Iteration-∞ (On Hold) on the Wikidata-Campsite board.Jan 29 2020, 3:03 PM

Addshore edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.

Maybe we can have a general timeout using timeout bash command instead. Something like $timeout 3600 php foo.php &?

Ottomata moved this task from Incoming to Radar on the Analytics board.Feb 4 2020, 6:15 PM

Rosalie_WMDE claimed this task.Feb 12 2020, 3:51 PM

Rosalie_WMDE moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Feb 12 2020, 3:58 PM

So this is for weekly runs:

ladsgroup@stat1007:/srv/analytics-wmde/graphite/log$ tail weekly.log-20200217 --lines 500
+ '[' -z /srv/analytics-wmde/graphite/src/scripts ']'
+ date '+%F %T weekly.sh Started!'
2020-02-16 01:00:01 weekly.sh Started!
+ date '+%F %T weekly.sh Waiting!'
+ eval /srv/analytics-wmde/graphite/src/scripts/src/dbtables/slots.php
++ /srv/analytics-wmde/graphite/src/scripts/src/dbtables/slots.php
2020-02-16 01:00:01 weekly.sh Waiting!
+ wait
2020-02-16 01:00:01 dbtables-slots Script Started!
2020-02-16 01:00:01 dbtables-slots Slots DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
+ date '+%F %T weekly.sh Ended!'
2020-02-16 01:20:56 weekly.sh Ended!
0.07user 0.01system 20:54.94elapsed 0%CPU (0avgtext+0avgdata 27752maxresident)k
16inputs+32outputs (0major+5099minor)pagefaults 0swaps

On daily ones:

2020-02-11 03:00:01 wikidata-sparql-instanceof Script Started!
2020-02-11 03:00:01 wikidata-phabricatorTasks Script Started!
2020-02-11 03:00:01 wikidata-site_stats-lexemes Script Started!
2020-02-11 03:00:01 wikidata-site_stats-active_users Script Started!
2020-02-11 03:00:01 wikidata-entityUsage Script Started!
2020-02-11 03:00:01 wikidata-datamodel-properties_by_datatype Script Started!
2020-02-11 03:00:01 wikibase-dockerStats Script Started!
2020-02-11 03:00:01 wikibase-dockerStats Running for: wikibase/wikibase
2020-02-11 03:00:01 wikidata-showcaseItems Script Started!
2020-02-11 03:00:01 wikidata-sparql-ranks Script Started!
2020-02-11 03:00:01 wikidata-site_stats-total_pages Script Started!
2020-02-11 03:00:01 wikidata-datamodel-statements_per_entity Script Started!
2020-02-11 03:00:01 wikidata-dumpDownloads Script Started!
2020-02-11 03:00:01 wikidata-dumpDownloads Targeting date: 07/Feb/2020
2020-02-11 03:00:01 wikidata-social-twitter Script Started!
2020-02-11 03:00:01 wikidata-social-newsletter Script Started!
2020-02-11 03:00:01 betafeature-counts Script Started!
2020-02-11 03:00:01 wikidata-social-irc Script Started!
2020-02-11 03:00:01 wikidata-datamodel-sitelinks_per_site Script Started!
2020-02-11 03:00:01 wikidata-site_stats-total_edits Script Started!
2020-02-11 03:00:01 wikidata-site_stats-pages_by_namespace Script Started!
2020-02-11 03:00:01 wikidata-site_stats-page_size Script Started!
2020-02-11 03:00:01 rollbackconf-userprops Script Started!
2020-02-11 03:00:01 wikidata-social-techmail Script Started!
2020-02-11 03:00:01 advancedsearch-userprops Script Started!
2020-02-11 03:00:01 wikidata-site_stats-recent_changes_by_namespace Script Started!
2020-02-11 03:00:01 wikidata-site_stats-user_languages Script Started!
2020-02-11 03:00:01 wikidata-dumpScanProcessing Script Started!
2020-02-11 03:00:01 wikidata-sparql-constraint-snaks Script Started!
2020-02-11 03:00:01 wikidata-datamodel-sitelinks_per_item Script Started!
2020-02-11 03:00:01 wikidata-site_stats-good_articles Script Started!
2020-02-11 03:00:01 wikidata-dumpScanProcessing File not found: /srv/analytics-wmde/graphite/data/dumpfiles/metrics.json
2020-02-11 03:00:01 wikidata-site_stats-users Script Started!
2020-02-11 03:00:01 wikidata-social-identica Script Started!
2020-02-11 03:00:01 wikidata-site_stats-user_groups Script Started!
2020-02-11 03:00:01 wikidata-social-mail Script Started!
2020-02-11 03:00:01 revslider-userprops Script Started!
2020-02-11 03:00:01 wikidata-site_stats-pagelinks_to_namespaces Script Started!
2020-02-11 03:00:01 catwatch-userprops Script Started!
2020-02-11 03:00:01 wikidata-site_stats-rolling_rc Script Started!
2020-02-11 03:00:01 echo-statusNotifications Script Started!
2020-02-11 03:00:01 wikidata-social-facebook Script Started!
2020-02-11 03:00:01 wikidata-social-irc Script Finished!
2020-02-11 03:00:01 wikidata-site_stats-users Script Finished!
2020-02-11 03:00:01 wikidata-site_stats-total_pages Script Finished!
2020-02-11 03:00:01 wikidata-social-newsletter Script Finished!
2020-02-11 03:00:01 wikidata-showcaseItems Script Finished!
2020-02-11 03:00:01 wikidata-site_stats-user_groups Running query for admins group
2020-02-11 03:00:01 wikidata-site_stats-user_groups Running query for bureaucrats group
2020-02-11 03:00:01 wikidata-site_stats-good_articles Script Finished!
2020-02-11 03:00:01 wikidata-site_stats-user_groups Running query for bots group
2020-02-11 03:00:01 wikidata-site_stats-user_groups Running query for propertycreators group
2020-02-11 03:00:01 wikidata-site_stats-total_edits Script Finished!
2020-02-11 03:00:01 wikidata-site_stats-user_groups Script Finished!
2020-02-11 03:00:01 wikibase-dockerStats Sending data for: wikibase/wikibase
2020-02-11 03:00:01 wikidata-social-techmail Script Finished!
2020-02-11 03:00:01 wikibase-dockerStats Running for: wikibase/wdqs
2020-02-11 03:00:01 wikidata-datamodel-properties_by_datatype Script Finished!
2020-02-11 03:00:01 wikidata-social-mail Script Finished!
2020-02-11 03:00:01 wikibase-dockerStats Sending data for: wikibase/wdqs
2020-02-11 03:00:01 wikibase-dockerStats Running for: wikibase/wdqs-frontend
2020-02-11 03:00:01 wikibase-dockerStats Sending data for: wikibase/wdqs-frontend
2020-02-11 03:00:01 wikibase-dockerStats Running for: wikibase/wdqs-proxy
2020-02-11 03:00:01 wikidata-dumpDownloads Script Finished!
2020-02-11 03:00:01 wikibase-dockerStats Sending data for: wikibase/wdqs-proxy
2020-02-11 03:00:01 wikibase-dockerStats Script Finished!
2020-02-11 03:00:02 wikidata-social-facebook Script Finished!
2020-02-11 03:00:03 wikidata-sparql-constraint-snaks Script Finished!
2020-02-11 03:00:03 wikidata-phabricatorTasks Script Finished!
2020-02-11 03:00:03 wikidata-social-twitter Script Finished!
2020-02-11 03:00:03 wikidata-social-identica Script Finished!
2020-02-11 03:00:05 wikidata-site_stats-recent_changes_by_namespace Script Finished!
2020-02-11 03:00:07 wikidata-site_stats-lexemes Script Finished!
2020-02-11 03:00:47 wikidata-site_stats-rolling_rc Script Finished!
2020-02-11 03:00:56 wikidata-sparql-ranks Script Finished!
2020-02-11 03:01:28 wikidata-site_stats-active_users Script Finished!
2020-02-11 03:01:36 wikidata-site_stats-pages_by_namespace Script Finished!
2020-02-11 03:02:25 wikidata-datamodel-sitelinks_per_site Script Finished!
2020-02-11 03:02:38 wikidata-dumpScanProcessing Script Finished!
2020-02-11 03:02:41 wikidata-site_stats-page_size Script Finished!
2020-02-11 03:02:46 wikidata-datamodel-sitelinks_per_item Script Finished!
2020-02-11 03:04:18 wikidata-site_stats-user_languages Script Finished!
2020-02-11 03:12:14 wikidata-site_stats-pagelinks_to_namespaces Script Finished!
2020-02-11 03:14:25 wikidata-sparql-instanceof Script Finished!
2020-02-11 03:16:44 wikidata-datamodel-statements_per_entity Script Finished!
2020-02-11 03:49:11 wikidata-entityUsage Script Finished!

Side note, these are failing:

2020-02-11 03:00:01 betafeature-counts SELECT 1 failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 betafeature-counts SELECT user_name, up_property failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 revslider-userprops RevSlider DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 rollbackconf-userprops RollbackConf DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 echo-statusNotifications SELECT FAILED for # NOTE: This file is automatically generated. Do not try to edit it directly. for property echo-subscriptions-web-mention-failure, Skipping!!
2020-02-11 03:00:01 echo-statusNotifications SELECT FAILED for # NOTE: This file is automatically generated. Do not try to edit it directly. for property echo-subscriptions-web-mention-success, Skipping!!
2020-02-11 03:00:01 wikidata-entityUsage EntityUsage DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 wikidata-entityUsage EntityUsage page DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 catwatch-userprops CatWatch DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:00:01 advancedsearch-userprops AdvancedSearch DB query failed for # NOTE: This file is automatically generated. Do not try to edit it directly., Skipping!!
2020-02-11 03:02:38 wikidata-dumpScanProcessing File not found: /srv/analytics-wmde/graphite/data/20190729/metrics.json
2020-02-11 03:02:38 wikidata-dumpScanProcessing File not found: /srv/analytics-wmde/graphite/data/20190722/metrics.json
2020-02-11 03:02:38 wikidata-dumpScanProcessing File not found: /srv/analytics-wmde/graphite/data/20190717/metrics.json

Rosalie_WMDE added a comment.Feb 17 2020, 4:07 PM

This comment was removed by Rosalie_WMDE.

Rosalie_WMDE moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Feb 17 2020, 4:07 PM

Marostegui unsubscribed.Feb 17 2020, 4:09 PM

Change 572705 had a related patch set uploaded (by Rosalie Perside (WMDE); owner: Rosalie Perside (WMDE)):
[analytics/wmde/scripts@master] Add Time limit to scripts executed on stat1007

https://gerrit.wikimedia.org/r/572705

gerritbot added a project: Patch-For-Review.Feb 17 2020, 4:20 PM

Change 573211 had a related patch set uploaded (by Addshore; owner: Rosalie Perside (WMDE)):
[analytics/wmde/scripts@production] Add Time limit to scripts executed on stat1007

https://gerrit.wikimedia.org/r/573211

Change 572705 merged by jenkins-bot:
[analytics/wmde/scripts@master] Add Time limit to scripts executed on stat1007

https://gerrit.wikimedia.org/r/572705

Change 573211 merged by jenkins-bot:
[analytics/wmde/scripts@production] Add Time limit to scripts executed on stat1007

https://gerrit.wikimedia.org/r/573211

Moving to verification, and we should wait a couple of days to make sure dashboard continue to work :).

In the longer run we will want to move a bunch of these scripts to use things that are now provided in hadoop and also to use the "reportupdater" stuff provided by analytics.
Neither of which were a thing when these scripts were first written.

Addshore mentioned this in rADES39ad046fcc27: Add Time limit to scripts executed on stat1007.Feb 19 2020, 7:46 AM

Maintenance_bot removed a project: Patch-For-Review.Feb 19 2020, 8:10 AM

Addshore moved this task from consider for next sprint to in progress on the Wikidata board.Feb 20 2020, 9:02 AM

Addshore closed this task as Resolved.Mar 4 2020, 4:10 PM

Aklapper edited projects, added Analytics-Radar; removed Analytics.Jun 10 2020, 6:44 AM