Page MenuHomePhabricator
Paste P2377

nodepool errors 2015-12-06
ActivePublic

Authored by hashar on Dec 7 2015, 3:07 PM.
Tags
None
Referenced Files
F3056299: nodepool errors 2015-12-06
Feb 24 2016, 5:27 AM
Subscribers
None
hashar@labnodepool1001:/var/log/nodepool$ grep -h ERROR debug.log.2015-12-06*
2015-12-06 08:00:30,030 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 08:11:40,142 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 08:22:23,287 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 08:33:22,133 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 08:44:25,374 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 08:55:37,121 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 09:06:22,673 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 09:17:23,313 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 09:28:54,441 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 09:39:37,792 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 09:50:24,027 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:01:23,846 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:13:06,130 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:24:24,131 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:35:22,464 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:46:22,697 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 10:57:23,262 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 11:08:23,106 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 11:19:22,696 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 11:30:23,200 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 11:41:23,226 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 11:52:24,065 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:03:23,180 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:14:23,363 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:25:23,356 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:36:23,104 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:47:23,701 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 12:58:23,181 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 13:09:22,739 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 13:20:23,121 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 13:31:23,658 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 13:42:22,558 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 13:53:23,844 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:04:22,385 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:12:22,242 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 14:13:03,837 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 14:13:18,223 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 14:15:26,324 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:20:23,574 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 14:23:32,828 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 14:24:29,299 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 14:24:49,743 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 14:26:33,892 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:31:35,324 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 14:34:32,921 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 14:35:25,390 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 14:35:46,486 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 14:37:28,380 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:42:35,520 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 14:43:07,780 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 14:43:43,878 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10711 in provider: wmflabs-eqiad error:
2015-12-06 14:44:04,357 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10712 in provider: wmflabs-eqiad error:
2015-12-06 14:44:34,834 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10713 in provider: wmflabs-eqiad error:
2015-12-06 14:45:06,189 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10714 in provider: wmflabs-eqiad error:
2015-12-06 14:45:26,729 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10715 in provider: wmflabs-eqiad error:
2015-12-06 14:45:29,559 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 14:45:47,162 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10716 in provider: wmflabs-eqiad error:
2015-12-06 14:46:07,540 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10717 in provider: wmflabs-eqiad error:
2015-12-06 14:46:32,849 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 14:46:38,142 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10718 in provider: wmflabs-eqiad error:
2015-12-06 14:46:53,844 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 14:47:26,328 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 14:48:26,364 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 14:48:31,178 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10720 in provider: wmflabs-eqiad error:
2015-12-06 14:48:38,401 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 14:49:18,414 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 14:53:34,343 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 14:54:57,068 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 14:56:49,039 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 14:57:30,768 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 14:57:40,731 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 14:58:54,305 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 14:59:24,334 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 14:59:44,873 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:00:46,802 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:02:12,073 ERROR nodepool.NodeLauncher: Exception launching node id: 10705 in provider: wmflabs-eqiad error:
2015-12-06 15:02:12,076 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:04:33,056 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 15:05:45,442 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 15:05:48,334 ERROR nodepool.NodeLauncher: Exception launching node id: 10706 in provider: wmflabs-eqiad error:
2015-12-06 15:05:48,335 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:07:36,937 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 15:08:27,623 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 15:08:59,435 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 15:09:39,396 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 15:10:21,589 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 15:10:41,131 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:11:33,127 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:15:01,581 ERROR nodepool.SnapshotImageUpdater: Exception updating image ci-jessie-wikimedia in wmflabs-eqiad:
2015-12-06 15:15:01,583 ERROR nodepool.NodePool: Could not delete image 385
2015-12-06 15:15:29,524 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 15:16:34,315 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 15:18:28,346 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 15:19:18,260 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 15:19:40,675 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 15:20:30,083 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 15:21:22,323 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 15:21:42,122 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:22:35,557 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:26:29,632 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 15:27:31,873 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 15:28:20,701 ERROR nodepool.NodeLauncher: Exception launching node id: 10707 in provider: wmflabs-eqiad error:
2015-12-06 15:28:20,703 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:29:34,743 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 15:30:25,748 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 15:30:46,905 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 15:31:26,662 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 15:32:19,170 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 15:32:39,997 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:33:29,503 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:35:41,597 ERROR nodepool.NodeLauncher: Exception launching node id: 10708 in provider: wmflabs-eqiad error:
2015-12-06 15:35:41,599 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:35:51,474 ERROR nodepool.NodeLauncher: Exception launching node id: 10709 in provider: wmflabs-eqiad error:
2015-12-06 15:35:51,477 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:37:37,809 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 15:38:28,567 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 15:40:33,615 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 15:41:26,700 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 15:41:46,243 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 15:42:26,271 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 15:43:28,601 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 15:43:48,375 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:44:30,271 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:48:22,844 ERROR nodepool.NodeLauncher: Exception launching node id: 10719 in provider: wmflabs-eqiad error:
2015-12-06 15:48:22,847 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 15:48:32,979 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 15:49:33,299 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 15:51:35,610 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 15:52:18,020 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 15:52:38,953 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 15:53:30,218 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 15:54:22,008 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 15:54:42,895 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 15:55:44,159 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 15:59:44,359 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:00:34,364 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:02:29,365 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:03:20,846 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 16:03:40,693 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:04:32,949 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:05:22,711 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 16:05:44,480 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 16:06:36,565 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 16:10:31,185 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:11:31,235 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:13:34,724 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:14:24,655 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 16:14:46,627 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:15:28,304 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:16:19,583 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 16:16:41,101 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 16:17:32,632 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 16:21:26,378 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:22:29,173 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:24:29,803 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:25:23,360 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 16:25:43,226 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:26:33,644 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:26:44,539 ERROR nodepool.NodeLauncher: Exception launching node id: 10723 in provider: wmflabs-eqiad error:
2015-12-06 16:26:44,541 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 16:27:46,982 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 16:27:55,722 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 16:28:29,140 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 16:32:33,445 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:33:34,539 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:35:36,407 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:36:26,718 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 16:36:38,651 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:36:49,152 ERROR nodepool.NodeLauncher: Exception launching node id: 10726 in provider: wmflabs-eqiad error:
2015-12-06 16:36:49,155 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 16:37:45,031 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:38:25,878 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 16:38:46,498 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 16:39:27,152 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 16:43:23,087 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:44:42,849 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:46:27,429 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:47:27,641 ERROR nodepool.NodeLauncher: Exception launching node id: 10729 in provider: wmflabs-eqiad error:
2015-12-06 16:47:27,644 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 16:47:41,838 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:48:13,902 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 16:48:43,544 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:48:50,837 ERROR nodepool.NodeLauncher: Exception launching node id: 10730 in provider: wmflabs-eqiad error:
2015-12-06 16:48:50,840 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 16:49:24,642 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 16:49:46,394 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 16:50:26,578 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 16:54:30,715 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 16:54:36,491 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10744 in provider: wmflabs-eqiad error:
2015-12-06 16:55:06,929 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10745 in provider: wmflabs-eqiad error:
2015-12-06 16:55:33,097 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 16:57:36,074 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 16:59:01,624 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 16:59:43,282 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 16:59:53,681 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:00:37,621 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:00:59,203 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:01:39,875 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:06:10,036 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 17:06:40,464 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 17:08:32,344 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 17:10:26,012 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 17:10:35,820 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:11:02,089 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 17:11:26,106 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:11:47,639 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:12:28,630 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:17:24,784 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 17:17:47,069 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 17:19:29,433 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 17:21:21,495 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:21:42,211 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 17:22:25,155 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 17:22:35,798 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:22:56,056 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:23:37,431 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:28:23,947 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 17:28:45,022 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 17:30:28,184 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 17:30:32,956 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10747 in provider: wmflabs-eqiad error:
2015-12-06 17:30:53,471 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10748 in provider: wmflabs-eqiad error:
2015-12-06 17:31:24,105 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10749 in provider: wmflabs-eqiad error:
2015-12-06 17:31:54,988 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10750 in provider: wmflabs-eqiad error:
2015-12-06 17:32:15,482 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10751 in provider: wmflabs-eqiad error:
2015-12-06 17:32:19,288 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:32:35,882 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10752 in provider: wmflabs-eqiad error:
2015-12-06 17:32:40,452 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 17:32:56,324 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10753 in provider: wmflabs-eqiad error:
2015-12-06 17:33:22,873 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 17:33:32,597 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:33:37,250 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10754 in provider: wmflabs-eqiad error:
2015-12-06 17:33:53,112 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:34:08,548 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10755 in provider: wmflabs-eqiad error:
2015-12-06 17:34:34,902 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:35:01,101 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10756 in provider: wmflabs-eqiad error:
2015-12-06 17:35:53,008 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10757 in provider: wmflabs-eqiad error:
2015-12-06 17:36:24,379 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10758 in provider: wmflabs-eqiad error:
2015-12-06 17:39:21,227 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 17:39:43,589 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 17:41:46,623 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 17:43:27,420 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:43:59,743 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 17:44:39,910 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 17:44:52,500 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:45:23,410 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:45:43,555 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:50:21,469 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 17:50:42,427 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 17:52:35,524 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 17:53:24,575 ERROR nodepool.NodeLauncher: Exception launching node id: 10736 in provider: wmflabs-eqiad error:
2015-12-06 17:53:24,577 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 17:53:53,514 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10760 in provider: wmflabs-eqiad error:
2015-12-06 17:53:56,464 ERROR nodepool.NodeLauncher: Exception launching node id: 10740 in provider: wmflabs-eqiad error:
2015-12-06 17:53:56,467 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 17:54:16,354 ERROR nodepool.NodeLauncher: Exception launching node id: 10742 in provider: wmflabs-eqiad error:
2015-12-06 17:54:16,357 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 17:54:17,933 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 17:54:38,716 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 17:55:20,378 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 17:55:40,225 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 17:55:49,497 ERROR nodepool.NodeLauncher: Exception launching node id: 10746 in provider: wmflabs-eqiad error:
2015-12-06 17:55:49,500 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 17:56:23,411 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 17:56:42,822 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 17:56:49,050 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10764 in provider: wmflabs-eqiad error:
2015-12-06 18:01:22,936 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:01:40,355 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:03:33,104 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 18:05:36,681 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 18:06:40,079 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 18:07:00,048 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 18:07:21,010 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 18:07:50,506 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10766 in provider: wmflabs-eqiad error:
2015-12-06 18:08:07,471 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 18:08:10,972 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10767 in provider: wmflabs-eqiad error:
2015-12-06 18:08:27,019 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 18:09:02,696 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10768 in provider: wmflabs-eqiad error:
2015-12-06 18:09:43,461 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10769 in provider: wmflabs-eqiad error:
2015-12-06 18:10:14,872 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10770 in provider: wmflabs-eqiad error:
2015-12-06 18:10:35,430 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10771 in provider: wmflabs-eqiad error:
2015-12-06 18:10:56,040 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10772 in provider: wmflabs-eqiad error:
2015-12-06 18:11:16,544 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10773 in provider: wmflabs-eqiad error:
2015-12-06 18:11:47,056 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10774 in provider: wmflabs-eqiad error:
2015-12-06 18:12:17,680 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10775 in provider: wmflabs-eqiad error:
2015-12-06 18:12:23,130 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:12:43,373 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:12:59,435 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10777 in provider: wmflabs-eqiad error:
2015-12-06 18:13:30,056 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10778 in provider: wmflabs-eqiad error:
2015-12-06 18:14:11,629 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10779 in provider: wmflabs-eqiad error:
2015-12-06 18:14:32,136 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10780 in provider: wmflabs-eqiad error:
2015-12-06 18:14:36,047 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 18:14:52,552 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10781 in provider: wmflabs-eqiad error:
2015-12-06 18:15:22,981 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10782 in provider: wmflabs-eqiad error:
2015-12-06 18:15:53,630 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10783 in provider: wmflabs-eqiad error:
2015-12-06 18:16:14,010 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10784 in provider: wmflabs-eqiad error:
2015-12-06 18:16:29,365 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 18:16:34,488 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10785 in provider: wmflabs-eqiad error:
2015-12-06 18:16:54,988 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10786 in provider: wmflabs-eqiad error:
2015-12-06 18:17:15,421 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10787 in provider: wmflabs-eqiad error:
2015-12-06 18:17:30,116 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 18:17:45,871 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10788 in provider: wmflabs-eqiad error:
2015-12-06 18:18:16,629 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10789 in provider: wmflabs-eqiad error:
2015-12-06 18:18:32,190 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 18:18:52,899 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 18:18:58,165 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10790 in provider: wmflabs-eqiad error:
2015-12-06 18:19:34,687 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 18:20:06,468 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 18:23:27,317 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10792 in provider: wmflabs-eqiad error:
2015-12-06 18:23:40,790 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:24:01,600 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:25:44,973 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 18:27:57,960 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 18:28:39,612 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 18:29:27,950 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10795 in provider: wmflabs-eqiad error:
2015-12-06 18:29:52,276 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 18:30:09,727 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10796 in provider: wmflabs-eqiad error:
2015-12-06 18:30:13,666 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 18:30:43,943 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 18:31:34,899 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 18:34:28,792 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:34:59,173 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10798 in provider: wmflabs-eqiad error:
2015-12-06 18:35:30,632 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:36:32,717 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 18:38:46,585 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 18:40:21,156 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 18:40:30,013 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10802 in provider: wmflabs-eqiad error:
2015-12-06 18:41:00,531 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10803 in provider: wmflabs-eqiad error:
2015-12-06 18:41:14,789 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 18:41:35,075 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 18:41:54,923 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 18:42:37,114 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 18:45:41,206 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:46:31,752 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:47:43,562 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 18:49:27,556 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 18:52:02,584 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 18:52:34,502 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 18:52:44,452 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 18:53:06,446 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 18:53:46,628 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 18:55:47,802 ERROR nodepool.NodeLauncher: Exception launching node id: 10761 in provider: wmflabs-eqiad error:
2015-12-06 18:55:47,803 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 18:56:30,510 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 18:57:31,620 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 18:57:46,326 ERROR nodepool.NodeLauncher: Exception launching node id: 10765 in provider: wmflabs-eqiad error:
2015-12-06 18:57:46,328 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 18:58:10,532 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10806 in provider: wmflabs-eqiad error:
2015-12-06 18:58:34,406 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:00:36,746 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:03:20,858 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:03:30,888 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:03:50,979 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:04:23,352 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 19:04:43,276 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 19:07:29,868 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 19:08:33,476 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 19:09:37,860 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:11:29,982 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:12:40,500 ERROR nodepool.NodeLauncher: Exception launching node id: 10776 in provider: wmflabs-eqiad error:
2015-12-06 19:12:40,502 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:14:22,625 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:14:35,002 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:14:55,077 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:15:26,648 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 19:15:46,334 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 19:18:31,903 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 19:19:32,920 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 19:19:40,924 ERROR nodepool.NodeLauncher: Exception launching node id: 10791 in provider: wmflabs-eqiad error:
2015-12-06 19:19:40,927 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:20:35,112 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:22:26,247 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:25:25,479 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:25:37,684 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:25:58,654 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:26:29,086 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 19:26:50,395 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 19:29:19,382 ERROR nodepool.NodeLauncher: Exception launching node id: 10794 in provider: wmflabs-eqiad error:
2015-12-06 19:29:19,385 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:29:32,841 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 19:29:43,019 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10810 in provider: wmflabs-eqiad error:
2015-12-06 19:30:53,369 ERROR nodepool.NodeLauncher: Exception launching node id: 10797 in provider: wmflabs-eqiad error:
2015-12-06 19:30:53,371 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:30:59,639 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 19:31:40,635 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10812 in provider: wmflabs-eqiad error:
2015-12-06 19:31:41,816 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:33:34,329 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:36:17,977 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:36:38,563 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:36:52,655 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:37:19,182 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 19:37:40,247 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 19:40:09,777 ERROR nodepool.NodeLauncher: Exception launching node id: 10800 in provider: wmflabs-eqiad error:
2015-12-06 19:40:09,780 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:40:37,305 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10814 in provider: wmflabs-eqiad error:
2015-12-06 19:41:00,943 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 19:41:30,825 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 19:42:54,354 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10816 in provider: wmflabs-eqiad error:
2015-12-06 19:43:06,595 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:43:14,831 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10817 in provider: wmflabs-eqiad error:
2015-12-06 19:43:35,488 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10818 in provider: wmflabs-eqiad error:
2015-12-06 19:44:27,381 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:44:57,821 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10820 in provider: wmflabs-eqiad error:
2015-12-06 19:46:31,171 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10822 in provider: wmflabs-eqiad error:
2015-12-06 19:47:01,686 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10823 in provider: wmflabs-eqiad error:
2015-12-06 19:47:21,942 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:47:32,439 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10824 in provider: wmflabs-eqiad error:
2015-12-06 19:47:33,573 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:47:52,670 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:48:25,488 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 19:48:45,297 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10826 in provider: wmflabs-eqiad error:
2015-12-06 19:48:45,595 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 19:49:16,594 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10827 in provider: wmflabs-eqiad error:
2015-12-06 19:49:57,286 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10828 in provider: wmflabs-eqiad error:
2015-12-06 19:50:28,716 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10829 in provider: wmflabs-eqiad error:
2015-12-06 19:50:49,292 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10830 in provider: wmflabs-eqiad error:
2015-12-06 19:51:09,833 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10831 in provider: wmflabs-eqiad error:
2015-12-06 19:51:30,329 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10832 in provider: wmflabs-eqiad error:
2015-12-06 19:51:51,108 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10833 in provider: wmflabs-eqiad error:
2015-12-06 19:52:11,564 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10834 in provider: wmflabs-eqiad error:
2015-12-06 19:52:23,003 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 19:52:44,813 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 19:54:25,086 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10836 in provider: wmflabs-eqiad error:
2015-12-06 19:54:27,493 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 19:54:45,477 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10837 in provider: wmflabs-eqiad error:
2015-12-06 19:55:05,864 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10838 in provider: wmflabs-eqiad error:
2015-12-06 19:55:28,993 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 19:55:36,303 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10839 in provider: wmflabs-eqiad error:
2015-12-06 19:56:07,095 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10840 in provider: wmflabs-eqiad error:
2015-12-06 19:56:30,232 ERROR nodepool.NodeLauncher: Exception launching node id: 10805 in provider: wmflabs-eqiad error:
2015-12-06 19:56:30,235 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:56:37,523 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10841 in provider: wmflabs-eqiad error:
2015-12-06 19:58:24,247 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 19:58:35,653 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 19:58:55,006 ERROR nodepool.NodeLauncher: Exception launching node id: 10807 in provider: wmflabs-eqiad error:
2015-12-06 19:58:55,008 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 19:59:16,402 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 19:59:45,087 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10844 in provider: wmflabs-eqiad error:
2015-12-06 19:59:47,759 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:00:08,947 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:03:33,850 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 20:03:55,866 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:05:48,316 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 20:06:49,980 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 20:09:24,768 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 20:09:36,729 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 20:10:39,496 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 20:11:00,487 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:11:41,127 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:13:22,686 ERROR nodepool.NodeLauncher: Exception launching node id: 10808 in provider: wmflabs-eqiad error:
2015-12-06 20:13:22,688 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:14:25,087 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 20:14:45,012 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:16:27,325 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 20:17:38,171 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 20:20:25,485 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 20:20:29,277 ERROR nodepool.NodeLauncher: Exception launching node id: 10809 in provider: wmflabs-eqiad error:
2015-12-06 20:20:29,280 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:20:45,712 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 20:21:37,663 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 20:22:17,356 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:22:38,066 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:25:24,873 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 20:25:45,763 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:27:27,565 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 20:28:28,591 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 20:30:17,862 ERROR nodepool.NodeLauncher: Exception launching node id: 10811 in provider: wmflabs-eqiad error:
2015-12-06 20:30:17,865 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:31:36,542 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 20:31:57,640 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 20:32:14,695 ERROR nodepool.NodeLauncher: Exception launching node id: 10813 in provider: wmflabs-eqiad error:
2015-12-06 20:32:14,698 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:32:36,720 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 20:33:28,462 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:33:40,984 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:36:22,398 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 20:36:45,087 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:38:35,803 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 20:39:28,707 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 20:41:12,747 ERROR nodepool.NodeLauncher: Exception launching node id: 10815 in provider: wmflabs-eqiad error:
2015-12-06 20:41:12,749 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:42:22,665 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 20:42:43,811 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 20:43:37,465 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 20:44:19,827 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:44:40,744 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:47:22,498 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 20:47:44,356 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:47:54,857 ERROR nodepool.NodeLauncher: Exception launching node id: 10825 in provider: wmflabs-eqiad error:
2015-12-06 20:47:54,860 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:49:37,103 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 20:50:27,339 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 20:53:23,205 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 20:53:45,227 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 20:54:35,586 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 20:55:27,896 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 20:55:47,361 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 20:58:03,539 ERROR nodepool.NodeLauncher: Exception launching node id: 10843 in provider: wmflabs-eqiad error:
2015-12-06 20:58:03,540 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 20:58:22,020 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 20:59:02,109 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:00:18,934 ERROR nodepool.NodeLauncher: Exception launching node id: 10845 in provider: wmflabs-eqiad error:
2015-12-06 21:00:18,935 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 21:00:36,793 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:01:28,001 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 21:04:21,087 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 21:04:42,058 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 21:05:32,918 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 21:06:23,907 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 21:06:58,404 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 21:09:31,071 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 21:10:32,480 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:11:56,358 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:12:36,637 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 21:16:15,624 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 21:16:37,161 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 21:16:47,603 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 21:17:17,415 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 21:17:39,404 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 21:20:33,611 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 21:21:35,990 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:22:36,492 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:23:26,966 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 21:27:22,828 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 21:27:34,909 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 21:27:54,929 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 21:28:26,665 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 21:28:47,645 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 21:31:30,620 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 21:32:32,436 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:33:34,315 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:34:35,297 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 21:38:21,202 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 21:38:32,348 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 21:38:54,541 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 21:39:24,433 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 21:39:45,064 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 21:42:29,652 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 21:43:31,104 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:44:33,431 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:45:09,157 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10860 in provider: wmflabs-eqiad error:
2015-12-06 21:45:35,015 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 21:45:49,945 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10861 in provider: wmflabs-eqiad error:
2015-12-06 21:46:10,577 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10862 in provider: wmflabs-eqiad error:
2015-12-06 21:46:51,459 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10863 in provider: wmflabs-eqiad error:
2015-12-06 21:49:19,826 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 21:49:31,671 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 21:49:51,623 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 21:50:21,517 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 21:50:44,342 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 21:53:28,098 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 21:54:29,833 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 21:55:41,327 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 21:56:31,118 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:00:19,265 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:00:35,935 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10865 in provider: wmflabs-eqiad error:
2015-12-06 22:00:48,159 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:00:50,326 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:00:56,384 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10866 in provider: wmflabs-eqiad error:
2015-12-06 22:01:17,052 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10867 in provider: wmflabs-eqiad error:
2015-12-06 22:01:22,947 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:01:43,178 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:02:08,102 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10868 in provider: wmflabs-eqiad error:
2015-12-06 22:02:59,859 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10869 in provider: wmflabs-eqiad error:
2015-12-06 22:03:30,626 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10870 in provider: wmflabs-eqiad error:
2015-12-06 22:03:51,071 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10871 in provider: wmflabs-eqiad error:
2015-12-06 22:04:11,787 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10872 in provider: wmflabs-eqiad error:
2015-12-06 22:04:21,821 ERROR nodepool.NodeLauncher: Exception launching node id: 10851 in provider: wmflabs-eqiad error:
2015-12-06 22:04:21,823 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 22:04:27,220 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 22:04:32,213 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10873 in provider: wmflabs-eqiad error:
2015-12-06 22:05:13,848 ERROR nodepool.NodeLauncher: Exception launching node id: 10856 in provider: wmflabs-eqiad error:
2015-12-06 22:05:13,851 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 22:05:23,813 ERROR nodepool.NodeLauncher: Exception launching node id: 10857 in provider: wmflabs-eqiad error:
2015-12-06 22:05:23,815 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 22:05:28,562 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 22:07:32,239 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:07:38,725 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 22:07:57,119 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10878 in provider: wmflabs-eqiad error:
2015-12-06 22:08:38,623 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10879 in provider: wmflabs-eqiad error:
2015-12-06 22:09:10,005 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10880 in provider: wmflabs-eqiad error:
2015-12-06 22:09:30,413 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10881 in provider: wmflabs-eqiad error:
2015-12-06 22:09:50,765 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10882 in provider: wmflabs-eqiad error:
2015-12-06 22:10:11,134 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10883 in provider: wmflabs-eqiad error:
2015-12-06 22:10:31,542 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10884 in provider: wmflabs-eqiad error:
2015-12-06 22:10:52,041 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10885 in provider: wmflabs-eqiad error:
2015-12-06 22:11:12,481 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10886 in provider: wmflabs-eqiad error:
2015-12-06 22:11:39,134 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:11:42,971 ERROR nodepool.NodeLauncher: Forbidden launching node id: 10887 in provider: wmflabs-eqiad error:
2015-12-06 22:11:48,770 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:12:26,183 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:12:57,393 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:13:10,765 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:15:36,924 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 22:16:31,667 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 22:18:31,310 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:19:01,567 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 22:22:47,123 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:23:07,458 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:23:39,410 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:23:49,189 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:24:41,038 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:26:37,030 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 22:27:29,596 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 22:29:31,253 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:30:33,234 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 22:33:27,394 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:34:20,847 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:34:31,545 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:34:51,960 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:35:33,566 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:37:36,831 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 22:38:27,322 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 22:40:31,211 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:41:54,140 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 22:44:28,879 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:45:21,856 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:45:31,649 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:45:51,913 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:46:33,980 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:47:24,108 ERROR nodepool.NodeLauncher: Exception launching node id: 10864 in provider: wmflabs-eqiad error:
2015-12-06 22:47:24,111 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 22:48:36,785 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 22:49:26,608 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 22:51:29,808 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 22:52:31,313 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 22:55:47,608 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 22:56:38,050 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 22:56:48,719 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 22:57:20,219 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 22:57:32,327 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 22:59:27,816 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:00:28,637 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:02:42,095 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:03:24,107 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 23:06:29,608 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 23:06:55,631 ERROR nodepool.NodeLauncher: Exception launching node id: 10874 in provider: wmflabs-eqiad error:
2015-12-06 23:06:55,634 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:07:16,014 ERROR nodepool.NodeLauncher: Exception launching node id: 10876 in provider: wmflabs-eqiad error:
2015-12-06 23:07:16,017 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:07:20,602 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 23:07:42,565 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 23:08:22,901 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 23:08:43,953 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 23:10:39,570 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:11:26,471 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:12:14,564 ERROR nodepool.NodeLauncher: Exception launching node id: 10888 in provider: wmflabs-eqiad error:
2015-12-06 23:12:14,567 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:13:30,472 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:14:32,459 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 23:17:26,461 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 23:18:32,868 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 23:18:53,591 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 23:19:47,689 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 23:20:08,637 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 23:21:14,092 ERROR nodepool.NodeLauncher: Exception launching node id: 10889 in provider: wmflabs-eqiad error:
2015-12-06 23:21:14,095 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:21:31,350 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:22:31,048 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:24:28,050 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:25:30,150 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 23:28:32,876 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 23:29:24,416 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 23:29:45,645 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 23:30:36,257 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 23:31:28,361 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 23:32:35,830 ERROR nodepool.NodeLauncher: Exception launching node id: 10890 in provider: wmflabs-eqiad error:
2015-12-06 23:32:35,834 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:32:44,075 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:32:46,326 ERROR nodepool.NodeLauncher: Exception launching node id: 10891 in provider: wmflabs-eqiad error:
2015-12-06 23:32:46,328 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:33:35,067 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:35:27,653 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:36:29,651 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 23:39:33,572 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 23:40:24,136 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 23:40:45,895 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 23:41:36,615 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 23:42:28,184 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 23:42:53,047 ERROR nodepool.NodeLauncher: Exception launching node id: 10893 in provider: wmflabs-eqiad error:
2015-12-06 23:42:53,049 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:43:53,841 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:44:35,771 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:46:07,749 ERROR nodepool.NodeLauncher: Exception launching node id: 10894 in provider: wmflabs-eqiad error:
2015-12-06 23:46:07,755 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:46:27,241 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:47:39,901 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-06 23:48:45,781 ERROR nodepool.NodeLauncher: Exception launching node id: 10895 in provider: wmflabs-eqiad error:
2015-12-06 23:48:45,784 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:50:32,171 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-06 23:51:24,677 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-06 23:51:44,789 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-06 23:52:01,156 ERROR nodepool.NodeLauncher: Exception launching node id: 10896 in provider: wmflabs-eqiad error:
2015-12-06 23:52:01,159 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-06 23:52:35,373 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-06 23:53:16,710 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-06 23:54:31,482 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-06 23:55:33,810 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
2015-12-06 23:57:28,784 ERROR nodepool.NodeDeleter: Exception deleting node 10699:
2015-12-06 23:58:30,035 ERROR nodepool.NodeDeleter: Exception deleting node 10692:
2015-12-07 00:01:28,017 ERROR nodepool.NodeDeleter: Exception deleting node 10700:
2015-12-07 00:02:18,592 ERROR nodepool.NodeDeleter: Exception deleting node 10697:
2015-12-07 00:02:52,946 ERROR nodepool.NodeDeleter: Exception deleting node 10694:
2015-12-07 00:03:44,961 ERROR nodepool.NodeDeleter: Exception deleting node 10695:
2015-12-07 00:04:35,311 ERROR nodepool.NodeDeleter: Exception deleting node 10667:
2015-12-07 00:05:22,775 ERROR nodepool.NodeLauncher: Exception launching node id: 10897 in provider: wmflabs-eqiad error:
2015-12-07 00:05:22,777 ERROR nodepool.NodeLauncher: Exception in run method:
2015-12-07 00:05:26,623 ERROR nodepool.NodeDeleter: Exception deleting node 10687:
2015-12-07 00:06:29,261 ERROR nodepool.NodeDeleter: Exception deleting node 10698:
hashar@labnodepool1001:/var/log/nodepool$

Event Timeline

hashar changed the title of this paste from untitled to nodepool errors 2015-12-06.Dec 7 2015, 3:07 PM
hashar edited the content of this paste. (Show Details)