Page MenuHomePhabricator
Paste P6441

mlr training container kill / nodemanager log
ActivePublic
Actions

Authored by EBernhardson on Dec 7 2017, 5:05 AM.
Tags
None
Referenced Files
F11350786: mlr training container kill / nodemanager log
Dec 7 2017, 5:05 AM
Subscribers
None
2017-12-07 04:22:56,995 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_e54_1512469367986_4908_01_000007 by user ebernhardson
2017-12-07 04:22:56,996 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=ebernhardson IP=10.64.21.105 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1512469367986_4908 CONTAINERID=container_e54_1512469367986_4908_01_000007
2017-12-07 04:22:57,003 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Adding container_e54_1512469367986_4908_01_000007 to application application_1512469367986_4908
2017-12-07 04:22:57,004 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from NEW to LOCALIZING
2017-12-07 04:22:57,004 INFO org.apache.spark.network.yarn.YarnShuffleService: Initializing container container_e54_1512469367986_4908_01_000007
2017-12-07 04:22:57,004 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_e54_1512469367986_4908_01_000007
2017-12-07 04:22:57,006 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Writing credentials to the nmPrivate file /var/lib/hadoop/data/b/yarn/local/nmPrivate/container_e54_1512469367986_4908_01_000007.tokens. Credentials list:
2017-12-07 04:22:57,158 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Copying from /var/lib/hadoop/data/b/yarn/local/nmPrivate/container_e54_1512469367986_4908_01_000007.tokens to /var/lib/hadoop/data/l/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007.tokens
2017-12-07 04:23:18,879 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from LOCALIZING to LOCALIZED
2017-12-07 04:23:18,907 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from LOCALIZED to RUNNING
2017-12-07 04:23:18,912 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: launchContainer: [bash, /var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/default_container_executor.sh]
2017-12-07 04:23:19,896 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_e54_1512469367986_4908_01_000007
2017-12-07 04:23:20,036 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 140.2 MB of 10 GB physical memory used; 4.8 GB of 21 GB virtual memory used
2017-12-07 04:23:23,097 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 427.0 MB of 10 GB physical memory used; 5.0 GB of 21 GB virtual memory used
2017-12-07 04:23:26,155 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 427.2 MB of 10 GB physical memory used; 5.0 GB of 21 GB virtual memory used
2017-12-07 04:23:29,210 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 427.2 MB of 10 GB physical memory used; 5.0 GB of 21 GB virtual memory used
2017-12-07 04:23:32,268 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 585.9 MB of 10 GB physical memory used; 5.0 GB of 21 GB virtual memory used
2017-12-07 04:23:35,336 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 782.3 MB of 10 GB physical memory used; 5.0 GB of 21 GB virtual memory used
2017-12-07 04:23:38,402 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 879.0 MB of 10 GB physical memory used; 5.1 GB of 21 GB virtual memory used
2017-12-07 04:23:41,468 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 879.3 MB of 10 GB physical memory used; 5.1 GB of 21 GB virtual memory used
2017-12-07 04:23:44,532 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 879.3 MB of 10 GB physical memory used; 5.1 GB of 21 GB virtual memory used
2017-12-07 04:23:47,596 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 1.1 GB of 10 GB physical memory used; 6.9 GB of 21 GB virtual memory used
2017-12-07 04:23:50,655 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 1.7 GB of 10 GB physical memory used; 6.9 GB of 21 GB virtual memory used
2017-12-07 04:23:53,707 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 1.8 GB of 10 GB physical memory used; 6.9 GB of 21 GB virtual memory used
2017-12-07 04:23:56,761 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 2.3 GB of 10 GB physical memory used; 7.0 GB of 21 GB virtual memory used
2017-12-07 04:23:59,817 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 2.3 GB of 10 GB physical memory used; 7.0 GB of 21 GB virtual memory used
2017-12-07 04:24:02,867 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 2.5 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:05,919 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 3.2 GB of 10 GB physical memory used; 7.2 GB of 21 GB virtual memory used
2017-12-07 04:24:08,975 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 3.8 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:12,028 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 3.8 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:15,081 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.2 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:18,133 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.4 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:21,186 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.4 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:24,230 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:27,277 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:30,332 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:33,384 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:36,440 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:39,496 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:42,550 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:45,601 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:48,652 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:51,705 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:54,749 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:24:57,802 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:00,860 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:03,919 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:06,978 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:10,037 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:13,094 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:16,146 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.6 GB of 10 GB physical memory used; 7.1 GB of 21 GB virtual memory used
2017-12-07 04:25:19,287 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 4.9 GB of 10 GB physical memory used; 7.4 GB of 21 GB virtual memory used
2017-12-07 04:25:22,341 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 5.1 GB of 10 GB physical memory used; 7.8 GB of 21 GB virtual memory used
2017-12-07 04:25:25,393 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 5.5 GB of 10 GB physical memory used; 8.7 GB of 21 GB virtual memory used
2017-12-07 04:25:28,445 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 6.0 GB of 10 GB physical memory used; 9.5 GB of 21 GB virtual memory used
2017-12-07 04:25:31,514 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 6.4 GB of 10 GB physical memory used; 10.4 GB of 21 GB virtual memory used
2017-12-07 04:25:34,582 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 7.3 GB of 10 GB physical memory used; 11.0 GB of 21 GB virtual memory used
2017-12-07 04:25:37,650 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 7.9 GB of 10 GB physical memory used; 11.7 GB of 21 GB virtual memory used
2017-12-07 04:25:40,719 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 8.2 GB of 10 GB physical memory used; 12.4 GB of 21 GB virtual memory used
2017-12-07 04:25:43,791 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 8.6 GB of 10 GB physical memory used; 12.7 GB of 21 GB virtual memory used
2017-12-07 04:25:46,865 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 8.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:25:49,943 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.2 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:25:53,013 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.5 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:25:56,085 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:25:59,161 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:02,231 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:05,309 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:08,380 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:11,451 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:14,523 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:17,596 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:20,668 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:23,731 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:26,793 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:29,861 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:32,913 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:35,963 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:39,014 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:42,066 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:45,123 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:48,178 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:51,229 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:54,286 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:26:57,341 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:00,396 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:03,449 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:06,499 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:09,551 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:12,675 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:15,747 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:18,813 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:21,881 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:24,951 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:28,018 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:31,082 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:34,152 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:37,223 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:40,290 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:43,356 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:46,422 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:49,490 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:52,556 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:55,623 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:27:58,692 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:01,767 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:04,835 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:07,883 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:10,952 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:13,994 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:17,044 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:20,101 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:23,154 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.2 GB of 21 GB virtual memory used
2017-12-07 04:28:26,205 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 9.9 GB of 10 GB physical memory used; 12.6 GB of 21 GB virtual memory used
2017-12-07 04:28:29,257 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 25864 for container-id container_e54_1512469367986_4908_01_000007: 10.2 GB of 10 GB physical memory used; 12.6 GB of 21 GB virtual memory used
2017-12-07 04:28:29,257 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Process tree for container: container_e54_1512469367986_4908_01_000007 has processes older than 1 iteration running over the configured limit. Limit=10737418240, current usage = 10931814400
2017-12-07 04:28:29,257 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Container [pid=25864,containerID=container_e54_1512469367986_4908_01_000007] is running beyond physical memory limits. Current usage: 10.2 GB of 10 GB physical memory used; 12.6 GB of 21 GB virtual memory used. Killing container.
Dump of the process-tree for container_e54_1512469367986_4908_01_000007 :
|- 25864 25860 25864 25864 (bash) 0 1 13725696 799 /bin/bash -c /usr/lib/jvm/default-java/bin/java -server -Xmx4096m -Djava.io.tmpdir=/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/tmp '-Dspark.driver.port=40931' -Dspark.yarn.app.container.log.dir=/var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007 -XX:MaxPermSize=256m -XX:OnOutOfMemoryError='kill %p' org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.64.53.30:40931 --executor-id 6 --hostname analytics1049.eqiad.wmnet --cores 6 --app-id application_1512469367986_4908 --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/__app__.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/ml.dmlc_xgboost4j-spark-0.7-wmf-2-SNAPSHOT.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.wikimedia.search_mjolnir-0.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.spark_spark-streaming-kafka-0-8_2.11-2.1.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/ml.dmlc_xgboost4j-0.7-wmf-2-SNAPSHOT.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.kryo_kryo-2.21.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang_scala-compiler-2.11.8.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang_scala-reflect-2.11.8.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/commons-logging_commons-logging-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.typesafe.akka_akka-actor_2.11-2.3.11.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.typesafe_config-1.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.reflectasm_reflectasm-1.07.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.minlog_minlog-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.objenesis_objenesis-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.ow2.asm_asm-4.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang.modules_scala-xml_2.11-1.0.4.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang.modules_scala-parser-combinators_2.11-1.0.4.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.kafka_kafka_2.11-0.8.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.spark_spark-tags_2.11-2.1.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.spark-project.spark_unused-1.0.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.yammer.metrics_metrics-core-2.2.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.101tec_zkclient-0.3.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.kafka_kafka-clients-0.8.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.slf4j_slf4j-api-1.7.16.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/log4j_log4j-1.2.17.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/net.jpountz.lz4_lz4-1.3.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.xerial.snappy_snappy-java-1.1.2.6.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scalatest_scalatest_2.11-2.2.6.jar 1>/var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/stdout 2>/var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/stderr
|- 25933 25864 25864 25864 (java) 27138 3396 12533579776 2654552 /usr/lib/jvm/default-java/bin/java -server -Xmx4096m -Djava.io.tmpdir=/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/tmp -Dspark.driver.port=40931 -Dspark.yarn.app.container.log.dir=/var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007 -XX:MaxPermSize=256m -XX:OnOutOfMemoryError=kill %p org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.64.53.30:40931 --executor-id 6 --hostname analytics1049.eqiad.wmnet --cores 6 --app-id application_1512469367986_4908 --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/__app__.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/ml.dmlc_xgboost4j-spark-0.7-wmf-2-SNAPSHOT.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.wikimedia.search_mjolnir-0.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.spark_spark-streaming-kafka-0-8_2.11-2.1.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/ml.dmlc_xgboost4j-0.7-wmf-2-SNAPSHOT.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.kryo_kryo-2.21.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang_scala-compiler-2.11.8.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang_scala-reflect-2.11.8.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/commons-logging_commons-logging-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.typesafe.akka_akka-actor_2.11-2.3.11.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.typesafe_config-1.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.reflectasm_reflectasm-1.07.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.esotericsoftware.minlog_minlog-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.objenesis_objenesis-1.2.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.ow2.asm_asm-4.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang.modules_scala-xml_2.11-1.0.4.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scala-lang.modules_scala-parser-combinators_2.11-1.0.4.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.kafka_kafka_2.11-0.8.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.spark_spark-tags_2.11-2.1.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.spark-project.spark_unused-1.0.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.yammer.metrics_metrics-core-2.2.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/com.101tec_zkclient-0.3.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.apache.kafka_kafka-clients-0.8.2.1.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.slf4j_slf4j-api-1.7.16.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/log4j_log4j-1.2.17.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/net.jpountz.lz4_lz4-1.3.0.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.xerial.snappy_snappy-java-1.1.2.6.jar --user-class-path file:/var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/org.scalatest_scalatest_2.11-2.2.6.jar
2017-12-07 04:28:29,258 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from RUNNING to KILLING
2017-12-07 04:28:29,258 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,268 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_e54_1512469367986_4908_01_000007 is : 143
2017-12-07 04:28:29,293 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL
2017-12-07 04:28:29,293 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/b/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,293 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/c/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,293 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/d/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/e/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/f/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/g/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/h/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/i/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/j/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,294 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/k/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/l/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop/data/m/yarn/local/usercache/ebernhardson/appcache/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=ebernhardson OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1512469367986_4908 CONTAINERID=container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e54_1512469367986_4908_01_000007 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_e54_1512469367986_4908_01_000007 from application application_1512469367986_4908
2017-12-07 04:28:29,295 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_e54_1512469367986_4908_01_000007 for log-aggregation
2017-12-07 04:28:29,295 INFO org.apache.spark.network.yarn.YarnShuffleService: Stopping container container_e54_1512469367986_4908_01_000007
2017-12-07 04:28:30,301 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_e54_1512469367986_4908_01_000007]
2017-12-07 04:28:32,274 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_e54_1512469367986_4908_01_000007
2017-12-07 04:48:37,936 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_e54_1512469367986_4908_01_000007. Current good log dirs are /var/lib/hadoop/data/b/yarn/logs,/var/lib/hadoop/data/c/yarn/logs,/var/lib/hadoop/data/d/yarn/logs,/var/lib/hadoop/data/e/yarn/logs,/var/lib/hadoop/data/f/yarn/logs,/var/lib/hadoop/data/g/yarn/logs,/var/lib/hadoop/data/h/yarn/logs,/var/lib/hadoop/data/i/yarn/logs,/var/lib/hadoop/data/j/yarn/logs,/var/lib/hadoop/data/k/yarn/logs,/var/lib/hadoop/data/l/yarn/logs,/var/lib/hadoop/data/m/yarn/logs
2017-12-07 04:48:37,960 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting path : /var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/stdout
2017-12-07 04:48:37,961 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting path : /var/lib/hadoop/data/d/yarn/logs/application_1512469367986_4908/container_e54_1512469367986_4908_01_000007/stderr
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits