Page MenuHomePhabricator
Paste P54716

(An Untitled Masterwork)
ActivePublic
Actions

Authored by • JEbe-WMF on Jan 12 2024, 2:43 PM.
Project Tags
None
Referenced Files
F41667712: raw-paste-data.txt
Jan 12 2024, 2:43 PM
Subscribers
None
jebe@stat1007:~$ spark3-submit --driver-cores 4 --conf spark.executorEnv.SPARK_HOME=/usr/lib/spark3 --conf spark.executorEnv.SPARK_CONF_DIR=/etc/spark3/conf --master yarn --conf spark.dynamicAllocation.enabled=true --conf spark.dynamicAllocation.maxExecutors=16 --conf spark.shuffle.service.enabled=true --conf spark.yarn.maxAppAttempts=1 --conf spark.executor.memoryOverhead=2G --conf spark.yarn.appMasterEnv.SPARK_CONF_DIR=/etc/spark3/conf --conf spark.yarn.appMasterEnv.SPARK_HOME=/usr/lib/spark3 --executor-cores 2 --executor-memory 4G --driver-memory 16G --name dumps_publish_wikitext_raw_to_xml__publish_simplewiki_to_xml__20240111 --class org.wikimedia.analytics.refinery.job.mediawikidumper.MediawikiDumper --queue default --deploy-mode client hdfs:///wmf/cache/artifacts/airflow/analytics/refinery-job-0.2.24-shaded.jar --wiki_id simplewiki --publish_until 2024-01-01 --source_table wmf_dumps.wikitext_raw_rc2 --output_folder /user/jebe/data/archive/content_dump_test/
Running /opt/conda-analytics/bin/spark-submit $@
SPARK_HOME: /opt/conda-analytics/lib/python3.10/site-packages/pyspark
Using Hadoop client lib jars at 3.2.0, provided by Spark.
PYSPARK_PYTHON=/opt/conda-analytics/bin/python3
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
24/01/12 13:40:15 INFO SparkContext: Running Spark version 3.1.2
24/01/12 13:40:15 WARN SparkConf: Note that spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone/kubernetes and LOCAL_DIRS in YARN).
24/01/12 13:40:15 INFO ResourceUtils: ==============================================================
24/01/12 13:40:15 INFO ResourceUtils: No custom resources configured for spark.driver.
24/01/12 13:40:15 INFO ResourceUtils: ==============================================================
24/01/12 13:40:15 INFO SparkContext: Submitted application: dumps_publish_wikitext_raw_to_xml__publish_simplewiki_to_xml__20240111
24/01/12 13:40:15 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(memoryOverhead -> name: memoryOverhead, amount: 2048, script: , vendor: , cores -> name: cores, amount: 2, script: , vendor: , memory -> name: memory, amount: 4096, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
24/01/12 13:40:15 INFO ResourceProfile: Limiting resource is cpus at 2 tasks per executor
24/01/12 13:40:15 INFO ResourceProfileManager: Added ResourceProfile id: 0
24/01/12 13:40:15 INFO SecurityManager: Changing view acls to: jebe
24/01/12 13:40:15 INFO SecurityManager: Changing modify acls to: jebe
24/01/12 13:40:15 INFO SecurityManager: Changing view acls groups to:
24/01/12 13:40:15 INFO SecurityManager: Changing modify acls groups to:
24/01/12 13:40:15 INFO SecurityManager: SecurityManager: authentication enabled; ui acls disabled; users with view permissions: Set(jebe); groups with view permissions: Set(); users with modify permissions: Set(jebe); groups with modify permissions: Set()
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12000. Attempting port 12001.
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12001. Attempting port 12002.
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12002. Attempting port 12003.
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12003. Attempting port 12004.
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12004. Attempting port 12005.
24/01/12 13:40:16 WARN Utils: Service 'sparkDriver' could not bind on port 12005. Attempting port 12006.
24/01/12 13:40:16 INFO Utils: Successfully started service 'sparkDriver' on port 12006.
24/01/12 13:40:16 INFO SparkEnv: Registering MapOutputTracker
24/01/12 13:40:16 INFO SparkEnv: Registering BlockManagerMaster
24/01/12 13:40:16 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
24/01/12 13:40:16 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
24/01/12 13:40:16 INFO SparkEnv: Registering BlockManagerMasterHeartbeat
24/01/12 13:40:16 INFO DiskBlockManager: Created local directory at /srv/spark-tmp/blockmgr-2da583c1-d9af-4c7c-8cf9-7ee58d53ec0a
24/01/12 13:40:16 INFO MemoryStore: MemoryStore started with capacity 8.4 GiB
24/01/12 13:40:16 INFO SparkEnv: Registering OutputCommitCoordinator
24/01/12 13:40:16 INFO log: Logging initialized @7794ms to org.sparkproject.jetty.util.log.Slf4jLog
24/01/12 13:40:16 INFO Server: jetty-9.4.40.v20210413; built: 2021-04-13T20:42:42.668Z; git: b881a572662e1943a14ae12e7e1207989f218b74; jvm 1.8.0_392-8u392-ga-1~deb10u1-b08
24/01/12 13:40:16 INFO Server: Started @7906ms
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4042. Attempting port 4043.
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4043. Attempting port 4044.
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4044. Attempting port 4045.
24/01/12 13:40:16 WARN Utils: Service 'SparkUI' could not bind on port 4045. Attempting port 4046.
24/01/12 13:40:16 INFO AbstractConnector: Started ServerConnector@4b4d124{HTTP/1.1, (http/1.1)}{0.0.0.0:4046}
24/01/12 13:40:16 INFO Utils: Successfully started service 'SparkUI' on port 4046.
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@75dd0f94{/jobs,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@1d7c9811{/jobs/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6451a288{/jobs/job,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@18b40fe6{/jobs/job/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@67d5ac2f{/stages,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@1d637673{/stages/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@636dbfe7{/stages/stage,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@7dd3981e{/stages/stage/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@4a04ca74{/stages/pool,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@24aedcc5{/stages/pool/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@1850f2da{/storage,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6ace919c{/storage/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@5f5c187d{/storage/rdd,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@58182b96{/storage/rdd/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@8f6b4ab{/environment,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@770cae59{/environment/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@4bdb04c8{/executors,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@3ce7394f{/executors/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@74cd798f{/executors/threadDump,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@63c99f7{/executors/threadDump/json,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@601f264d{/static,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@35e07e19{/,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@2036f83{/api,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@48fc0211{/jobs/job/kill,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@7d088813{/stages/stage/kill,null,AVAILABLE,@Spark}
24/01/12 13:40:17 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://stat1007.eqiad.wmnet:4046
24/01/12 13:40:17 INFO SparkContext: Added JAR hdfs:///wmf/cache/artifacts/airflow/analytics/refinery-job-0.2.24-shaded.jar at hdfs:///wmf/cache/artifacts/airflow/analytics/refinery-job-0.2.24-shaded.jar with timestamp 1705066815671
24/01/12 13:40:17 INFO HadoopDelegationTokenManager: Attempting to load user's ticket cache.
24/01/12 13:40:17 INFO HiveConf: Found configuration file file:/etc/spark3/conf/hive-site.xml
24/01/12 13:40:17 INFO HadoopFSDelegationTokenProvider: getting token for: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-1980631662_1, ugi=jebe@WIKIMEDIA (auth:KERBEROS)]] with renewer yarn/an-master1001.eqiad.wmnet@WIKIMEDIA
24/01/12 13:40:18 INFO DFSClient: Created token for jebe: HDFS_DELEGATION_TOKEN owner=jebe@WIKIMEDIA, renewer=yarn, realUser=, issueDate=1705066817837, maxDate=1705671617837, sequenceNumber=22392873, masterKeyId=1767 on ha-hdfs:analytics-hadoop
24/01/12 13:40:18 INFO HadoopFSDelegationTokenProvider: getting token for: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-1980631662_1, ugi=jebe@WIKIMEDIA (auth:KERBEROS)]] with renewer jebe@WIKIMEDIA
24/01/12 13:40:18 INFO DFSClient: Created token for jebe: HDFS_DELEGATION_TOKEN owner=jebe@WIKIMEDIA, renewer=jebe, realUser=, issueDate=1705066818033, maxDate=1705671618033, sequenceNumber=22392874, masterKeyId=1767 on ha-hdfs:analytics-hadoop
24/01/12 13:40:18 INFO HadoopFSDelegationTokenProvider: Renewal interval is 86400283 for token HDFS_DELEGATION_TOKEN
24/01/12 13:40:19 INFO SparkHadoopUtil: Updating delegation tokens for current user.
24/01/12 13:40:19 INFO Utils: Using initial executors = 0, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
24/01/12 13:40:19 INFO Client: Requesting a new application from cluster with 86 NodeManagers
24/01/12 13:40:20 INFO Configuration: resource-types.xml not found
24/01/12 13:40:20 INFO ResourceUtils: Unable to find 'resource-types.xml'.
24/01/12 13:40:20 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (49152 MB per container)
24/01/12 13:40:20 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
24/01/12 13:40:20 INFO Client: Setting up container launch context for our AM
24/01/12 13:40:20 INFO Client: Setting up the launch environment for our AM container
24/01/12 13:40:20 INFO Client: Preparing resources for our AM container
24/01/12 13:40:20 INFO Client: Source and destination file systems are the same. Not copying hdfs:/user/spark/share/lib/spark-3.1.2-assembly.jar
24/01/12 13:40:20 INFO Client: Uploading resource file:/srv/spark-tmp/spark-3ed69458-8cf6-496f-b5dd-8cd5c38e33ab/__spark_conf__6845374601472918198.zip -> hdfs://analytics-hadoop/user/jebe/.sparkStaging/application_1704805203730_16243/__spark_conf__.zip
24/01/12 13:40:21 INFO SecurityManager: Changing view acls to: jebe
24/01/12 13:40:21 INFO SecurityManager: Changing modify acls to: jebe
24/01/12 13:40:21 INFO SecurityManager: Changing view acls groups to:
24/01/12 13:40:21 INFO SecurityManager: Changing modify acls groups to:
24/01/12 13:40:21 INFO SecurityManager: SecurityManager: authentication enabled; ui acls disabled; users with view permissions: Set(jebe); groups with view permissions: Set(); users with modify permissions: Set(jebe); groups with modify permissions: Set()
24/01/12 13:40:21 INFO Client: Submitting application application_1704805203730_16243 to ResourceManager
24/01/12 13:40:22 INFO YarnClientImpl: Submitted application application_1704805203730_16243
24/01/12 13:40:23 INFO Client: Application report for application_1704805203730_16243 (state: ACCEPTED)
24/01/12 13:40:23 INFO Client:
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: AM container is launched, waiting for AM container to Register with RM
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: default
start time: 1705066821852
final status: UNDEFINED
tracking URL: http://an-master1001.eqiad.wmnet:8088/proxy/application_1704805203730_16243/
user: jebe
24/01/12 13:40:24 INFO Client: Application report for application_1704805203730_16243 (state: ACCEPTED)
24/01/12 13:40:25 INFO Client: Application report for application_1704805203730_16243 (state: ACCEPTED)
24/01/12 13:40:26 INFO Client: Application report for application_1704805203730_16243 (state: ACCEPTED)
24/01/12 13:40:27 INFO Client: Application report for application_1704805203730_16243 (state: ACCEPTED)
24/01/12 13:40:28 INFO YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> an-master1001.eqiad.wmnet,an-master1002.eqiad.wmnet, PROXY_URI_BASES -> http://an-master1001.eqiad.wmnet:8088/proxy/application_1704805203730_16243,http://an-master1002.eqiad.wmnet:8088/proxy/application_1704805203730_16243, RM_HA_URLS -> an-master1001.eqiad.wmnet:8088,an-master1002.eqiad.wmnet:8088), /proxy/application_1704805203730_16243
24/01/12 13:40:28 INFO Client: Application report for application_1704805203730_16243 (state: RUNNING)
24/01/12 13:40:28 INFO Client:
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: N/A
ApplicationMaster host: 10.64.144.4
ApplicationMaster RPC port: -1
queue: default
start time: 1705066821852
final status: UNDEFINED
tracking URL: http://an-master1001.eqiad.wmnet:8088/proxy/application_1704805203730_16243/
user: jebe
24/01/12 13:40:28 INFO YarnClientSchedulerBackend: Application application_1704805203730_16243 has started running.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13000. Attempting port 13001.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13001. Attempting port 13002.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13002. Attempting port 13003.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13003. Attempting port 13004.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13004. Attempting port 13005.
24/01/12 13:40:28 WARN Utils: Service 'org.apache.spark.network.netty.NettyBlockTransferService' could not bind on port 13005. Attempting port 13006.
24/01/12 13:40:28 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 13006.
24/01/12 13:40:28 INFO NettyBlockTransferService: Server created on stat1007.eqiad.wmnet:13006
24/01/12 13:40:28 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
24/01/12 13:40:28 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, stat1007.eqiad.wmnet, 13006, None)
24/01/12 13:40:28 INFO BlockManagerMasterEndpoint: Registering block manager stat1007.eqiad.wmnet:13006 with 8.4 GiB RAM, BlockManagerId(driver, stat1007.eqiad.wmnet, 13006, None)
24/01/12 13:40:28 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, stat1007.eqiad.wmnet, 13006, None)
24/01/12 13:40:28 INFO BlockManager: external shuffle service port = 7337
24/01/12 13:40:28 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, stat1007.eqiad.wmnet, 13006, None)
24/01/12 13:40:28 INFO ServerInfo: Adding filter to /metrics/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:28 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6a97517{/metrics/json,null,AVAILABLE,@Spark}
24/01/12 13:40:28 INFO SingleEventLogFileWriter: Logging events to hdfs:/var/log/spark/application_1704805203730_16243.lz4.inprogress
24/01/12 13:40:28 INFO Utils: Using initial executors = 0, max of spark.dynamicAllocation.initialExecutors, spark.dynamicAllocation.minExecutors and spark.executor.instances
24/01/12 13:40:28 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
24/01/12 13:40:28 INFO YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
24/01/12 13:40:29 INFO YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark-client://YarnAM)
24/01/12 13:40:29 INFO ServerInfo: Adding filter to /SQL: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:29 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@1117cc7c{/SQL,null,AVAILABLE,@Spark}
24/01/12 13:40:29 INFO ServerInfo: Adding filter to /SQL/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:29 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@2b06f498{/SQL/json,null,AVAILABLE,@Spark}
24/01/12 13:40:29 INFO ServerInfo: Adding filter to /SQL/execution: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:29 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@642d34f1{/SQL/execution,null,AVAILABLE,@Spark}
24/01/12 13:40:29 INFO ServerInfo: Adding filter to /SQL/execution/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:29 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6f740044{/SQL/execution/json,null,AVAILABLE,@Spark}
24/01/12 13:40:29 INFO ServerInfo: Adding filter to /static/sql: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
24/01/12 13:40:29 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@21948bd1{/static/sql,null,AVAILABLE,@Spark}
24/01/12 13:40:30 INFO MediawikiDumper: Mediawiki Dumper
24/01/12 13:40:30 INFO MediawikiDumper: Mediawiki Dumper: About to dump simplewiki at 2024-01-01
24/01/12 13:40:30 INFO MediawikiDumper: Mediawiki Dumper: from wmf_dumps.wikitext_raw_rc2 to /user/jebe/data/archive/content_dump_test/
24/01/12 13:40:30 INFO MediawikiDumper: Mediawiki Dumper: step 1/5: build base revisions dataframe
24/01/12 13:40:31 INFO HiveConf: Found configuration file file:/etc/spark3/conf/hive-site.xml
24/01/12 13:40:32 INFO SessionState: Created HDFS directory: /tmp/hive/jebe/10e9eac7-a84a-4a98-9ec0-c85a0f7c1af0
24/01/12 13:40:32 INFO SessionState: Created local directory: /tmp/jebe/10e9eac7-a84a-4a98-9ec0-c85a0f7c1af0
24/01/12 13:40:32 INFO SessionState: Created HDFS directory: /tmp/hive/jebe/10e9eac7-a84a-4a98-9ec0-c85a0f7c1af0/_tmp_space.db
24/01/12 13:40:32 INFO metastore: Trying to connect to metastore with URI thrift://analytics-hive.eqiad.wmnet:9083
24/01/12 13:40:32 INFO metastore: Opened a connection to metastore, current connections: 1
24/01/12 13:40:32 INFO metastore: Connected to metastore.
24/01/12 13:40:33 INFO Hive: Registering function english_stemmer org.wikimedia.analytics.refinery.hive.Stemmer
24/01/12 13:40:33 INFO metastore: Trying to connect to metastore with URI thrift://analytics-hive.eqiad.wmnet:9083
24/01/12 13:40:33 INFO metastore: Opened a connection to metastore, current connections: 1
24/01/12 13:40:33 INFO metastore: Connected to metastore.
24/01/12 13:40:33 INFO BaseMetastoreTableOperations: Refreshing table metadata from new version: /wmf/data/wmf_dumps/wikitext_raw_rc2/metadata/03975-38b5d4e9-aae0-415d-a805-baad369a73da.metadata.json
24/01/12 13:40:34 INFO BaseMetastoreCatalog: Table loaded by catalog: spark_catalog.wmf_dumps.wikitext_raw_rc2
24/01/12 13:40:34 INFO MediawikiDumper: Mediawiki Dumper: step 2/5: define page partitions
24/01/12 13:40:36 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 32.0 KiB, free 8.4 GiB)
24/01/12 13:40:36 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 30.9 KiB, free 8.4 GiB)
24/01/12 13:40:36 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 30.9 KiB, free: 8.4 GiB)
24/01/12 13:40:36 INFO SparkContext: Created broadcast 0 from broadcast at SparkBatchScan.java:142
24/01/12 13:40:36 INFO SnapshotScan: Scanning table spark_catalog.wmf_dumps.wikitext_raw_rc2 snapshot 4861734485253907190 created at 2024-01-12T13:26:32.899+00:00 with filter (((revision_timestamp IS NOT NULL AND wiki_db IS NOT NULL) AND revision_timestamp < (16-digit-int)) AND wiki_db = (hash-10b04362))
24/01/12 13:40:39 INFO LoggingMetricsReporter: Received metrics report: ScanReport{tableName=spark_catalog.wmf_dumps.wikitext_raw_rc2, snapshotId=4861734485253907190, filter=(((not_null(ref(name="revision_timestamp")) and not_null(ref(name="wiki_db"))) and ref(name="revision_timestamp") < "(16-digit-int)") and ref(name="wiki_db") == "(hash-10b04362)"), schemaId=0, projectedFieldIds=[1, 18, 8, 24, 25, 10, 26, 27, 28], projectedFieldNames=[page_id, wiki_db, revision_id, revision_content_slots.value.content_body, revision_content_slots.value.content_format, revision_timestamp, revision_content_slots.value.content_model, revision_content_slots.value.content_sha1, revision_content_slots.value.content_size], scanMetrics=ScanMetricsResult{totalPlanningDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT2.960861525S, count=1}, resultDataFiles=CounterResult{unit=COUNT, value=1236}, resultDeleteFiles=CounterResult{unit=COUNT, value=0}, totalDataManifests=CounterResult{unit=COUNT, value=132}, totalDeleteManifests=CounterResult{unit=COUNT, value=0}, scannedDataManifests=CounterResult{unit=COUNT, value=81}, skippedDataManifests=CounterResult{unit=COUNT, value=51}, totalFileSizeInBytes=CounterResult{unit=BYTES, value=16576313022}, totalDeleteFileSizeInBytes=CounterResult{unit=BYTES, value=0}, skippedDataFiles=CounterResult{unit=COUNT, value=484574}, skippedDeleteFiles=CounterResult{unit=COUNT, value=0}, scannedDeleteManifests=CounterResult{unit=COUNT, value=0}, skippedDeleteManifests=CounterResult{unit=COUNT, value=0}, indexedDeleteFiles=CounterResult{unit=COUNT, value=0}, equalityDeleteFiles=CounterResult{unit=COUNT, value=0}, positionalDeleteFiles=CounterResult{unit=COUNT, value=0}}, metadata={engine-version=3.1.2, iceberg-version=Apache Iceberg 1.2.1 (commit 4e2cdccd7453603af42a090fc5530f2bd20cf1be), app-id=application_1704805203730_16243, engine-name=spark}}
24/01/12 13:40:39 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 32.0 KiB, free 8.4 GiB)
24/01/12 13:40:39 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 30.9 KiB, free 8.4 GiB)
24/01/12 13:40:39 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 30.9 KiB, free: 8.4 GiB)
24/01/12 13:40:39 INFO SparkContext: Created broadcast 1 from broadcast at SparkBatchScan.java:142
24/01/12 13:40:41 INFO SparkContext: Starting job: rdd at PagesPartitionsDefiner.scala:88
24/01/12 13:40:41 INFO DAGScheduler: Registering RDD 3 (rdd at PagesPartitionsDefiner.scala:88) as input to shuffle 0
24/01/12 13:40:41 INFO DAGScheduler: Got job 0 (rdd at PagesPartitionsDefiner.scala:88) with 200 output partitions
24/01/12 13:40:41 INFO DAGScheduler: Final stage: ResultStage 1 (rdd at PagesPartitionsDefiner.scala:88)
24/01/12 13:40:41 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 0)
24/01/12 13:40:41 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 0)
24/01/12 13:40:41 INFO DAGScheduler: Submitting ShuffleMapStage 0 (MapPartitionsRDD[3] at rdd at PagesPartitionsDefiner.scala:88), which has no missing parents
24/01/12 13:40:41 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 30.1 KiB, free 8.4 GiB)
24/01/12 13:40:41 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 13.4 KiB, free 8.4 GiB)
24/01/12 13:40:41 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 13.4 KiB, free: 8.4 GiB)
24/01/12 13:40:41 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1388
24/01/12 13:40:41 INFO DAGScheduler: Submitting 148 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[3] at rdd at PagesPartitionsDefiner.scala:88) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
24/01/12 13:40:41 INFO YarnScheduler: Adding task set 0.0 with 148 tasks resource profile 0
24/01/12 13:40:41 INFO ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 1 for resource profile id: 0)
24/01/12 13:40:46 INFO ExecutorAllocationManager: Requesting 2 new executors because tasks are backlogged (new desired total will be 3 for resource profile id: 0)
24/01/12 13:40:46 INFO ExecutorAllocationManager: Requesting 4 new executors because tasks are backlogged (new desired total will be 7 for resource profile id: 0)
24/01/12 13:40:47 INFO ExecutorAllocationManager: Requesting 8 new executors because tasks are backlogged (new desired total will be 15 for resource profile id: 0)
24/01/12 13:40:48 INFO ExecutorAllocationManager: Requesting 1 new executor because tasks are backlogged (new desired total will be 16 for resource profile id: 0)
24/01/12 13:40:51 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.40:41718) with ID 5, ResourceProfileId 0
24/01/12 13:40:51 INFO ExecutorMonitor: New executor 5 has registered (new total is 2)
24/01/12 13:40:51 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1102.eqiad.wmnet:33061 with 2004.6 MiB RAM, BlockManagerId(5, an-worker1102.eqiad.wmnet, 33061, None)
24/01/12 13:40:51 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.53.46:49450) with ID 9, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 9 has registered (new total is 3)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.53.46:49458) with ID 10, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 10 has registered (new total is 4)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1116.eqiad.wmnet:42965 with 2004.6 MiB RAM, BlockManagerId(9, an-worker1116.eqiad.wmnet, 42965, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.21.13:37168) with ID 15, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 15 has registered (new total is 5)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1116.eqiad.wmnet:38999 with 2004.6 MiB RAM, BlockManagerId(10, an-worker1116.eqiad.wmnet, 38999, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.21.119:43390) with ID 2, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 2 has registered (new total is 6)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1128.eqiad.wmnet:35005 with 2004.6 MiB RAM, BlockManagerId(15, an-worker1128.eqiad.wmnet, 35005, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.21.119:43406) with ID 3, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 3 has registered (new total is 7)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.8:53522) with ID 14, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 14 has registered (new total is 8)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1083.eqiad.wmnet:45911 with 2004.6 MiB RAM, BlockManagerId(2, an-worker1083.eqiad.wmnet, 45911, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.8:53536) with ID 13, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 13 has registered (new total is 9)
24/01/12 13:40:52 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 0) (an-worker1102.eqiad.wmnet, executor 5, partition 1, NODE_LOCAL, 24379 bytes) taskResourceAssignments Map()
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1119.eqiad.wmnet:37683 with 2004.6 MiB RAM, BlockManagerId(14, an-worker1119.eqiad.wmnet, 37683, None)
24/01/12 13:40:52 INFO TaskSetManager: Starting task 4.0 in stage 0.0 (TID 1) (an-worker1102.eqiad.wmnet, executor 5, partition 4, NODE_LOCAL, 28959 bytes) taskResourceAssignments Map()
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.8:53552) with ID 11, ResourceProfileId 0
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.8:53562) with ID 12, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 11 has registered (new total is 10)
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 12 has registered (new total is 11)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.53.37:49346) with ID 7, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 7 has registered (new total is 12)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1083.eqiad.wmnet:40809 with 2004.6 MiB RAM, BlockManagerId(3, an-worker1083.eqiad.wmnet, 40809, None)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1119.eqiad.wmnet:34183 with 2004.6 MiB RAM, BlockManagerId(13, an-worker1119.eqiad.wmnet, 34183, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.53.37:49352) with ID 6, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 6 has registered (new total is 13)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1119.eqiad.wmnet:45881 with 2004.6 MiB RAM, BlockManagerId(12, an-worker1119.eqiad.wmnet, 45881, None)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1119.eqiad.wmnet:38383 with 2004.6 MiB RAM, BlockManagerId(11, an-worker1119.eqiad.wmnet, 38383, None)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1095.eqiad.wmnet:38457 with 2004.6 MiB RAM, BlockManagerId(7, an-worker1095.eqiad.wmnet, 38457, None)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1095.eqiad.wmnet:45475 with 2004.6 MiB RAM, BlockManagerId(6, an-worker1095.eqiad.wmnet, 45475, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.138.9:50050) with ID 8, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 8 has registered (new total is 14)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1153.eqiad.wmnet:46731 with 2004.6 MiB RAM, BlockManagerId(8, an-worker1153.eqiad.wmnet, 46731, None)
24/01/12 13:40:52 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.139.2:37658) with ID 16, ResourceProfileId 0
24/01/12 13:40:52 INFO ExecutorMonitor: New executor 16 has registered (new total is 15)
24/01/12 13:40:52 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1143.eqiad.wmnet:42097 with 2004.6 MiB RAM, BlockManagerId(16, an-worker1143.eqiad.wmnet, 42097, None)
24/01/12 13:40:53 INFO TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2) (an-worker1116.eqiad.wmnet, executor 10, partition 2, NODE_LOCAL, 21406 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3) (an-worker1116.eqiad.wmnet, executor 10, partition 3, NODE_LOCAL, 18850 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.5.28:37500) with ID 1, ResourceProfileId 0
24/01/12 13:40:53 INFO ExecutorMonitor: New executor 1 has registered (new total is 16)
24/01/12 13:40:53 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1102.eqiad.wmnet:33061 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:53 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1079.eqiad.wmnet:44645 with 2004.6 MiB RAM, BlockManagerId(1, an-worker1079.eqiad.wmnet, 44645, None)
24/01/12 13:40:53 INFO TaskSetManager: Starting task 6.0 in stage 0.0 (TID 4) (an-worker1119.eqiad.wmnet, executor 14, partition 6, NODE_LOCAL, 28368 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 7.0 in stage 0.0 (TID 5) (an-worker1119.eqiad.wmnet, executor 14, partition 7, NODE_LOCAL, 32699 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 9.0 in stage 0.0 (TID 6) (an-worker1119.eqiad.wmnet, executor 13, partition 9, NODE_LOCAL, 29512 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 11.0 in stage 0.0 (TID 7) (an-worker1119.eqiad.wmnet, executor 13, partition 11, NODE_LOCAL, 19594 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 22.0 in stage 0.0 (TID 8) (an-worker1119.eqiad.wmnet, executor 11, partition 22, NODE_LOCAL, 30952 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 24.0 in stage 0.0 (TID 9) (an-worker1119.eqiad.wmnet, executor 11, partition 24, NODE_LOCAL, 26517 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 40.0 in stage 0.0 (TID 10) (an-worker1119.eqiad.wmnet, executor 12, partition 40, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 51.0 in stage 0.0 (TID 11) (an-worker1119.eqiad.wmnet, executor 12, partition 51, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1116.eqiad.wmnet:38999 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:53 INFO TaskSetManager: Starting task 14.0 in stage 0.0 (TID 12) (an-worker1143.eqiad.wmnet, executor 16, partition 14, NODE_LOCAL, 28793 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 54.0 in stage 0.0 (TID 13) (an-worker1143.eqiad.wmnet, executor 16, partition 54, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 14) (an-worker1153.eqiad.wmnet, executor 8, partition 0, NODE_LOCAL, 18497 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 8.0 in stage 0.0 (TID 15) (an-worker1153.eqiad.wmnet, executor 8, partition 8, NODE_LOCAL, 31624 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1102.eqiad.wmnet:33061 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:53 INFO TaskSetManager: Starting task 5.0 in stage 0.0 (TID 16) (an-worker1128.eqiad.wmnet, executor 15, partition 5, NODE_LOCAL, 18495 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 13.0 in stage 0.0 (TID 17) (an-worker1128.eqiad.wmnet, executor 15, partition 13, NODE_LOCAL, 24694 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 10.0 in stage 0.0 (TID 18) (an-worker1116.eqiad.wmnet, executor 9, partition 10, NODE_LOCAL, 26217 bytes) taskResourceAssignments Map()
24/01/12 13:40:53 INFO TaskSetManager: Starting task 16.0 in stage 0.0 (TID 19) (an-worker1116.eqiad.wmnet, executor 9, partition 16, NODE_LOCAL, 23150 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1119.eqiad.wmnet:34183 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1119.eqiad.wmnet:45881 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO TaskSetManager: Starting task 15.0 in stage 0.0 (TID 20) (an-worker1095.eqiad.wmnet, executor 7, partition 15, NODE_LOCAL, 28801 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 20.0 in stage 0.0 (TID 21) (an-worker1095.eqiad.wmnet, executor 7, partition 20, NODE_LOCAL, 20260 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 17.0 in stage 0.0 (TID 22) (an-worker1083.eqiad.wmnet, executor 3, partition 17, NODE_LOCAL, 21056 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 19.0 in stage 0.0 (TID 23) (an-worker1083.eqiad.wmnet, executor 3, partition 19, NODE_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 27.0 in stage 0.0 (TID 24) (an-worker1083.eqiad.wmnet, executor 2, partition 27, NODE_LOCAL, 28119 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 32.0 in stage 0.0 (TID 25) (an-worker1083.eqiad.wmnet, executor 2, partition 32, NODE_LOCAL, 20695 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1116.eqiad.wmnet:42965 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1119.eqiad.wmnet:37683 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO TaskSetManager: Starting task 61.0 in stage 0.0 (TID 26) (an-worker1095.eqiad.wmnet, executor 6, partition 61, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO TaskSetManager: Starting task 63.0 in stage 0.0 (TID 27) (an-worker1095.eqiad.wmnet, executor 6, partition 63, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1128.eqiad.wmnet:35005 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1116.eqiad.wmnet:38999 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1143.eqiad.wmnet:42097 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:54 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1153.eqiad.wmnet:46731 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO TaskSetManager: Starting task 26.0 in stage 0.0 (TID 28) (an-worker1079.eqiad.wmnet, executor 1, partition 26, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:40:55 INFO TaskSetManager: Starting task 35.0 in stage 0.0 (TID 29) (an-worker1079.eqiad.wmnet, executor 1, partition 35, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1116.eqiad.wmnet:42965 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1119.eqiad.wmnet:45881 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1119.eqiad.wmnet:37683 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1095.eqiad.wmnet:45475 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1095.eqiad.wmnet:38457 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1128.eqiad.wmnet:35005 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1119.eqiad.wmnet:34183 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1083.eqiad.wmnet:40809 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1083.eqiad.wmnet:45911 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1143.eqiad.wmnet:42097 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1153.eqiad.wmnet:46731 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:56 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1079.eqiad.wmnet:44645 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:40:56 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1095.eqiad.wmnet:45475 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:56 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1095.eqiad.wmnet:38457 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:56 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1083.eqiad.wmnet:45911 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:56 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1083.eqiad.wmnet:40809 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:57 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.64.21.121:56590) with ID 4, ResourceProfileId 0
24/01/12 13:40:57 INFO ExecutorMonitor: New executor 4 has registered (new total is 17)
24/01/12 13:40:57 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1079.eqiad.wmnet:44645 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:40:57 INFO TaskSetManager: Starting task 12.0 in stage 0.0 (TID 30) (an-worker1102.eqiad.wmnet, executor 5, partition 12, NODE_LOCAL, 24592 bytes) taskResourceAssignments Map()
24/01/12 13:40:57 INFO BlockManagerMasterEndpoint: Registering block manager an-worker1085.eqiad.wmnet:43491 with 2004.6 MiB RAM, BlockManagerId(4, an-worker1085.eqiad.wmnet, 43491, None)
24/01/12 13:40:57 INFO TaskSetManager: Finished task 4.0 in stage 0.0 (TID 1) in 4869 ms on an-worker1102.eqiad.wmnet (executor 5) (1/148)
24/01/12 13:40:58 INFO TaskSetManager: Starting task 18.0 in stage 0.0 (TID 31) (an-worker1102.eqiad.wmnet, executor 5, partition 18, NODE_LOCAL, 21719 bytes) taskResourceAssignments Map()
24/01/12 13:40:58 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 0) in 5859 ms on an-worker1102.eqiad.wmnet (executor 5) (2/148)
24/01/12 13:40:58 INFO TaskSetManager: Starting task 66.0 in stage 0.0 (TID 32) (an-worker1143.eqiad.wmnet, executor 16, partition 66, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:40:58 INFO TaskSetManager: Finished task 14.0 in stage 0.0 (TID 12) in 5005 ms on an-worker1143.eqiad.wmnet (executor 16) (3/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 80.0 in stage 0.0 (TID 33) (an-worker1119.eqiad.wmnet, executor 14, partition 80, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 7.0 in stage 0.0 (TID 5) in 5601 ms on an-worker1119.eqiad.wmnet (executor 14) (4/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 28.0 in stage 0.0 (TID 34) (an-worker1116.eqiad.wmnet, executor 9, partition 28, NODE_LOCAL, 20690 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 10.0 in stage 0.0 (TID 18) in 5243 ms on an-worker1116.eqiad.wmnet (executor 9) (5/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 25.0 in stage 0.0 (TID 35) (an-worker1085.eqiad.wmnet, executor 4, partition 25, NODE_LOCAL, 20645 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Starting task 50.0 in stage 0.0 (TID 36) (an-worker1085.eqiad.wmnet, executor 4, partition 50, NODE_LOCAL, 21761 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Starting task 41.0 in stage 0.0 (TID 37) (an-worker1116.eqiad.wmnet, executor 10, partition 41, NODE_LOCAL, 19570 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 6389 ms on an-worker1116.eqiad.wmnet (executor 10) (6/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 53.0 in stage 0.0 (TID 38) (an-worker1116.eqiad.wmnet, executor 10, partition 53, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 6446 ms on an-worker1116.eqiad.wmnet (executor 10) (7/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 92.0 in stage 0.0 (TID 39) (an-worker1119.eqiad.wmnet, executor 11, partition 92, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 22.0 in stage 0.0 (TID 8) in 6127 ms on an-worker1119.eqiad.wmnet (executor 11) (8/148)
24/01/12 13:40:59 INFO TaskSetManager: Starting task 108.0 in stage 0.0 (TID 40) (an-worker1119.eqiad.wmnet, executor 14, partition 108, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:40:59 INFO TaskSetManager: Finished task 6.0 in stage 0.0 (TID 4) in 6363 ms on an-worker1119.eqiad.wmnet (executor 14) (9/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 117.0 in stage 0.0 (TID 41) (an-worker1119.eqiad.wmnet, executor 13, partition 117, NODE_LOCAL, 21106 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 9.0 in stage 0.0 (TID 6) in 6395 ms on an-worker1119.eqiad.wmnet (executor 13) (10/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 42.0 in stage 0.0 (TID 42) (an-worker1102.eqiad.wmnet, executor 5, partition 42, NODE_LOCAL, 18854 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 12.0 in stage 0.0 (TID 30) in 2772 ms on an-worker1102.eqiad.wmnet (executor 5) (11/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 84.0 in stage 0.0 (TID 43) (an-worker1143.eqiad.wmnet, executor 16, partition 84, NODE_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 54.0 in stage 0.0 (TID 13) in 6170 ms on an-worker1143.eqiad.wmnet (executor 16) (12/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 118.0 in stage 0.0 (TID 44) (an-worker1119.eqiad.wmnet, executor 11, partition 118, NODE_LOCAL, 21464 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 24.0 in stage 0.0 (TID 9) in 6382 ms on an-worker1119.eqiad.wmnet (executor 11) (13/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 119.0 in stage 0.0 (TID 45) (an-worker1119.eqiad.wmnet, executor 12, partition 119, NODE_LOCAL, 31350 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 40.0 in stage 0.0 (TID 10) in 6339 ms on an-worker1119.eqiad.wmnet (executor 12) (14/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 38.0 in stage 0.0 (TID 46) (an-worker1128.eqiad.wmnet, executor 15, partition 38, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 13.0 in stage 0.0 (TID 17) in 6304 ms on an-worker1128.eqiad.wmnet (executor 15) (15/148)
24/01/12 13:41:00 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on an-worker1085.eqiad.wmnet:43491 (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 48.0 in stage 0.0 (TID 47) (an-worker1128.eqiad.wmnet, executor 15, partition 48, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 16) in 6438 ms on an-worker1128.eqiad.wmnet (executor 15) (16/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 121.0 in stage 0.0 (TID 48) (an-worker1119.eqiad.wmnet, executor 13, partition 121, NODE_LOCAL, 28169 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 11.0 in stage 0.0 (TID 7) in 6805 ms on an-worker1119.eqiad.wmnet (executor 13) (17/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 29.0 in stage 0.0 (TID 49) (an-worker1153.eqiad.wmnet, executor 8, partition 29, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 8.0 in stage 0.0 (TID 15) in 6759 ms on an-worker1153.eqiad.wmnet (executor 8) (18/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 33.0 in stage 0.0 (TID 50) (an-worker1153.eqiad.wmnet, executor 8, partition 33, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 14) in 6816 ms on an-worker1153.eqiad.wmnet (executor 8) (19/148)
24/01/12 13:41:00 INFO TaskSetManager: Starting task 123.0 in stage 0.0 (TID 51) (an-worker1119.eqiad.wmnet, executor 12, partition 123, NODE_LOCAL, 21107 bytes) taskResourceAssignments Map()
24/01/12 13:41:00 INFO TaskSetManager: Finished task 51.0 in stage 0.0 (TID 11) in 6898 ms on an-worker1119.eqiad.wmnet (executor 12) (20/148)
24/01/12 13:41:01 INFO TaskSetManager: Starting task 59.0 in stage 0.0 (TID 52) (an-worker1116.eqiad.wmnet, executor 9, partition 59, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:01 INFO TaskSetManager: Finished task 16.0 in stage 0.0 (TID 19) in 7151 ms on an-worker1116.eqiad.wmnet (executor 9) (21/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 120.0 in stage 0.0 (TID 53) (an-worker1143.eqiad.wmnet, executor 16, partition 120, NODE_LOCAL, 28160 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 66.0 in stage 0.0 (TID 32) in 3071 ms on an-worker1143.eqiad.wmnet (executor 16) (22/148)
24/01/12 13:41:02 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on an-worker1085.eqiad.wmnet:43491 (size: 30.9 KiB, free: 2004.6 MiB)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 62.0 in stage 0.0 (TID 54) (an-worker1102.eqiad.wmnet, executor 5, partition 62, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 18.0 in stage 0.0 (TID 31) in 3803 ms on an-worker1102.eqiad.wmnet (executor 5) (23/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 58.0 in stage 0.0 (TID 55) (an-worker1083.eqiad.wmnet, executor 2, partition 58, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 27.0 in stage 0.0 (TID 24) in 7577 ms on an-worker1083.eqiad.wmnet (executor 2) (24/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 70.0 in stage 0.0 (TID 56) (an-worker1095.eqiad.wmnet, executor 7, partition 70, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 15.0 in stage 0.0 (TID 20) in 7603 ms on an-worker1095.eqiad.wmnet (executor 7) (25/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 73.0 in stage 0.0 (TID 57) (an-worker1116.eqiad.wmnet, executor 9, partition 73, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 28.0 in stage 0.0 (TID 34) in 3042 ms on an-worker1116.eqiad.wmnet (executor 9) (26/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 71.0 in stage 0.0 (TID 58) (an-worker1095.eqiad.wmnet, executor 6, partition 71, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 63.0 in stage 0.0 (TID 27) in 7718 ms on an-worker1095.eqiad.wmnet (executor 6) (27/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 127.0 in stage 0.0 (TID 59) (an-worker1095.eqiad.wmnet, executor 6, partition 127, NODE_LOCAL, 33051 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 61.0 in stage 0.0 (TID 26) in 7738 ms on an-worker1095.eqiad.wmnet (executor 6) (28/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 125.0 in stage 0.0 (TID 60) (an-worker1119.eqiad.wmnet, executor 14, partition 125, NODE_LOCAL, 18109 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 80.0 in stage 0.0 (TID 33) in 3395 ms on an-worker1119.eqiad.wmnet (executor 14) (29/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 128.0 in stage 0.0 (TID 61) (an-worker1095.eqiad.wmnet, executor 7, partition 128, NODE_LOCAL, 26143 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 20.0 in stage 0.0 (TID 21) in 8344 ms on an-worker1095.eqiad.wmnet (executor 7) (30/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 64.0 in stage 0.0 (TID 62) (an-worker1083.eqiad.wmnet, executor 2, partition 64, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 32.0 in stage 0.0 (TID 25) in 8389 ms on an-worker1083.eqiad.wmnet (executor 2) (31/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 74.0 in stage 0.0 (TID 63) (an-worker1102.eqiad.wmnet, executor 5, partition 74, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 42.0 in stage 0.0 (TID 42) in 2845 ms on an-worker1102.eqiad.wmnet (executor 5) (32/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 129.0 in stage 0.0 (TID 64) (an-worker1119.eqiad.wmnet, executor 12, partition 129, NODE_LOCAL, 22917 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 119.0 in stage 0.0 (TID 45) in 2714 ms on an-worker1119.eqiad.wmnet (executor 12) (33/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 76.0 in stage 0.0 (TID 65) (an-worker1116.eqiad.wmnet, executor 10, partition 76, NODE_LOCAL, 19213 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 41.0 in stage 0.0 (TID 37) in 3293 ms on an-worker1116.eqiad.wmnet (executor 10) (34/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 130.0 in stage 0.0 (TID 66) (an-worker1119.eqiad.wmnet, executor 13, partition 130, NODE_LOCAL, 32042 bytes) taskResourceAssignments Map()
24/01/12 13:41:02 INFO TaskSetManager: Finished task 121.0 in stage 0.0 (TID 48) in 2568 ms on an-worker1119.eqiad.wmnet (executor 13) (35/148)
24/01/12 13:41:02 INFO TaskSetManager: Starting task 75.0 in stage 0.0 (TID 67) (an-worker1083.eqiad.wmnet, executor 3, partition 75, NODE_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 17.0 in stage 0.0 (TID 22) in 8494 ms on an-worker1083.eqiad.wmnet (executor 3) (36/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 88.0 in stage 0.0 (TID 68) (an-worker1116.eqiad.wmnet, executor 10, partition 88, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 53.0 in stage 0.0 (TID 38) in 3312 ms on an-worker1116.eqiad.wmnet (executor 10) (37/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 131.0 in stage 0.0 (TID 69) (an-worker1119.eqiad.wmnet, executor 14, partition 131, NODE_LOCAL, 24734 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 108.0 in stage 0.0 (TID 40) in 3094 ms on an-worker1119.eqiad.wmnet (executor 14) (38/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 135.0 in stage 0.0 (TID 70) (an-worker1119.eqiad.wmnet, executor 13, partition 135, NODE_LOCAL, 29091 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 117.0 in stage 0.0 (TID 41) in 3179 ms on an-worker1119.eqiad.wmnet (executor 13) (39/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 136.0 in stage 0.0 (TID 71) (an-worker1143.eqiad.wmnet, executor 16, partition 136, NODE_LOCAL, 27945 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 84.0 in stage 0.0 (TID 43) in 3197 ms on an-worker1143.eqiad.wmnet (executor 16) (40/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 137.0 in stage 0.0 (TID 72) (an-worker1119.eqiad.wmnet, executor 11, partition 137, NODE_LOCAL, 26389 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 92.0 in stage 0.0 (TID 39) in 3468 ms on an-worker1119.eqiad.wmnet (executor 11) (41/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 87.0 in stage 0.0 (TID 73) (an-worker1083.eqiad.wmnet, executor 3, partition 87, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 19.0 in stage 0.0 (TID 23) in 8908 ms on an-worker1083.eqiad.wmnet (executor 3) (42/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 145.0 in stage 0.0 (TID 74) (an-worker1119.eqiad.wmnet, executor 11, partition 145, NODE_LOCAL, 30951 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 118.0 in stage 0.0 (TID 44) in 3288 ms on an-worker1119.eqiad.wmnet (executor 11) (43/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 36.0 in stage 0.0 (TID 75) (an-worker1079.eqiad.wmnet, executor 1, partition 36, NODE_LOCAL, 19186 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 26.0 in stage 0.0 (TID 28) in 8492 ms on an-worker1079.eqiad.wmnet (executor 1) (44/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 57.0 in stage 0.0 (TID 76) (an-worker1079.eqiad.wmnet, executor 1, partition 57, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 35.0 in stage 0.0 (TID 29) in 8509 ms on an-worker1079.eqiad.wmnet (executor 1) (45/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 89.0 in stage 0.0 (TID 77) (an-worker1128.eqiad.wmnet, executor 15, partition 89, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 48.0 in stage 0.0 (TID 47) in 3360 ms on an-worker1128.eqiad.wmnet (executor 15) (46/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 146.0 in stage 0.0 (TID 78) (an-worker1119.eqiad.wmnet, executor 12, partition 146, NODE_LOCAL, 27362 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 123.0 in stage 0.0 (TID 51) in 3050 ms on an-worker1119.eqiad.wmnet (executor 12) (47/148)
24/01/12 13:41:03 INFO TaskSetManager: Starting task 95.0 in stage 0.0 (TID 79) (an-worker1128.eqiad.wmnet, executor 15, partition 95, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:03 INFO TaskSetManager: Finished task 38.0 in stage 0.0 (TID 46) in 3616 ms on an-worker1128.eqiad.wmnet (executor 15) (48/148)
24/01/12 13:41:04 INFO TaskSetManager: Finished task 120.0 in stage 0.0 (TID 53) in 2169 ms on an-worker1143.eqiad.wmnet (executor 16) (49/148)
24/01/12 13:41:04 INFO TaskSetManager: Starting task 37.0 in stage 0.0 (TID 80) (an-worker1153.eqiad.wmnet, executor 8, partition 37, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:04 INFO TaskSetManager: Finished task 29.0 in stage 0.0 (TID 49) in 3492 ms on an-worker1153.eqiad.wmnet (executor 8) (50/148)
24/01/12 13:41:04 INFO TaskSetManager: Starting task 107.0 in stage 0.0 (TID 81) (an-worker1116.eqiad.wmnet, executor 9, partition 107, NODE_LOCAL, 17396 bytes) taskResourceAssignments Map()
24/01/12 13:41:04 INFO TaskSetManager: Finished task 59.0 in stage 0.0 (TID 52) in 3181 ms on an-worker1116.eqiad.wmnet (executor 9) (51/148)
24/01/12 13:41:04 INFO TaskSetManager: Starting task 79.0 in stage 0.0 (TID 82) (an-worker1153.eqiad.wmnet, executor 8, partition 79, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:04 INFO TaskSetManager: Finished task 33.0 in stage 0.0 (TID 50) in 3665 ms on an-worker1153.eqiad.wmnet (executor 8) (52/148)
24/01/12 13:41:04 INFO TaskSetManager: Starting task 132.0 in stage 0.0 (TID 83) (an-worker1095.eqiad.wmnet, executor 6, partition 132, NODE_LOCAL, 26215 bytes) taskResourceAssignments Map()
24/01/12 13:41:04 INFO TaskSetManager: Finished task 127.0 in stage 0.0 (TID 59) in 2308 ms on an-worker1095.eqiad.wmnet (executor 6) (53/148)
24/01/12 13:41:05 INFO TaskSetManager: Starting task 90.0 in stage 0.0 (TID 84) (an-worker1102.eqiad.wmnet, executor 5, partition 90, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:05 INFO TaskSetManager: Finished task 62.0 in stage 0.0 (TID 54) in 3144 ms on an-worker1102.eqiad.wmnet (executor 5) (54/148)
24/01/12 13:41:05 INFO TaskSetManager: Starting task 147.0 in stage 0.0 (TID 85) (an-worker1119.eqiad.wmnet, executor 13, partition 147, NODE_LOCAL, 25715 bytes) taskResourceAssignments Map()
24/01/12 13:41:05 INFO TaskSetManager: Finished task 130.0 in stage 0.0 (TID 66) in 2393 ms on an-worker1119.eqiad.wmnet (executor 13) (55/148)
24/01/12 13:41:05 INFO TaskSetManager: Finished task 136.0 in stage 0.0 (TID 71) in 2126 ms on an-worker1143.eqiad.wmnet (executor 16) (56/148)
24/01/12 13:41:05 INFO TaskSetManager: Finished task 135.0 in stage 0.0 (TID 70) in 2282 ms on an-worker1119.eqiad.wmnet (executor 13) (57/148)
24/01/12 13:41:05 INFO TaskSetManager: Finished task 145.0 in stage 0.0 (TID 74) in 2180 ms on an-worker1119.eqiad.wmnet (executor 11) (58/148)
24/01/12 13:41:05 INFO TaskSetManager: Starting task 109.0 in stage 0.0 (TID 86) (an-worker1116.eqiad.wmnet, executor 9, partition 109, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:05 INFO TaskSetManager: Finished task 73.0 in stage 0.0 (TID 57) in 3443 ms on an-worker1116.eqiad.wmnet (executor 9) (59/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 94.0 in stage 0.0 (TID 87) (an-worker1102.eqiad.wmnet, executor 5, partition 94, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 74.0 in stage 0.0 (TID 63) in 3122 ms on an-worker1102.eqiad.wmnet (executor 5) (60/148)
24/01/12 13:41:06 INFO TaskSetManager: Finished task 129.0 in stage 0.0 (TID 64) in 3200 ms on an-worker1119.eqiad.wmnet (executor 12) (61/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 124.0 in stage 0.0 (TID 88) (an-worker1116.eqiad.wmnet, executor 10, partition 124, NODE_LOCAL, 19622 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 88.0 in stage 0.0 (TID 68) in 3214 ms on an-worker1116.eqiad.wmnet (executor 10) (62/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 139.0 in stage 0.0 (TID 89) (an-worker1116.eqiad.wmnet, executor 10, partition 139, NODE_LOCAL, 19979 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 76.0 in stage 0.0 (TID 65) in 3292 ms on an-worker1116.eqiad.wmnet (executor 10) (63/148)
24/01/12 13:41:06 INFO TaskSetManager: Finished task 131.0 in stage 0.0 (TID 69) in 3389 ms on an-worker1119.eqiad.wmnet (executor 14) (64/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 102.0 in stage 0.0 (TID 90) (an-worker1083.eqiad.wmnet, executor 2, partition 102, NODE_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 58.0 in stage 0.0 (TID 55) in 4381 ms on an-worker1083.eqiad.wmnet (executor 2) (65/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 133.0 in stage 0.0 (TID 91) (an-worker1095.eqiad.wmnet, executor 7, partition 133, NODE_LOCAL, 27305 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 128.0 in stage 0.0 (TID 61) in 3696 ms on an-worker1095.eqiad.wmnet (executor 7) (66/148)
24/01/12 13:41:06 INFO TaskSetManager: Finished task 147.0 in stage 0.0 (TID 85) in 1202 ms on an-worker1119.eqiad.wmnet (executor 13) (67/148)
24/01/12 13:41:06 INFO TaskSetManager: Finished task 125.0 in stage 0.0 (TID 60) in 4105 ms on an-worker1119.eqiad.wmnet (executor 14) (68/148)
24/01/12 13:41:06 INFO TaskSetManager: Starting task 142.0 in stage 0.0 (TID 92) (an-worker1095.eqiad.wmnet, executor 6, partition 142, NODE_LOCAL, 19206 bytes) taskResourceAssignments Map()
24/01/12 13:41:06 INFO TaskSetManager: Finished task 71.0 in stage 0.0 (TID 58) in 4530 ms on an-worker1095.eqiad.wmnet (executor 6) (69/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 44.0 in stage 0.0 (TID 93) (an-worker1128.eqiad.wmnet, executor 15, partition 44, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 89.0 in stage 0.0 (TID 77) in 3230 ms on an-worker1128.eqiad.wmnet (executor 15) (70/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 77.0 in stage 0.0 (TID 94) (an-worker1128.eqiad.wmnet, executor 15, partition 77, RACK_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 95.0 in stage 0.0 (TID 79) in 3111 ms on an-worker1128.eqiad.wmnet (executor 15) (71/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 126.0 in stage 0.0 (TID 95) (an-worker1083.eqiad.wmnet, executor 2, partition 126, NODE_LOCAL, 20721 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 64.0 in stage 0.0 (TID 62) in 4238 ms on an-worker1083.eqiad.wmnet (executor 2) (72/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 34.0 in stage 0.0 (TID 96) (an-worker1119.eqiad.wmnet, executor 14, partition 34, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 68.0 in stage 0.0 (TID 97) (an-worker1119.eqiad.wmnet, executor 13, partition 68, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 72.0 in stage 0.0 (TID 98) (an-worker1119.eqiad.wmnet, executor 11, partition 72, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 116.0 in stage 0.0 (TID 99) (an-worker1119.eqiad.wmnet, executor 12, partition 116, RACK_LOCAL, 20693 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 138.0 in stage 0.0 (TID 100) (an-worker1119.eqiad.wmnet, executor 14, partition 138, RACK_LOCAL, 21839 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 143.0 in stage 0.0 (TID 101) (an-worker1119.eqiad.wmnet, executor 13, partition 143, RACK_LOCAL, 30162 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Starting task 144.0 in stage 0.0 (TID 102) (an-worker1095.eqiad.wmnet, executor 7, partition 144, NODE_LOCAL, 26376 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 70.0 in stage 0.0 (TID 56) in 5261 ms on an-worker1095.eqiad.wmnet (executor 7) (73/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 141.0 in stage 0.0 (TID 103) (an-worker1083.eqiad.wmnet, executor 3, partition 141, NODE_LOCAL, 28763 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 75.0 in stage 0.0 (TID 67) in 4420 ms on an-worker1083.eqiad.wmnet (executor 3) (74/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 106.0 in stage 0.0 (TID 104) (an-worker1153.eqiad.wmnet, executor 8, partition 106, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 37.0 in stage 0.0 (TID 80) in 3280 ms on an-worker1153.eqiad.wmnet (executor 8) (75/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 85.0 in stage 0.0 (TID 105) (an-worker1116.eqiad.wmnet, executor 9, partition 85, RACK_LOCAL, 18854 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 107.0 in stage 0.0 (TID 81) in 3211 ms on an-worker1116.eqiad.wmnet (executor 9) (76/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 122.0 in stage 0.0 (TID 106) (an-worker1119.eqiad.wmnet, executor 11, partition 122, RACK_LOCAL, 22948 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 137.0 in stage 0.0 (TID 72) in 4131 ms on an-worker1119.eqiad.wmnet (executor 11) (77/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 112.0 in stage 0.0 (TID 107) (an-worker1153.eqiad.wmnet, executor 8, partition 112, NODE_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 79.0 in stage 0.0 (TID 82) in 3374 ms on an-worker1153.eqiad.wmnet (executor 8) (78/148)
24/01/12 13:41:07 INFO TaskSetManager: Starting task 31.0 in stage 0.0 (TID 108) (an-worker1083.eqiad.wmnet, executor 3, partition 31, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:07 INFO TaskSetManager: Finished task 87.0 in stage 0.0 (TID 73) in 4482 ms on an-worker1083.eqiad.wmnet (executor 3) (79/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 134.0 in stage 0.0 (TID 109) (an-worker1079.eqiad.wmnet, executor 1, partition 134, NODE_LOCAL, 25119 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 57.0 in stage 0.0 (TID 76) in 4298 ms on an-worker1079.eqiad.wmnet (executor 1) (80/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 140.0 in stage 0.0 (TID 110) (an-worker1119.eqiad.wmnet, executor 12, partition 140, RACK_LOCAL, 21848 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 146.0 in stage 0.0 (TID 78) in 4581 ms on an-worker1119.eqiad.wmnet (executor 12) (81/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 93.0 in stage 0.0 (TID 111) (an-worker1095.eqiad.wmnet, executor 6, partition 93, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 132.0 in stage 0.0 (TID 83) in 3804 ms on an-worker1095.eqiad.wmnet (executor 6) (82/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 96.0 in stage 0.0 (TID 112) (an-worker1102.eqiad.wmnet, executor 5, partition 96, NODE_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 90.0 in stage 0.0 (TID 84) in 3365 ms on an-worker1102.eqiad.wmnet (executor 5) (83/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 43.0 in stage 0.0 (TID 113) (an-worker1079.eqiad.wmnet, executor 1, partition 43, RACK_LOCAL, 19186 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 36.0 in stage 0.0 (TID 75) in 4952 ms on an-worker1079.eqiad.wmnet (executor 1) (84/148)
24/01/12 13:41:08 INFO TaskSetManager: Starting task 99.0 in stage 0.0 (TID 114) (an-worker1116.eqiad.wmnet, executor 9, partition 99, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:08 INFO TaskSetManager: Finished task 109.0 in stage 0.0 (TID 86) in 2996 ms on an-worker1116.eqiad.wmnet (executor 9) (85/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 39.0 in stage 0.0 (TID 115) (an-worker1102.eqiad.wmnet, executor 5, partition 39, RACK_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 94.0 in stage 0.0 (TID 87) in 3208 ms on an-worker1102.eqiad.wmnet (executor 5) (86/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 78.0 in stage 0.0 (TID 116) (an-worker1085.eqiad.wmnet, executor 4, partition 78, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 25.0 in stage 0.0 (TID 35) in 9979 ms on an-worker1085.eqiad.wmnet (executor 4) (87/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 103.0 in stage 0.0 (TID 117) (an-worker1085.eqiad.wmnet, executor 4, partition 103, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 50.0 in stage 0.0 (TID 36) in 9980 ms on an-worker1085.eqiad.wmnet (executor 4) (88/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 115.0 in stage 0.0 (TID 118) (an-worker1116.eqiad.wmnet, executor 10, partition 115, RACK_LOCAL, 18478 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 124.0 in stage 0.0 (TID 88) in 3169 ms on an-worker1116.eqiad.wmnet (executor 10) (89/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 23.0 in stage 0.0 (TID 119) (an-worker1116.eqiad.wmnet, executor 10, partition 23, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 139.0 in stage 0.0 (TID 89) in 3194 ms on an-worker1116.eqiad.wmnet (executor 10) (90/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 45.0 in stage 0.0 (TID 120) (an-worker1119.eqiad.wmnet, executor 13, partition 45, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 143.0 in stage 0.0 (TID 101) in 2661 ms on an-worker1119.eqiad.wmnet (executor 13) (91/148)
24/01/12 13:41:09 INFO TaskSetManager: Starting task 69.0 in stage 0.0 (TID 121) (an-worker1095.eqiad.wmnet, executor 7, partition 69, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:09 INFO TaskSetManager: Finished task 133.0 in stage 0.0 (TID 91) in 3352 ms on an-worker1095.eqiad.wmnet (executor 7) (92/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 113.0 in stage 0.0 (TID 122) (an-worker1128.eqiad.wmnet, executor 15, partition 113, RACK_LOCAL, 17396 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 44.0 in stage 0.0 (TID 93) in 3147 ms on an-worker1128.eqiad.wmnet (executor 15) (93/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 21.0 in stage 0.0 (TID 123) (an-worker1143.eqiad.wmnet, executor 16, partition 21, ANY, 17752 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Starting task 30.0 in stage 0.0 (TID 124) (an-worker1143.eqiad.wmnet, executor 16, partition 30, ANY, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Starting task 91.0 in stage 0.0 (TID 125) (an-worker1095.eqiad.wmnet, executor 7, partition 91, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 144.0 in stage 0.0 (TID 102) in 2854 ms on an-worker1095.eqiad.wmnet (executor 7) (94/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 46.0 in stage 0.0 (TID 126) (an-worker1119.eqiad.wmnet, executor 14, partition 46, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 138.0 in stage 0.0 (TID 100) in 3138 ms on an-worker1119.eqiad.wmnet (executor 14) (95/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 49.0 in stage 0.0 (TID 127) (an-worker1119.eqiad.wmnet, executor 12, partition 49, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 116.0 in stage 0.0 (TID 99) in 3228 ms on an-worker1119.eqiad.wmnet (executor 12) (96/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 114.0 in stage 0.0 (TID 128) (an-worker1128.eqiad.wmnet, executor 15, partition 114, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 77.0 in stage 0.0 (TID 94) in 3529 ms on an-worker1128.eqiad.wmnet (executor 15) (97/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 47.0 in stage 0.0 (TID 129) (an-worker1153.eqiad.wmnet, executor 8, partition 47, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 106.0 in stage 0.0 (TID 104) in 3051 ms on an-worker1153.eqiad.wmnet (executor 8) (98/148)
24/01/12 13:41:10 INFO TaskSetManager: Finished task 141.0 in stage 0.0 (TID 103) in 3220 ms on an-worker1083.eqiad.wmnet (executor 3) (99/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 60.0 in stage 0.0 (TID 130) (an-worker1119.eqiad.wmnet, executor 11, partition 60, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 72.0 in stage 0.0 (TID 98) in 3530 ms on an-worker1119.eqiad.wmnet (executor 11) (100/148)
24/01/12 13:41:10 INFO TaskSetManager: Finished task 102.0 in stage 0.0 (TID 90) in 4281 ms on an-worker1083.eqiad.wmnet (executor 2) (101/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 97.0 in stage 0.0 (TID 131) (an-worker1095.eqiad.wmnet, executor 6, partition 97, RACK_LOCAL, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 142.0 in stage 0.0 (TID 92) in 3949 ms on an-worker1095.eqiad.wmnet (executor 6) (102/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 101.0 in stage 0.0 (TID 132) (an-worker1119.eqiad.wmnet, executor 14, partition 101, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 34.0 in stage 0.0 (TID 96) in 3616 ms on an-worker1119.eqiad.wmnet (executor 14) (103/148)
24/01/12 13:41:10 INFO TaskSetManager: Finished task 68.0 in stage 0.0 (TID 97) in 3628 ms on an-worker1119.eqiad.wmnet (executor 13) (104/148)
24/01/12 13:41:10 INFO TaskSetManager: Starting task 81.0 in stage 0.0 (TID 133) (an-worker1116.eqiad.wmnet, executor 9, partition 81, RACK_LOCAL, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:10 INFO TaskSetManager: Finished task 85.0 in stage 0.0 (TID 105) in 3353 ms on an-worker1116.eqiad.wmnet (executor 9) (105/148)
24/01/12 13:41:10 INFO TaskSetManager: Finished task 122.0 in stage 0.0 (TID 106) in 3359 ms on an-worker1119.eqiad.wmnet (executor 11) (106/148)
24/01/12 13:41:11 INFO TaskSetManager: Finished task 112.0 in stage 0.0 (TID 107) in 3197 ms on an-worker1153.eqiad.wmnet (executor 8) (107/148)
24/01/12 13:41:11 INFO TaskSetManager: Finished task 140.0 in stage 0.0 (TID 110) in 2731 ms on an-worker1119.eqiad.wmnet (executor 12) (108/148)
24/01/12 13:41:11 INFO TaskSetManager: Finished task 126.0 in stage 0.0 (TID 95) in 4111 ms on an-worker1083.eqiad.wmnet (executor 2) (109/148)
24/01/12 13:41:11 INFO TaskSetManager: Starting task 65.0 in stage 0.0 (TID 134) (an-worker1102.eqiad.wmnet, executor 5, partition 65, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:11 INFO TaskSetManager: Finished task 96.0 in stage 0.0 (TID 112) in 3188 ms on an-worker1102.eqiad.wmnet (executor 5) (110/148)
24/01/12 13:41:11 INFO TaskSetManager: Finished task 31.0 in stage 0.0 (TID 108) in 3929 ms on an-worker1083.eqiad.wmnet (executor 3) (111/148)
24/01/12 13:41:11 INFO TaskSetManager: Starting task 86.0 in stage 0.0 (TID 135) (an-worker1116.eqiad.wmnet, executor 9, partition 86, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:11 INFO TaskSetManager: Finished task 99.0 in stage 0.0 (TID 114) in 3246 ms on an-worker1116.eqiad.wmnet (executor 9) (112/148)
24/01/12 13:41:11 INFO TaskSetManager: Finished task 134.0 in stage 0.0 (TID 109) in 3935 ms on an-worker1079.eqiad.wmnet (executor 1) (113/148)
24/01/12 13:41:12 INFO TaskSetManager: Starting task 83.0 in stage 0.0 (TID 136) (an-worker1102.eqiad.wmnet, executor 5, partition 83, RACK_LOCAL, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:12 INFO TaskSetManager: Finished task 39.0 in stage 0.0 (TID 115) in 3131 ms on an-worker1102.eqiad.wmnet (executor 5) (114/148)
24/01/12 13:41:12 INFO TaskSetManager: Starting task 100.0 in stage 0.0 (TID 137) (an-worker1095.eqiad.wmnet, executor 6, partition 100, RACK_LOCAL, 18478 bytes) taskResourceAssignments Map()
24/01/12 13:41:12 INFO TaskSetManager: Finished task 93.0 in stage 0.0 (TID 111) in 4079 ms on an-worker1095.eqiad.wmnet (executor 6) (115/148)
24/01/12 13:41:12 INFO TaskSetManager: Finished task 115.0 in stage 0.0 (TID 118) in 3135 ms on an-worker1116.eqiad.wmnet (executor 10) (116/148)
24/01/12 13:41:12 INFO TaskSetManager: Finished task 23.0 in stage 0.0 (TID 119) in 3211 ms on an-worker1116.eqiad.wmnet (executor 10) (117/148)
24/01/12 13:41:12 INFO TaskSetManager: Finished task 45.0 in stage 0.0 (TID 120) in 3146 ms on an-worker1119.eqiad.wmnet (executor 13) (118/148)
24/01/12 13:41:13 INFO TaskSetManager: Finished task 43.0 in stage 0.0 (TID 113) in 4406 ms on an-worker1079.eqiad.wmnet (executor 1) (119/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 67.0 in stage 0.0 (TID 138) (an-worker1085.eqiad.wmnet, executor 4, partition 67, RACK_LOCAL, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 103.0 in stage 0.0 (TID 117) in 4004 ms on an-worker1085.eqiad.wmnet (executor 4) (120/148)
24/01/12 13:41:13 INFO TaskSetManager: Finished task 21.0 in stage 0.0 (TID 123) in 3146 ms on an-worker1143.eqiad.wmnet (executor 16) (121/148)
24/01/12 13:41:13 INFO TaskSetManager: Finished task 30.0 in stage 0.0 (TID 124) in 3289 ms on an-worker1143.eqiad.wmnet (executor 16) (122/148)
24/01/12 13:41:13 INFO TaskSetManager: Finished task 49.0 in stage 0.0 (TID 127) in 3077 ms on an-worker1119.eqiad.wmnet (executor 12) (123/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 52.0 in stage 0.0 (TID 139) (an-worker1085.eqiad.wmnet, executor 4, partition 52, ANY, 18470 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 78.0 in stage 0.0 (TID 116) in 4329 ms on an-worker1085.eqiad.wmnet (executor 4) (124/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 55.0 in stage 0.0 (TID 140) (an-worker1128.eqiad.wmnet, executor 15, partition 55, ANY, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 114.0 in stage 0.0 (TID 128) in 3069 ms on an-worker1128.eqiad.wmnet (executor 15) (125/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 56.0 in stage 0.0 (TID 141) (an-worker1128.eqiad.wmnet, executor 15, partition 56, ANY, 18828 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 113.0 in stage 0.0 (TID 122) in 3467 ms on an-worker1128.eqiad.wmnet (executor 15) (126/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 82.0 in stage 0.0 (TID 142) (an-worker1119.eqiad.wmnet, executor 14, partition 82, ANY, 18112 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 46.0 in stage 0.0 (TID 126) in 3293 ms on an-worker1119.eqiad.wmnet (executor 14) (127/148)
24/01/12 13:41:13 INFO TaskSetManager: Starting task 98.0 in stage 0.0 (TID 143) (an-worker1153.eqiad.wmnet, executor 8, partition 98, ANY, 17404 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Starting task 104.0 in stage 0.0 (TID 144) (an-worker1153.eqiad.wmnet, executor 8, partition 104, ANY, 17754 bytes) taskResourceAssignments Map()
24/01/12 13:41:13 INFO TaskSetManager: Finished task 47.0 in stage 0.0 (TID 129) in 3158 ms on an-worker1153.eqiad.wmnet (executor 8) (128/148)
24/01/12 13:41:14 INFO TaskSetManager: Starting task 105.0 in stage 0.0 (TID 145) (an-worker1119.eqiad.wmnet, executor 14, partition 105, ANY, 17396 bytes) taskResourceAssignments Map()
24/01/12 13:41:14 INFO TaskSetManager: Finished task 101.0 in stage 0.0 (TID 132) in 3249 ms on an-worker1119.eqiad.wmnet (executor 14) (129/148)
24/01/12 13:41:14 INFO TaskSetManager: Starting task 110.0 in stage 0.0 (TID 146) (an-worker1119.eqiad.wmnet, executor 11, partition 110, ANY, 17396 bytes) taskResourceAssignments Map()
24/01/12 13:41:14 INFO TaskSetManager: Starting task 111.0 in stage 0.0 (TID 147) (an-worker1119.eqiad.wmnet, executor 11, partition 111, ANY, 17404 bytes) taskResourceAssignments Map()
24/01/12 13:41:14 INFO TaskSetManager: Finished task 60.0 in stage 0.0 (TID 130) in 3388 ms on an-worker1119.eqiad.wmnet (executor 11) (130/148)
24/01/12 13:41:14 INFO TaskSetManager: Finished task 91.0 in stage 0.0 (TID 125) in 4173 ms on an-worker1095.eqiad.wmnet (executor 7) (131/148)
24/01/12 13:41:14 INFO TaskSetManager: Finished task 69.0 in stage 0.0 (TID 121) in 4555 ms on an-worker1095.eqiad.wmnet (executor 7) (132/148)
24/01/12 13:41:14 INFO TaskSetManager: Finished task 81.0 in stage 0.0 (TID 133) in 3617 ms on an-worker1116.eqiad.wmnet (executor 9) (133/148)
24/01/12 13:41:14 INFO TaskSetManager: Finished task 97.0 in stage 0.0 (TID 131) in 3753 ms on an-worker1095.eqiad.wmnet (executor 6) (134/148)
24/01/12 13:41:14 INFO TaskSetManager: Finished task 65.0 in stage 0.0 (TID 134) in 3153 ms on an-worker1102.eqiad.wmnet (executor 5) (135/148)
24/01/12 13:41:15 INFO TaskSetManager: Finished task 86.0 in stage 0.0 (TID 135) in 3114 ms on an-worker1116.eqiad.wmnet (executor 9) (136/148)
24/01/12 13:41:15 INFO TaskSetManager: Finished task 83.0 in stage 0.0 (TID 136) in 3423 ms on an-worker1102.eqiad.wmnet (executor 5) (137/148)
24/01/12 13:41:16 INFO TaskSetManager: Finished task 100.0 in stage 0.0 (TID 137) in 3836 ms on an-worker1095.eqiad.wmnet (executor 6) (138/148)
24/01/12 13:41:16 INFO TaskSetManager: Finished task 55.0 in stage 0.0 (TID 140) in 3223 ms on an-worker1128.eqiad.wmnet (executor 15) (139/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 98.0 in stage 0.0 (TID 143) in 3355 ms on an-worker1153.eqiad.wmnet (executor 8) (140/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 82.0 in stage 0.0 (TID 142) in 3443 ms on an-worker1119.eqiad.wmnet (executor 14) (141/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 105.0 in stage 0.0 (TID 145) in 3085 ms on an-worker1119.eqiad.wmnet (executor 14) (142/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 104.0 in stage 0.0 (TID 144) in 3478 ms on an-worker1153.eqiad.wmnet (executor 8) (143/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 56.0 in stage 0.0 (TID 141) in 3580 ms on an-worker1128.eqiad.wmnet (executor 15) (144/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 110.0 in stage 0.0 (TID 146) in 3111 ms on an-worker1119.eqiad.wmnet (executor 11) (145/148)
24/01/12 13:41:17 INFO TaskSetManager: Finished task 111.0 in stage 0.0 (TID 147) in 3179 ms on an-worker1119.eqiad.wmnet (executor 11) (146/148)
24/01/12 13:41:18 INFO TaskSetManager: Finished task 67.0 in stage 0.0 (TID 138) in 5141 ms on an-worker1085.eqiad.wmnet (executor 4) (147/148)
24/01/12 13:41:18 INFO TaskSetManager: Finished task 52.0 in stage 0.0 (TID 139) in 5117 ms on an-worker1085.eqiad.wmnet (executor 4) (148/148)
24/01/12 13:41:18 INFO YarnScheduler: Removed TaskSet 0.0, whose tasks have all completed, from pool
24/01/12 13:41:18 INFO DAGScheduler: ShuffleMapStage 0 (rdd at PagesPartitionsDefiner.scala:88) finished in 37.362 s
24/01/12 13:41:18 INFO DAGScheduler: looking for newly runnable stages
24/01/12 13:41:18 INFO DAGScheduler: running: Set()
24/01/12 13:41:18 INFO DAGScheduler: waiting: Set(ResultStage 1)
24/01/12 13:41:18 INFO DAGScheduler: failed: Set()
24/01/12 13:41:18 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[8] at rdd at PagesPartitionsDefiner.scala:88), which has no missing parents
24/01/12 13:41:18 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 31.4 KiB, free 8.4 GiB)
24/01/12 13:41:18 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 15.1 KiB, free 8.4 GiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 15.1 KiB, free: 8.4 GiB)
24/01/12 13:41:18 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1388
24/01/12 13:41:18 INFO DAGScheduler: Submitting 200 missing tasks from ResultStage 1 (MapPartitionsRDD[8] at rdd at PagesPartitionsDefiner.scala:88) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
24/01/12 13:41:18 INFO YarnScheduler: Adding task set 1.0 with 200 tasks resource profile 0
24/01/12 13:41:18 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 148) (an-worker1153.eqiad.wmnet, executor 8, partition 0, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 149) (an-worker1119.eqiad.wmnet, executor 11, partition 1, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 2.0 in stage 1.0 (TID 150) (an-worker1095.eqiad.wmnet, executor 6, partition 2, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 3.0 in stage 1.0 (TID 151) (an-worker1085.eqiad.wmnet, executor 4, partition 3, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 4.0 in stage 1.0 (TID 152) (an-worker1083.eqiad.wmnet, executor 3, partition 4, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 5.0 in stage 1.0 (TID 153) (an-worker1102.eqiad.wmnet, executor 5, partition 5, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 6.0 in stage 1.0 (TID 154) (an-worker1128.eqiad.wmnet, executor 15, partition 6, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 7.0 in stage 1.0 (TID 155) (an-worker1119.eqiad.wmnet, executor 13, partition 7, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 8.0 in stage 1.0 (TID 156) (an-worker1095.eqiad.wmnet, executor 7, partition 8, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 9.0 in stage 1.0 (TID 157) (an-worker1116.eqiad.wmnet, executor 9, partition 9, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 10.0 in stage 1.0 (TID 158) (an-worker1083.eqiad.wmnet, executor 2, partition 10, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 11.0 in stage 1.0 (TID 159) (an-worker1079.eqiad.wmnet, executor 1, partition 11, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 12.0 in stage 1.0 (TID 160) (an-worker1116.eqiad.wmnet, executor 10, partition 12, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 13.0 in stage 1.0 (TID 161) (an-worker1119.eqiad.wmnet, executor 14, partition 13, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 14.0 in stage 1.0 (TID 162) (an-worker1119.eqiad.wmnet, executor 12, partition 14, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 15.0 in stage 1.0 (TID 163) (an-worker1143.eqiad.wmnet, executor 16, partition 15, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 16.0 in stage 1.0 (TID 164) (an-worker1153.eqiad.wmnet, executor 8, partition 16, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 17.0 in stage 1.0 (TID 165) (an-worker1119.eqiad.wmnet, executor 11, partition 17, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 18.0 in stage 1.0 (TID 166) (an-worker1095.eqiad.wmnet, executor 6, partition 18, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 19.0 in stage 1.0 (TID 167) (an-worker1085.eqiad.wmnet, executor 4, partition 19, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 20.0 in stage 1.0 (TID 168) (an-worker1083.eqiad.wmnet, executor 3, partition 20, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 21.0 in stage 1.0 (TID 169) (an-worker1102.eqiad.wmnet, executor 5, partition 21, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 22.0 in stage 1.0 (TID 170) (an-worker1128.eqiad.wmnet, executor 15, partition 22, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 23.0 in stage 1.0 (TID 171) (an-worker1119.eqiad.wmnet, executor 13, partition 23, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 24.0 in stage 1.0 (TID 172) (an-worker1095.eqiad.wmnet, executor 7, partition 24, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 25.0 in stage 1.0 (TID 173) (an-worker1116.eqiad.wmnet, executor 9, partition 25, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 26.0 in stage 1.0 (TID 174) (an-worker1083.eqiad.wmnet, executor 2, partition 26, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 27.0 in stage 1.0 (TID 175) (an-worker1079.eqiad.wmnet, executor 1, partition 27, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 28.0 in stage 1.0 (TID 176) (an-worker1116.eqiad.wmnet, executor 10, partition 28, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 29.0 in stage 1.0 (TID 177) (an-worker1119.eqiad.wmnet, executor 14, partition 29, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 30.0 in stage 1.0 (TID 178) (an-worker1119.eqiad.wmnet, executor 12, partition 30, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO TaskSetManager: Starting task 31.0 in stage 1.0 (TID 179) (an-worker1143.eqiad.wmnet, executor 16, partition 31, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1083.eqiad.wmnet:45911 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1102.eqiad.wmnet:33061 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1128.eqiad.wmnet:35005 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1119.eqiad.wmnet:45881 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1095.eqiad.wmnet:45475 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1085.eqiad.wmnet:43491 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1079.eqiad.wmnet:44645 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1083.eqiad.wmnet:40809 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1119.eqiad.wmnet:34183 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1116.eqiad.wmnet:38999 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1095.eqiad.wmnet:38457 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1143.eqiad.wmnet:42097 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1116.eqiad.wmnet:42965 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1119.eqiad.wmnet:37683 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on an-worker1153.eqiad.wmnet:46731 (size: 15.1 KiB, free: 2004.5 MiB)
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.40:41718
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.21.13:37168
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.8:53562
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.139.2:37658
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.53.46:49458
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.8:53552
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.53.46:49450
24/01/12 13:41:18 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.8:53522
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.8:53536
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.138.9:50050
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.21.119:43406
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.21.119:43390
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.5.28:37500
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.53.37:49346
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.53.37:49352
24/01/12 13:41:19 INFO TaskSetManager: Starting task 32.0 in stage 1.0 (TID 180) (an-worker1083.eqiad.wmnet, executor 3, partition 32, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 33.0 in stage 1.0 (TID 181) (an-worker1102.eqiad.wmnet, executor 5, partition 33, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 34.0 in stage 1.0 (TID 182) (an-worker1102.eqiad.wmnet, executor 5, partition 34, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 21.0 in stage 1.0 (TID 169) in 652 ms on an-worker1102.eqiad.wmnet (executor 5) (1/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 35.0 in stage 1.0 (TID 183) (an-worker1083.eqiad.wmnet, executor 3, partition 35, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 5.0 in stage 1.0 (TID 153) in 659 ms on an-worker1102.eqiad.wmnet (executor 5) (2/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 20.0 in stage 1.0 (TID 168) in 653 ms on an-worker1083.eqiad.wmnet (executor 3) (3/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 4.0 in stage 1.0 (TID 152) in 660 ms on an-worker1083.eqiad.wmnet (executor 3) (4/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 36.0 in stage 1.0 (TID 184) (an-worker1116.eqiad.wmnet, executor 10, partition 36, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 37.0 in stage 1.0 (TID 185) (an-worker1116.eqiad.wmnet, executor 10, partition 37, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 12.0 in stage 1.0 (TID 160) in 665 ms on an-worker1116.eqiad.wmnet (executor 10) (5/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 28.0 in stage 1.0 (TID 176) in 661 ms on an-worker1116.eqiad.wmnet (executor 10) (6/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 38.0 in stage 1.0 (TID 186) (an-worker1116.eqiad.wmnet, executor 9, partition 38, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 39.0 in stage 1.0 (TID 187) (an-worker1119.eqiad.wmnet, executor 13, partition 39, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 9.0 in stage 1.0 (TID 157) in 682 ms on an-worker1116.eqiad.wmnet (executor 9) (7/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 40.0 in stage 1.0 (TID 188) (an-worker1116.eqiad.wmnet, executor 9, partition 40, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 7.0 in stage 1.0 (TID 155) in 685 ms on an-worker1119.eqiad.wmnet (executor 13) (8/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 41.0 in stage 1.0 (TID 189) (an-worker1119.eqiad.wmnet, executor 13, partition 41, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 25.0 in stage 1.0 (TID 173) in 681 ms on an-worker1116.eqiad.wmnet (executor 9) (9/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 42.0 in stage 1.0 (TID 190) (an-worker1128.eqiad.wmnet, executor 15, partition 42, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 23.0 in stage 1.0 (TID 171) in 683 ms on an-worker1119.eqiad.wmnet (executor 13) (10/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 43.0 in stage 1.0 (TID 191) (an-worker1128.eqiad.wmnet, executor 15, partition 43, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 22.0 in stage 1.0 (TID 170) in 687 ms on an-worker1128.eqiad.wmnet (executor 15) (11/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 44.0 in stage 1.0 (TID 192) (an-worker1143.eqiad.wmnet, executor 16, partition 44, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 6.0 in stage 1.0 (TID 154) in 694 ms on an-worker1128.eqiad.wmnet (executor 15) (12/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 45.0 in stage 1.0 (TID 193) (an-worker1143.eqiad.wmnet, executor 16, partition 45, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 15.0 in stage 1.0 (TID 163) in 693 ms on an-worker1143.eqiad.wmnet (executor 16) (13/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 31.0 in stage 1.0 (TID 179) in 690 ms on an-worker1143.eqiad.wmnet (executor 16) (14/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 46.0 in stage 1.0 (TID 194) (an-worker1102.eqiad.wmnet, executor 5, partition 46, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 34.0 in stage 1.0 (TID 182) in 48 ms on an-worker1102.eqiad.wmnet (executor 5) (15/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 47.0 in stage 1.0 (TID 195) (an-worker1102.eqiad.wmnet, executor 5, partition 47, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 48.0 in stage 1.0 (TID 196) (an-worker1083.eqiad.wmnet, executor 3, partition 48, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 33.0 in stage 1.0 (TID 181) in 58 ms on an-worker1102.eqiad.wmnet (executor 5) (16/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 49.0 in stage 1.0 (TID 197) (an-worker1119.eqiad.wmnet, executor 11, partition 49, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 35.0 in stage 1.0 (TID 183) in 57 ms on an-worker1083.eqiad.wmnet (executor 3) (17/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 50.0 in stage 1.0 (TID 198) (an-worker1119.eqiad.wmnet, executor 11, partition 50, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 149) in 719 ms on an-worker1119.eqiad.wmnet (executor 11) (18/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 51.0 in stage 1.0 (TID 199) (an-worker1119.eqiad.wmnet, executor 12, partition 51, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 17.0 in stage 1.0 (TID 165) in 713 ms on an-worker1119.eqiad.wmnet (executor 11) (19/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 52.0 in stage 1.0 (TID 200) (an-worker1119.eqiad.wmnet, executor 12, partition 52, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 30.0 in stage 1.0 (TID 178) in 712 ms on an-worker1119.eqiad.wmnet (executor 12) (20/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 53.0 in stage 1.0 (TID 201) (an-worker1119.eqiad.wmnet, executor 14, partition 53, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 14.0 in stage 1.0 (TID 162) in 719 ms on an-worker1119.eqiad.wmnet (executor 12) (21/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 29.0 in stage 1.0 (TID 177) in 716 ms on an-worker1119.eqiad.wmnet (executor 14) (22/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 54.0 in stage 1.0 (TID 202) (an-worker1119.eqiad.wmnet, executor 14, partition 54, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 55.0 in stage 1.0 (TID 203) (an-worker1116.eqiad.wmnet, executor 10, partition 55, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 13.0 in stage 1.0 (TID 161) in 724 ms on an-worker1119.eqiad.wmnet (executor 14) (23/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 56.0 in stage 1.0 (TID 204) (an-worker1116.eqiad.wmnet, executor 10, partition 56, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 36.0 in stage 1.0 (TID 184) in 64 ms on an-worker1116.eqiad.wmnet (executor 10) (24/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 57.0 in stage 1.0 (TID 205) (an-worker1083.eqiad.wmnet, executor 3, partition 57, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 37.0 in stage 1.0 (TID 185) in 65 ms on an-worker1116.eqiad.wmnet (executor 10) (25/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 32.0 in stage 1.0 (TID 180) in 83 ms on an-worker1083.eqiad.wmnet (executor 3) (26/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 58.0 in stage 1.0 (TID 206) (an-worker1095.eqiad.wmnet, executor 7, partition 58, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 59.0 in stage 1.0 (TID 207) (an-worker1102.eqiad.wmnet, executor 5, partition 59, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 8.0 in stage 1.0 (TID 156) in 734 ms on an-worker1095.eqiad.wmnet (executor 7) (27/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 60.0 in stage 1.0 (TID 208) (an-worker1095.eqiad.wmnet, executor 7, partition 60, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 61.0 in stage 1.0 (TID 209) (an-worker1102.eqiad.wmnet, executor 5, partition 61, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 46.0 in stage 1.0 (TID 194) in 36 ms on an-worker1102.eqiad.wmnet (executor 5) (28/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 62.0 in stage 1.0 (TID 210) (an-worker1116.eqiad.wmnet, executor 9, partition 62, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 24.0 in stage 1.0 (TID 172) in 733 ms on an-worker1095.eqiad.wmnet (executor 7) (29/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 47.0 in stage 1.0 (TID 195) in 30 ms on an-worker1102.eqiad.wmnet (executor 5) (30/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 63.0 in stage 1.0 (TID 211) (an-worker1116.eqiad.wmnet, executor 9, partition 63, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 38.0 in stage 1.0 (TID 186) in 60 ms on an-worker1116.eqiad.wmnet (executor 9) (31/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 64.0 in stage 1.0 (TID 212) (an-worker1079.eqiad.wmnet, executor 1, partition 64, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 40.0 in stage 1.0 (TID 188) in 57 ms on an-worker1116.eqiad.wmnet (executor 9) (32/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 11.0 in stage 1.0 (TID 159) in 741 ms on an-worker1079.eqiad.wmnet (executor 1) (33/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 65.0 in stage 1.0 (TID 213) (an-worker1128.eqiad.wmnet, executor 15, partition 65, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 66.0 in stage 1.0 (TID 214) (an-worker1128.eqiad.wmnet, executor 15, partition 66, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 67.0 in stage 1.0 (TID 215) (an-worker1079.eqiad.wmnet, executor 1, partition 67, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 43.0 in stage 1.0 (TID 191) in 56 ms on an-worker1128.eqiad.wmnet (executor 15) (34/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 42.0 in stage 1.0 (TID 190) in 59 ms on an-worker1128.eqiad.wmnet (executor 15) (35/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 68.0 in stage 1.0 (TID 216) (an-worker1143.eqiad.wmnet, executor 16, partition 68, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 27.0 in stage 1.0 (TID 175) in 742 ms on an-worker1079.eqiad.wmnet (executor 1) (36/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 69.0 in stage 1.0 (TID 217) (an-worker1143.eqiad.wmnet, executor 16, partition 69, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 45.0 in stage 1.0 (TID 193) in 55 ms on an-worker1143.eqiad.wmnet (executor 16) (37/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 44.0 in stage 1.0 (TID 192) in 59 ms on an-worker1143.eqiad.wmnet (executor 16) (38/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 70.0 in stage 1.0 (TID 218) (an-worker1119.eqiad.wmnet, executor 13, partition 70, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 71.0 in stage 1.0 (TID 219) (an-worker1116.eqiad.wmnet, executor 10, partition 71, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 41.0 in stage 1.0 (TID 189) in 75 ms on an-worker1119.eqiad.wmnet (executor 13) (39/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 72.0 in stage 1.0 (TID 220) (an-worker1119.eqiad.wmnet, executor 13, partition 72, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 55.0 in stage 1.0 (TID 203) in 36 ms on an-worker1116.eqiad.wmnet (executor 10) (40/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 73.0 in stage 1.0 (TID 221) (an-worker1102.eqiad.wmnet, executor 5, partition 73, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 39.0 in stage 1.0 (TID 187) in 81 ms on an-worker1119.eqiad.wmnet (executor 13) (41/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 74.0 in stage 1.0 (TID 222) (an-worker1083.eqiad.wmnet, executor 3, partition 74, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 59.0 in stage 1.0 (TID 207) in 30 ms on an-worker1102.eqiad.wmnet (executor 5) (42/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 48.0 in stage 1.0 (TID 196) in 55 ms on an-worker1083.eqiad.wmnet (executor 3) (43/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 75.0 in stage 1.0 (TID 223) (an-worker1116.eqiad.wmnet, executor 10, partition 75, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 76.0 in stage 1.0 (TID 224) (an-worker1119.eqiad.wmnet, executor 11, partition 76, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 56.0 in stage 1.0 (TID 204) in 40 ms on an-worker1116.eqiad.wmnet (executor 10) (44/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 50.0 in stage 1.0 (TID 198) in 54 ms on an-worker1119.eqiad.wmnet (executor 11) (45/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 77.0 in stage 1.0 (TID 225) (an-worker1102.eqiad.wmnet, executor 5, partition 77, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 78.0 in stage 1.0 (TID 226) (an-worker1083.eqiad.wmnet, executor 3, partition 78, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 61.0 in stage 1.0 (TID 209) in 35 ms on an-worker1102.eqiad.wmnet (executor 5) (46/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 79.0 in stage 1.0 (TID 227) (an-worker1119.eqiad.wmnet, executor 11, partition 79, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 57.0 in stage 1.0 (TID 205) in 44 ms on an-worker1083.eqiad.wmnet (executor 3) (47/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 80.0 in stage 1.0 (TID 228) (an-worker1119.eqiad.wmnet, executor 12, partition 80, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 49.0 in stage 1.0 (TID 197) in 62 ms on an-worker1119.eqiad.wmnet (executor 11) (48/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 81.0 in stage 1.0 (TID 229) (an-worker1119.eqiad.wmnet, executor 14, partition 81, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 51.0 in stage 1.0 (TID 199) in 60 ms on an-worker1119.eqiad.wmnet (executor 12) (49/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 82.0 in stage 1.0 (TID 230) (an-worker1119.eqiad.wmnet, executor 12, partition 82, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 83.0 in stage 1.0 (TID 231) (an-worker1116.eqiad.wmnet, executor 9, partition 83, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 52.0 in stage 1.0 (TID 200) in 61 ms on an-worker1119.eqiad.wmnet (executor 12) (50/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 53.0 in stage 1.0 (TID 201) in 58 ms on an-worker1119.eqiad.wmnet (executor 14) (51/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 84.0 in stage 1.0 (TID 232) (an-worker1116.eqiad.wmnet, executor 9, partition 84, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 85.0 in stage 1.0 (TID 233) (an-worker1095.eqiad.wmnet, executor 6, partition 85, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 62.0 in stage 1.0 (TID 210) in 44 ms on an-worker1116.eqiad.wmnet (executor 9) (52/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 63.0 in stage 1.0 (TID 211) in 42 ms on an-worker1116.eqiad.wmnet (executor 9) (53/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 86.0 in stage 1.0 (TID 234) (an-worker1128.eqiad.wmnet, executor 15, partition 86, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 18.0 in stage 1.0 (TID 166) in 779 ms on an-worker1095.eqiad.wmnet (executor 6) (54/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 87.0 in stage 1.0 (TID 235) (an-worker1119.eqiad.wmnet, executor 14, partition 87, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 66.0 in stage 1.0 (TID 214) in 39 ms on an-worker1128.eqiad.wmnet (executor 15) (55/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 88.0 in stage 1.0 (TID 236) (an-worker1128.eqiad.wmnet, executor 15, partition 88, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 54.0 in stage 1.0 (TID 202) in 62 ms on an-worker1119.eqiad.wmnet (executor 14) (56/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 89.0 in stage 1.0 (TID 237) (an-worker1143.eqiad.wmnet, executor 16, partition 89, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 65.0 in stage 1.0 (TID 213) in 42 ms on an-worker1128.eqiad.wmnet (executor 15) (57/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 90.0 in stage 1.0 (TID 238) (an-worker1143.eqiad.wmnet, executor 16, partition 90, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 69.0 in stage 1.0 (TID 217) in 38 ms on an-worker1143.eqiad.wmnet (executor 16) (58/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 68.0 in stage 1.0 (TID 216) in 41 ms on an-worker1143.eqiad.wmnet (executor 16) (59/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 91.0 in stage 1.0 (TID 239) (an-worker1095.eqiad.wmnet, executor 6, partition 91, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 92.0 in stage 1.0 (TID 240) (an-worker1102.eqiad.wmnet, executor 5, partition 92, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 2.0 in stage 1.0 (TID 150) in 794 ms on an-worker1095.eqiad.wmnet (executor 6) (60/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 73.0 in stage 1.0 (TID 221) in 30 ms on an-worker1102.eqiad.wmnet (executor 5) (61/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 93.0 in stage 1.0 (TID 241) (an-worker1119.eqiad.wmnet, executor 13, partition 93, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 70.0 in stage 1.0 (TID 218) in 36 ms on an-worker1119.eqiad.wmnet (executor 13) (62/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 94.0 in stage 1.0 (TID 242) (an-worker1095.eqiad.wmnet, executor 7, partition 94, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 95.0 in stage 1.0 (TID 243) (an-worker1102.eqiad.wmnet, executor 5, partition 95, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 60.0 in stage 1.0 (TID 208) in 62 ms on an-worker1095.eqiad.wmnet (executor 7) (63/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 96.0 in stage 1.0 (TID 244) (an-worker1095.eqiad.wmnet, executor 7, partition 96, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 77.0 in stage 1.0 (TID 225) in 29 ms on an-worker1102.eqiad.wmnet (executor 5) (64/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 97.0 in stage 1.0 (TID 245) (an-worker1079.eqiad.wmnet, executor 1, partition 97, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 58.0 in stage 1.0 (TID 206) in 67 ms on an-worker1095.eqiad.wmnet (executor 7) (65/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 64.0 in stage 1.0 (TID 212) in 60 ms on an-worker1079.eqiad.wmnet (executor 1) (66/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 98.0 in stage 1.0 (TID 246) (an-worker1083.eqiad.wmnet, executor 2, partition 98, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 99.0 in stage 1.0 (TID 247) (an-worker1083.eqiad.wmnet, executor 2, partition 99, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 10.0 in stage 1.0 (TID 158) in 806 ms on an-worker1083.eqiad.wmnet (executor 2) (67/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 100.0 in stage 1.0 (TID 248) (an-worker1119.eqiad.wmnet, executor 13, partition 100, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 26.0 in stage 1.0 (TID 174) in 804 ms on an-worker1083.eqiad.wmnet (executor 2) (68/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 101.0 in stage 1.0 (TID 249) (an-worker1119.eqiad.wmnet, executor 11, partition 101, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 72.0 in stage 1.0 (TID 220) in 50 ms on an-worker1119.eqiad.wmnet (executor 13) (69/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 102.0 in stage 1.0 (TID 250) (an-worker1119.eqiad.wmnet, executor 11, partition 102, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 76.0 in stage 1.0 (TID 224) in 46 ms on an-worker1119.eqiad.wmnet (executor 11) (70/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 79.0 in stage 1.0 (TID 227) in 40 ms on an-worker1119.eqiad.wmnet (executor 11) (71/200)
24/01/12 13:41:19 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.64.21.121:56590
24/01/12 13:41:19 INFO TaskSetManager: Starting task 103.0 in stage 1.0 (TID 251) (an-worker1116.eqiad.wmnet, executor 10, partition 103, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 71.0 in stage 1.0 (TID 219) in 57 ms on an-worker1116.eqiad.wmnet (executor 10) (72/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 104.0 in stage 1.0 (TID 252) (an-worker1143.eqiad.wmnet, executor 16, partition 104, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 105.0 in stage 1.0 (TID 253) (an-worker1119.eqiad.wmnet, executor 12, partition 105, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 89.0 in stage 1.0 (TID 237) in 32 ms on an-worker1143.eqiad.wmnet (executor 16) (73/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 106.0 in stage 1.0 (TID 254) (an-worker1128.eqiad.wmnet, executor 15, partition 106, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 82.0 in stage 1.0 (TID 230) in 43 ms on an-worker1119.eqiad.wmnet (executor 12) (74/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 107.0 in stage 1.0 (TID 255) (an-worker1143.eqiad.wmnet, executor 16, partition 107, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 108.0 in stage 1.0 (TID 256) (an-worker1128.eqiad.wmnet, executor 15, partition 108, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 86.0 in stage 1.0 (TID 234) in 39 ms on an-worker1128.eqiad.wmnet (executor 15) (75/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 90.0 in stage 1.0 (TID 238) in 35 ms on an-worker1143.eqiad.wmnet (executor 16) (76/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 109.0 in stage 1.0 (TID 257) (an-worker1083.eqiad.wmnet, executor 3, partition 109, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 110.0 in stage 1.0 (TID 258) (an-worker1079.eqiad.wmnet, executor 1, partition 110, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 74.0 in stage 1.0 (TID 222) in 60 ms on an-worker1083.eqiad.wmnet (executor 3) (77/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 111.0 in stage 1.0 (TID 259) (an-worker1102.eqiad.wmnet, executor 5, partition 111, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 67.0 in stage 1.0 (TID 215) in 78 ms on an-worker1079.eqiad.wmnet (executor 1) (78/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 88.0 in stage 1.0 (TID 236) in 40 ms on an-worker1128.eqiad.wmnet (executor 15) (79/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 112.0 in stage 1.0 (TID 260) (an-worker1119.eqiad.wmnet, executor 14, partition 112, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 92.0 in stage 1.0 (TID 240) in 36 ms on an-worker1102.eqiad.wmnet (executor 5) (80/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 113.0 in stage 1.0 (TID 261) (an-worker1116.eqiad.wmnet, executor 10, partition 113, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 81.0 in stage 1.0 (TID 229) in 52 ms on an-worker1119.eqiad.wmnet (executor 14) (81/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 114.0 in stage 1.0 (TID 262) (an-worker1119.eqiad.wmnet, executor 12, partition 114, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 75.0 in stage 1.0 (TID 223) in 63 ms on an-worker1116.eqiad.wmnet (executor 10) (82/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 115.0 in stage 1.0 (TID 263) (an-worker1102.eqiad.wmnet, executor 5, partition 115, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 116.0 in stage 1.0 (TID 264) (an-worker1119.eqiad.wmnet, executor 14, partition 116, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 95.0 in stage 1.0 (TID 243) in 33 ms on an-worker1102.eqiad.wmnet (executor 5) (83/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 80.0 in stage 1.0 (TID 228) in 55 ms on an-worker1119.eqiad.wmnet (executor 12) (84/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 117.0 in stage 1.0 (TID 265) (an-worker1116.eqiad.wmnet, executor 9, partition 117, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 87.0 in stage 1.0 (TID 235) in 47 ms on an-worker1119.eqiad.wmnet (executor 14) (85/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 118.0 in stage 1.0 (TID 266) (an-worker1116.eqiad.wmnet, executor 9, partition 118, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 84.0 in stage 1.0 (TID 232) in 52 ms on an-worker1116.eqiad.wmnet (executor 9) (86/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 119.0 in stage 1.0 (TID 267) (an-worker1083.eqiad.wmnet, executor 3, partition 119, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 83.0 in stage 1.0 (TID 231) in 55 ms on an-worker1116.eqiad.wmnet (executor 9) (87/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 120.0 in stage 1.0 (TID 268) (an-worker1119.eqiad.wmnet, executor 13, partition 120, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 78.0 in stage 1.0 (TID 226) in 62 ms on an-worker1083.eqiad.wmnet (executor 3) (88/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 93.0 in stage 1.0 (TID 241) in 41 ms on an-worker1119.eqiad.wmnet (executor 13) (89/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 121.0 in stage 1.0 (TID 269) (an-worker1095.eqiad.wmnet, executor 6, partition 121, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 122.0 in stage 1.0 (TID 270) (an-worker1079.eqiad.wmnet, executor 1, partition 122, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 85.0 in stage 1.0 (TID 233) in 61 ms on an-worker1095.eqiad.wmnet (executor 6) (90/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 123.0 in stage 1.0 (TID 271) (an-worker1095.eqiad.wmnet, executor 7, partition 123, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 97.0 in stage 1.0 (TID 245) in 43 ms on an-worker1079.eqiad.wmnet (executor 1) (91/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 124.0 in stage 1.0 (TID 272) (an-worker1095.eqiad.wmnet, executor 7, partition 124, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 94.0 in stage 1.0 (TID 242) in 48 ms on an-worker1095.eqiad.wmnet (executor 7) (92/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 96.0 in stage 1.0 (TID 244) in 47 ms on an-worker1095.eqiad.wmnet (executor 7) (93/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 125.0 in stage 1.0 (TID 273) (an-worker1119.eqiad.wmnet, executor 11, partition 125, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 102.0 in stage 1.0 (TID 250) in 35 ms on an-worker1119.eqiad.wmnet (executor 11) (94/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 126.0 in stage 1.0 (TID 274) (an-worker1119.eqiad.wmnet, executor 11, partition 126, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 127.0 in stage 1.0 (TID 275) (an-worker1119.eqiad.wmnet, executor 13, partition 127, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 101.0 in stage 1.0 (TID 249) in 37 ms on an-worker1119.eqiad.wmnet (executor 11) (95/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 128.0 in stage 1.0 (TID 276) (an-worker1143.eqiad.wmnet, executor 16, partition 128, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 100.0 in stage 1.0 (TID 248) in 40 ms on an-worker1119.eqiad.wmnet (executor 13) (96/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 104.0 in stage 1.0 (TID 252) in 33 ms on an-worker1143.eqiad.wmnet (executor 16) (97/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 129.0 in stage 1.0 (TID 277) (an-worker1143.eqiad.wmnet, executor 16, partition 129, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 107.0 in stage 1.0 (TID 255) in 31 ms on an-worker1143.eqiad.wmnet (executor 16) (98/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 130.0 in stage 1.0 (TID 278) (an-worker1128.eqiad.wmnet, executor 15, partition 130, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 131.0 in stage 1.0 (TID 279) (an-worker1119.eqiad.wmnet, executor 12, partition 131, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 106.0 in stage 1.0 (TID 254) in 36 ms on an-worker1128.eqiad.wmnet (executor 15) (99/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 105.0 in stage 1.0 (TID 253) in 37 ms on an-worker1119.eqiad.wmnet (executor 12) (100/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 132.0 in stage 1.0 (TID 280) (an-worker1102.eqiad.wmnet, executor 5, partition 132, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 111.0 in stage 1.0 (TID 259) in 32 ms on an-worker1102.eqiad.wmnet (executor 5) (101/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 133.0 in stage 1.0 (TID 281) (an-worker1128.eqiad.wmnet, executor 15, partition 133, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 134.0 in stage 1.0 (TID 282) (an-worker1102.eqiad.wmnet, executor 5, partition 134, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 108.0 in stage 1.0 (TID 256) in 37 ms on an-worker1128.eqiad.wmnet (executor 15) (102/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 115.0 in stage 1.0 (TID 263) in 31 ms on an-worker1102.eqiad.wmnet (executor 5) (103/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 135.0 in stage 1.0 (TID 283) (an-worker1116.eqiad.wmnet, executor 10, partition 135, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 103.0 in stage 1.0 (TID 251) in 47 ms on an-worker1116.eqiad.wmnet (executor 10) (104/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 136.0 in stage 1.0 (TID 284) (an-worker1095.eqiad.wmnet, executor 6, partition 136, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 91.0 in stage 1.0 (TID 239) in 73 ms on an-worker1095.eqiad.wmnet (executor 6) (105/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 137.0 in stage 1.0 (TID 285) (an-worker1119.eqiad.wmnet, executor 14, partition 137, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 112.0 in stage 1.0 (TID 260) in 43 ms on an-worker1119.eqiad.wmnet (executor 14) (106/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 138.0 in stage 1.0 (TID 286) (an-worker1119.eqiad.wmnet, executor 14, partition 138, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 139.0 in stage 1.0 (TID 287) (an-worker1116.eqiad.wmnet, executor 10, partition 139, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 116.0 in stage 1.0 (TID 264) in 44 ms on an-worker1119.eqiad.wmnet (executor 14) (107/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 113.0 in stage 1.0 (TID 261) in 46 ms on an-worker1116.eqiad.wmnet (executor 10) (108/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 140.0 in stage 1.0 (TID 288) (an-worker1119.eqiad.wmnet, executor 13, partition 140, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 141.0 in stage 1.0 (TID 289) (an-worker1079.eqiad.wmnet, executor 1, partition 141, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 120.0 in stage 1.0 (TID 268) in 46 ms on an-worker1119.eqiad.wmnet (executor 13) (109/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 142.0 in stage 1.0 (TID 290) (an-worker1095.eqiad.wmnet, executor 6, partition 142, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 110.0 in stage 1.0 (TID 258) in 57 ms on an-worker1079.eqiad.wmnet (executor 1) (110/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 143.0 in stage 1.0 (TID 291) (an-worker1119.eqiad.wmnet, executor 12, partition 143, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 121.0 in stage 1.0 (TID 269) in 41 ms on an-worker1095.eqiad.wmnet (executor 6) (111/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 144.0 in stage 1.0 (TID 292) (an-worker1128.eqiad.wmnet, executor 15, partition 144, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 114.0 in stage 1.0 (TID 262) in 55 ms on an-worker1119.eqiad.wmnet (executor 12) (112/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 145.0 in stage 1.0 (TID 293) (an-worker1119.eqiad.wmnet, executor 11, partition 145, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 130.0 in stage 1.0 (TID 278) in 32 ms on an-worker1128.eqiad.wmnet (executor 15) (113/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 146.0 in stage 1.0 (TID 294) (an-worker1143.eqiad.wmnet, executor 16, partition 146, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 126.0 in stage 1.0 (TID 274) in 40 ms on an-worker1119.eqiad.wmnet (executor 11) (114/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 147.0 in stage 1.0 (TID 295) (an-worker1102.eqiad.wmnet, executor 5, partition 147, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 148.0 in stage 1.0 (TID 296) (an-worker1143.eqiad.wmnet, executor 16, partition 148, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 128.0 in stage 1.0 (TID 276) in 40 ms on an-worker1143.eqiad.wmnet (executor 16) (115/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 149.0 in stage 1.0 (TID 297) (an-worker1095.eqiad.wmnet, executor 7, partition 149, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 150.0 in stage 1.0 (TID 298) (an-worker1095.eqiad.wmnet, executor 7, partition 150, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 132.0 in stage 1.0 (TID 280) in 34 ms on an-worker1102.eqiad.wmnet (executor 5) (116/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 129.0 in stage 1.0 (TID 277) in 40 ms on an-worker1143.eqiad.wmnet (executor 16) (117/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 123.0 in stage 1.0 (TID 271) in 48 ms on an-worker1095.eqiad.wmnet (executor 7) (118/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 151.0 in stage 1.0 (TID 299) (an-worker1119.eqiad.wmnet, executor 11, partition 151, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 124.0 in stage 1.0 (TID 272) in 48 ms on an-worker1095.eqiad.wmnet (executor 7) (119/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 152.0 in stage 1.0 (TID 300) (an-worker1083.eqiad.wmnet, executor 3, partition 152, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 153.0 in stage 1.0 (TID 301) (an-worker1116.eqiad.wmnet, executor 10, partition 153, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 154.0 in stage 1.0 (TID 302) (an-worker1128.eqiad.wmnet, executor 15, partition 154, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 125.0 in stage 1.0 (TID 273) in 48 ms on an-worker1119.eqiad.wmnet (executor 11) (120/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 135.0 in stage 1.0 (TID 283) in 34 ms on an-worker1116.eqiad.wmnet (executor 10) (121/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 155.0 in stage 1.0 (TID 303) (an-worker1119.eqiad.wmnet, executor 13, partition 155, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 109.0 in stage 1.0 (TID 257) in 72 ms on an-worker1083.eqiad.wmnet (executor 3) (122/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 156.0 in stage 1.0 (TID 304) (an-worker1102.eqiad.wmnet, executor 5, partition 156, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 133.0 in stage 1.0 (TID 281) in 38 ms on an-worker1128.eqiad.wmnet (executor 15) (123/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 157.0 in stage 1.0 (TID 305) (an-worker1153.eqiad.wmnet, executor 8, partition 157, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 158.0 in stage 1.0 (TID 306) (an-worker1153.eqiad.wmnet, executor 8, partition 158, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 134.0 in stage 1.0 (TID 282) in 39 ms on an-worker1102.eqiad.wmnet (executor 5) (124/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 127.0 in stage 1.0 (TID 275) in 50 ms on an-worker1119.eqiad.wmnet (executor 13) (125/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 16.0 in stage 1.0 (TID 164) in 894 ms on an-worker1153.eqiad.wmnet (executor 8) (126/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 159.0 in stage 1.0 (TID 307) (an-worker1116.eqiad.wmnet, executor 9, partition 159, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 148) in 906 ms on an-worker1153.eqiad.wmnet (executor 8) (127/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 160.0 in stage 1.0 (TID 308) (an-worker1116.eqiad.wmnet, executor 10, partition 160, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 118.0 in stage 1.0 (TID 266) in 68 ms on an-worker1116.eqiad.wmnet (executor 9) (128/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 161.0 in stage 1.0 (TID 309) (an-worker1116.eqiad.wmnet, executor 9, partition 161, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 139.0 in stage 1.0 (TID 287) in 28 ms on an-worker1116.eqiad.wmnet (executor 10) (129/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 117.0 in stage 1.0 (TID 265) in 71 ms on an-worker1116.eqiad.wmnet (executor 9) (130/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 162.0 in stage 1.0 (TID 310) (an-worker1119.eqiad.wmnet, executor 14, partition 162, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 163.0 in stage 1.0 (TID 311) (an-worker1079.eqiad.wmnet, executor 1, partition 163, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 137.0 in stage 1.0 (TID 285) in 37 ms on an-worker1119.eqiad.wmnet (executor 14) (131/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 164.0 in stage 1.0 (TID 312) (an-worker1119.eqiad.wmnet, executor 12, partition 164, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 122.0 in stage 1.0 (TID 270) in 65 ms on an-worker1079.eqiad.wmnet (executor 1) (132/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 165.0 in stage 1.0 (TID 313) (an-worker1083.eqiad.wmnet, executor 3, partition 165, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 131.0 in stage 1.0 (TID 279) in 52 ms on an-worker1119.eqiad.wmnet (executor 12) (133/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 166.0 in stage 1.0 (TID 314) (an-worker1119.eqiad.wmnet, executor 14, partition 166, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 119.0 in stage 1.0 (TID 267) in 75 ms on an-worker1083.eqiad.wmnet (executor 3) (134/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 167.0 in stage 1.0 (TID 315) (an-worker1128.eqiad.wmnet, executor 15, partition 167, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 144.0 in stage 1.0 (TID 292) in 26 ms on an-worker1128.eqiad.wmnet (executor 15) (135/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 138.0 in stage 1.0 (TID 286) in 38 ms on an-worker1119.eqiad.wmnet (executor 14) (136/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 168.0 in stage 1.0 (TID 316) (an-worker1119.eqiad.wmnet, executor 12, partition 168, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 143.0 in stage 1.0 (TID 291) in 32 ms on an-worker1119.eqiad.wmnet (executor 12) (137/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 169.0 in stage 1.0 (TID 317) (an-worker1102.eqiad.wmnet, executor 5, partition 169, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 147.0 in stage 1.0 (TID 295) in 28 ms on an-worker1102.eqiad.wmnet (executor 5) (138/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 170.0 in stage 1.0 (TID 318) (an-worker1119.eqiad.wmnet, executor 13, partition 170, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 140.0 in stage 1.0 (TID 288) in 41 ms on an-worker1119.eqiad.wmnet (executor 13) (139/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 171.0 in stage 1.0 (TID 319) (an-worker1143.eqiad.wmnet, executor 16, partition 171, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 172.0 in stage 1.0 (TID 320) (an-worker1119.eqiad.wmnet, executor 12, partition 172, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 173.0 in stage 1.0 (TID 321) (an-worker1128.eqiad.wmnet, executor 15, partition 173, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 168.0 in stage 1.0 (TID 316) in 78 ms on an-worker1119.eqiad.wmnet (executor 12) (140/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 146.0 in stage 1.0 (TID 294) in 104 ms on an-worker1143.eqiad.wmnet (executor 16) (141/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 174.0 in stage 1.0 (TID 322) (an-worker1119.eqiad.wmnet, executor 12, partition 174, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 154.0 in stage 1.0 (TID 302) in 98 ms on an-worker1128.eqiad.wmnet (executor 15) (142/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 175.0 in stage 1.0 (TID 323) (an-worker1095.eqiad.wmnet, executor 7, partition 175, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 164.0 in stage 1.0 (TID 312) in 88 ms on an-worker1119.eqiad.wmnet (executor 12) (143/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 176.0 in stage 1.0 (TID 324) (an-worker1116.eqiad.wmnet, executor 10, partition 176, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO ExecutorAllocationManager: Requesting 16 new executors because tasks are backlogged (new desired total will be 16 for resource profile id: 0)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 149.0 in stage 1.0 (TID 297) in 105 ms on an-worker1095.eqiad.wmnet (executor 7) (144/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 177.0 in stage 1.0 (TID 325) (an-worker1143.eqiad.wmnet, executor 16, partition 177, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 153.0 in stage 1.0 (TID 301) in 104 ms on an-worker1116.eqiad.wmnet (executor 10) (145/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 178.0 in stage 1.0 (TID 326) (an-worker1083.eqiad.wmnet, executor 3, partition 178, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 148.0 in stage 1.0 (TID 296) in 109 ms on an-worker1143.eqiad.wmnet (executor 16) (146/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 179.0 in stage 1.0 (TID 327) (an-worker1119.eqiad.wmnet, executor 11, partition 179, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 152.0 in stage 1.0 (TID 300) in 106 ms on an-worker1083.eqiad.wmnet (executor 3) (147/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 180.0 in stage 1.0 (TID 328) (an-worker1119.eqiad.wmnet, executor 13, partition 180, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 151.0 in stage 1.0 (TID 299) in 109 ms on an-worker1119.eqiad.wmnet (executor 11) (148/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 181.0 in stage 1.0 (TID 329) (an-worker1083.eqiad.wmnet, executor 3, partition 181, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 155.0 in stage 1.0 (TID 303) in 107 ms on an-worker1119.eqiad.wmnet (executor 13) (149/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 182.0 in stage 1.0 (TID 330) (an-worker1128.eqiad.wmnet, executor 15, partition 182, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 165.0 in stage 1.0 (TID 313) in 96 ms on an-worker1083.eqiad.wmnet (executor 3) (150/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 183.0 in stage 1.0 (TID 331) (an-worker1095.eqiad.wmnet, executor 7, partition 183, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 184.0 in stage 1.0 (TID 332) (an-worker1116.eqiad.wmnet, executor 10, partition 184, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 167.0 in stage 1.0 (TID 315) in 95 ms on an-worker1128.eqiad.wmnet (executor 15) (151/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 150.0 in stage 1.0 (TID 298) in 115 ms on an-worker1095.eqiad.wmnet (executor 7) (152/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 185.0 in stage 1.0 (TID 333) (an-worker1102.eqiad.wmnet, executor 5, partition 185, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 160.0 in stage 1.0 (TID 308) in 105 ms on an-worker1116.eqiad.wmnet (executor 10) (153/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 186.0 in stage 1.0 (TID 334) (an-worker1119.eqiad.wmnet, executor 14, partition 186, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 187.0 in stage 1.0 (TID 335) (an-worker1095.eqiad.wmnet, executor 6, partition 187, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 166.0 in stage 1.0 (TID 314) in 99 ms on an-worker1119.eqiad.wmnet (executor 14) (154/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 156.0 in stage 1.0 (TID 304) in 111 ms on an-worker1102.eqiad.wmnet (executor 5) (155/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 188.0 in stage 1.0 (TID 336) (an-worker1119.eqiad.wmnet, executor 11, partition 188, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 142.0 in stage 1.0 (TID 290) in 128 ms on an-worker1095.eqiad.wmnet (executor 6) (156/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 189.0 in stage 1.0 (TID 337) (an-worker1119.eqiad.wmnet, executor 13, partition 189, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 145.0 in stage 1.0 (TID 293) in 123 ms on an-worker1119.eqiad.wmnet (executor 11) (157/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 190.0 in stage 1.0 (TID 338) (an-worker1102.eqiad.wmnet, executor 5, partition 190, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 170.0 in stage 1.0 (TID 318) in 91 ms on an-worker1119.eqiad.wmnet (executor 13) (158/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 191.0 in stage 1.0 (TID 339) (an-worker1119.eqiad.wmnet, executor 14, partition 191, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Starting task 192.0 in stage 1.0 (TID 340) (an-worker1095.eqiad.wmnet, executor 6, partition 192, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 162.0 in stage 1.0 (TID 310) in 106 ms on an-worker1119.eqiad.wmnet (executor 14) (159/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 169.0 in stage 1.0 (TID 317) in 95 ms on an-worker1102.eqiad.wmnet (executor 5) (160/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 193.0 in stage 1.0 (TID 341) (an-worker1116.eqiad.wmnet, executor 9, partition 193, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 136.0 in stage 1.0 (TID 284) in 149 ms on an-worker1095.eqiad.wmnet (executor 6) (161/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 194.0 in stage 1.0 (TID 342) (an-worker1079.eqiad.wmnet, executor 1, partition 194, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 159.0 in stage 1.0 (TID 307) in 114 ms on an-worker1116.eqiad.wmnet (executor 9) (162/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 195.0 in stage 1.0 (TID 343) (an-worker1083.eqiad.wmnet, executor 2, partition 195, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 141.0 in stage 1.0 (TID 289) in 136 ms on an-worker1079.eqiad.wmnet (executor 1) (163/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 196.0 in stage 1.0 (TID 344) (an-worker1116.eqiad.wmnet, executor 9, partition 196, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 98.0 in stage 1.0 (TID 246) in 210 ms on an-worker1083.eqiad.wmnet (executor 2) (164/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 197.0 in stage 1.0 (TID 345) (an-worker1079.eqiad.wmnet, executor 1, partition 197, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 161.0 in stage 1.0 (TID 309) in 116 ms on an-worker1116.eqiad.wmnet (executor 9) (165/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 198.0 in stage 1.0 (TID 346) (an-worker1083.eqiad.wmnet, executor 2, partition 198, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 163.0 in stage 1.0 (TID 311) in 113 ms on an-worker1079.eqiad.wmnet (executor 1) (166/200)
24/01/12 13:41:19 INFO TaskSetManager: Starting task 199.0 in stage 1.0 (TID 347) (an-worker1119.eqiad.wmnet, executor 12, partition 199, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:19 INFO TaskSetManager: Finished task 99.0 in stage 1.0 (TID 247) in 212 ms on an-worker1083.eqiad.wmnet (executor 2) (167/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 172.0 in stage 1.0 (TID 320) in 40 ms on an-worker1119.eqiad.wmnet (executor 12) (168/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 158.0 in stage 1.0 (TID 306) in 128 ms on an-worker1153.eqiad.wmnet (executor 8) (169/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 157.0 in stage 1.0 (TID 305) in 128 ms on an-worker1153.eqiad.wmnet (executor 8) (170/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 175.0 in stage 1.0 (TID 323) in 35 ms on an-worker1095.eqiad.wmnet (executor 7) (171/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 178.0 in stage 1.0 (TID 326) in 33 ms on an-worker1083.eqiad.wmnet (executor 3) (172/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 171.0 in stage 1.0 (TID 319) in 57 ms on an-worker1143.eqiad.wmnet (executor 16) (173/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 173.0 in stage 1.0 (TID 321) in 51 ms on an-worker1128.eqiad.wmnet (executor 15) (174/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 179.0 in stage 1.0 (TID 327) in 38 ms on an-worker1119.eqiad.wmnet (executor 11) (175/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 174.0 in stage 1.0 (TID 322) in 46 ms on an-worker1119.eqiad.wmnet (executor 12) (176/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 177.0 in stage 1.0 (TID 325) in 44 ms on an-worker1143.eqiad.wmnet (executor 16) (177/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 176.0 in stage 1.0 (TID 324) in 46 ms on an-worker1116.eqiad.wmnet (executor 10) (178/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 185.0 in stage 1.0 (TID 333) in 37 ms on an-worker1102.eqiad.wmnet (executor 5) (179/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 180.0 in stage 1.0 (TID 328) in 42 ms on an-worker1119.eqiad.wmnet (executor 13) (180/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 190.0 in stage 1.0 (TID 338) in 35 ms on an-worker1102.eqiad.wmnet (executor 5) (181/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 182.0 in stage 1.0 (TID 330) in 41 ms on an-worker1128.eqiad.wmnet (executor 15) (182/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 184.0 in stage 1.0 (TID 332) in 40 ms on an-worker1116.eqiad.wmnet (executor 10) (183/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 188.0 in stage 1.0 (TID 336) in 38 ms on an-worker1119.eqiad.wmnet (executor 11) (184/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 181.0 in stage 1.0 (TID 329) in 46 ms on an-worker1083.eqiad.wmnet (executor 3) (185/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 189.0 in stage 1.0 (TID 337) in 39 ms on an-worker1119.eqiad.wmnet (executor 13) (186/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 186.0 in stage 1.0 (TID 334) in 43 ms on an-worker1119.eqiad.wmnet (executor 14) (187/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1119.eqiad.wmnet:37683 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 199.0 in stage 1.0 (TID 347) in 35 ms on an-worker1119.eqiad.wmnet (executor 12) (188/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 191.0 in stage 1.0 (TID 339) in 44 ms on an-worker1119.eqiad.wmnet (executor 14) (189/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1095.eqiad.wmnet:38457 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1095.eqiad.wmnet:45475 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1119.eqiad.wmnet:38383 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on stat1007.eqiad.wmnet:13006 in memory (size: 13.4 KiB, free: 8.4 GiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 183.0 in stage 1.0 (TID 331) in 52 ms on an-worker1095.eqiad.wmnet (executor 7) (190/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1119.eqiad.wmnet:34183 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1128.eqiad.wmnet:35005 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 193.0 in stage 1.0 (TID 341) in 45 ms on an-worker1116.eqiad.wmnet (executor 9) (191/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1083.eqiad.wmnet:40809 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 196.0 in stage 1.0 (TID 344) in 43 ms on an-worker1116.eqiad.wmnet (executor 9) (192/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1102.eqiad.wmnet:33061 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1116.eqiad.wmnet:38999 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1119.eqiad.wmnet:45881 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1083.eqiad.wmnet:45911 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1143.eqiad.wmnet:42097 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1153.eqiad.wmnet:46731 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 192.0 in stage 1.0 (TID 340) in 53 ms on an-worker1095.eqiad.wmnet (executor 6) (193/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1116.eqiad.wmnet:42965 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 187.0 in stage 1.0 (TID 335) in 60 ms on an-worker1095.eqiad.wmnet (executor 6) (194/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 198.0 in stage 1.0 (TID 346) in 60 ms on an-worker1083.eqiad.wmnet (executor 2) (195/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 195.0 in stage 1.0 (TID 343) in 65 ms on an-worker1083.eqiad.wmnet (executor 2) (196/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1079.eqiad.wmnet:44645 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 197.0 in stage 1.0 (TID 345) in 88 ms on an-worker1079.eqiad.wmnet (executor 1) (197/200)
24/01/12 13:41:19 INFO TaskSetManager: Finished task 194.0 in stage 1.0 (TID 342) in 97 ms on an-worker1079.eqiad.wmnet (executor 1) (198/200)
24/01/12 13:41:19 INFO BlockManagerInfo: Removed broadcast_2_piece0 on an-worker1085.eqiad.wmnet:43491 in memory (size: 13.4 KiB, free: 2004.6 MiB)
24/01/12 13:41:20 INFO TaskSetManager: Finished task 19.0 in stage 1.0 (TID 167) in 1862 ms on an-worker1085.eqiad.wmnet (executor 4) (199/200)
24/01/12 13:41:20 INFO TaskSetManager: Finished task 3.0 in stage 1.0 (TID 151) in 1870 ms on an-worker1085.eqiad.wmnet (executor 4) (200/200)
24/01/12 13:41:20 INFO YarnScheduler: Removed TaskSet 1.0, whose tasks have all completed, from pool
24/01/12 13:41:20 INFO DAGScheduler: ResultStage 1 (rdd at PagesPartitionsDefiner.scala:88) finished in 1.901 s
24/01/12 13:41:20 INFO DAGScheduler: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job
24/01/12 13:41:20 INFO YarnScheduler: Killing all running tasks in stage 1: Stage finished
24/01/12 13:41:20 INFO DAGScheduler: Job 0 finished: rdd at PagesPartitionsDefiner.scala:88, took 39.418946 s
24/01/12 13:41:20 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 32.0 KiB, free 8.4 GiB)
24/01/12 13:41:20 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 30.9 KiB, free 8.4 GiB)
24/01/12 13:41:20 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 30.9 KiB, free: 8.4 GiB)
24/01/12 13:41:20 INFO SparkContext: Created broadcast 4 from broadcast at SparkBatchScan.java:142
24/01/12 13:41:20 INFO SnapshotScan: Scanning table spark_catalog.wmf_dumps.wikitext_raw_rc2 snapshot 4861734485253907190 created at 2024-01-12T13:26:32.899+00:00 with filter (((revision_timestamp IS NOT NULL AND wiki_db IS NOT NULL) AND revision_timestamp < (16-digit-int)) AND wiki_db = (hash-10b04362))
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on stat1007.eqiad.wmnet:13006 in memory (size: 15.1 KiB, free: 8.4 GiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1128.eqiad.wmnet:35005 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1116.eqiad.wmnet:38999 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1119.eqiad.wmnet:45881 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1083.eqiad.wmnet:40809 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1102.eqiad.wmnet:33061 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1083.eqiad.wmnet:45911 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1119.eqiad.wmnet:37683 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1116.eqiad.wmnet:42965 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1143.eqiad.wmnet:42097 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1119.eqiad.wmnet:38383 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1085.eqiad.wmnet:43491 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1153.eqiad.wmnet:46731 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1095.eqiad.wmnet:45475 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1095.eqiad.wmnet:38457 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1119.eqiad.wmnet:34183 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:21 INFO BlockManagerInfo: Removed broadcast_3_piece0 on an-worker1079.eqiad.wmnet:44645 in memory (size: 15.1 KiB, free: 2004.6 MiB)
24/01/12 13:41:22 INFO LoggingMetricsReporter: Received metrics report: ScanReport{tableName=spark_catalog.wmf_dumps.wikitext_raw_rc2, snapshotId=4861734485253907190, filter=(((not_null(ref(name="revision_timestamp")) and not_null(ref(name="wiki_db"))) and ref(name="revision_timestamp") < "(16-digit-int)") and ref(name="wiki_db") == "(hash-10b04362)"), schemaId=0, projectedFieldIds=[1, 18, 8, 10], projectedFieldNames=[page_id, wiki_db, revision_id, revision_timestamp], scanMetrics=ScanMetricsResult{totalPlanningDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT1.558591386S, count=1}, resultDataFiles=CounterResult{unit=COUNT, value=1236}, resultDeleteFiles=CounterResult{unit=COUNT, value=0}, totalDataManifests=CounterResult{unit=COUNT, value=132}, totalDeleteManifests=CounterResult{unit=COUNT, value=0}, scannedDataManifests=CounterResult{unit=COUNT, value=81}, skippedDataManifests=CounterResult{unit=COUNT, value=51}, totalFileSizeInBytes=CounterResult{unit=BYTES, value=16576313022}, totalDeleteFileSizeInBytes=CounterResult{unit=BYTES, value=0}, skippedDataFiles=CounterResult{unit=COUNT, value=484574}, skippedDeleteFiles=CounterResult{unit=COUNT, value=0}, scannedDeleteManifests=CounterResult{unit=COUNT, value=0}, skippedDeleteManifests=CounterResult{unit=COUNT, value=0}, indexedDeleteFiles=CounterResult{unit=COUNT, value=0}, equalityDeleteFiles=CounterResult{unit=COUNT, value=0}, positionalDeleteFiles=CounterResult{unit=COUNT, value=0}}, metadata={engine-version=3.1.2, iceberg-version=Apache Iceberg 1.2.1 (commit 4e2cdccd7453603af42a090fc5530f2bd20cf1be), app-id=application_1704805203730_16243, engine-name=spark}}
24/01/12 13:41:23 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 32.0 KiB, free 8.4 GiB)
24/01/12 13:41:23 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 30.9 KiB, free 8.4 GiB)
24/01/12 13:41:23 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 30.9 KiB, free: 8.4 GiB)
24/01/12 13:41:23 INFO SparkContext: Created broadcast 5 from broadcast at SparkBatchScan.java:142
24/01/12 13:41:24 INFO SparkContext: Starting job: count at PagesPartitionsDefiner.scala:101
24/01/12 13:41:24 INFO DAGScheduler: Registering RDD 18 (count at PagesPartitionsDefiner.scala:101) as input to shuffle 2
24/01/12 13:41:24 INFO DAGScheduler: Got job 1 (count at PagesPartitionsDefiner.scala:101) with 1 output partitions
24/01/12 13:41:24 INFO DAGScheduler: Final stage: ResultStage 3 (count at PagesPartitionsDefiner.scala:101)
24/01/12 13:41:24 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 2)
24/01/12 13:41:24 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 2)
24/01/12 13:41:24 INFO DAGScheduler: Submitting ShuffleMapStage 2 (MapPartitionsRDD[18] at count at PagesPartitionsDefiner.scala:101), which has no missing parents
24/01/12 13:41:24 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 14.5 KiB, free 8.4 GiB)
24/01/12 13:41:24 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 6.5 KiB, free 8.4 GiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 6.5 KiB, free: 8.4 GiB)
24/01/12 13:41:24 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:1388
24/01/12 13:41:24 INFO DAGScheduler: Submitting 149 missing tasks from ShuffleMapStage 2 (MapPartitionsRDD[18] at count at PagesPartitionsDefiner.scala:101) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
24/01/12 13:41:24 INFO YarnScheduler: Adding task set 2.0 with 149 tasks resource profile 0
24/01/12 13:41:24 INFO TaskSetManager: Starting task 9.0 in stage 2.0 (TID 348) (an-worker1079.eqiad.wmnet, executor 1, partition 9, NODE_LOCAL, 32119 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 349) (an-worker1128.eqiad.wmnet, executor 15, partition 0, NODE_LOCAL, 21828 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 4.0 in stage 2.0 (TID 350) (an-worker1119.eqiad.wmnet, executor 12, partition 4, NODE_LOCAL, 30310 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 2.0 in stage 2.0 (TID 351) (an-worker1116.eqiad.wmnet, executor 10, partition 2, NODE_LOCAL, 18193 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 6.0 in stage 2.0 (TID 352) (an-worker1116.eqiad.wmnet, executor 9, partition 6, NODE_LOCAL, 29917 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 1.0 in stage 2.0 (TID 353) (an-worker1102.eqiad.wmnet, executor 5, partition 1, NODE_LOCAL, 18197 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 11.0 in stage 2.0 (TID 354) (an-worker1119.eqiad.wmnet, executor 13, partition 11, NODE_LOCAL, 29525 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 12.0 in stage 2.0 (TID 355) (an-worker1119.eqiad.wmnet, executor 14, partition 12, NODE_LOCAL, 24011 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 5.0 in stage 2.0 (TID 356) (an-worker1083.eqiad.wmnet, executor 2, partition 5, NODE_LOCAL, 27472 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 13.0 in stage 2.0 (TID 357) (an-worker1095.eqiad.wmnet, executor 7, partition 13, NODE_LOCAL, 27399 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 7.0 in stage 2.0 (TID 358) (an-worker1083.eqiad.wmnet, executor 3, partition 7, NODE_LOCAL, 18193 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 14.0 in stage 2.0 (TID 359) (an-worker1119.eqiad.wmnet, executor 11, partition 14, NODE_LOCAL, 33577 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 8.0 in stage 2.0 (TID 360) (an-worker1085.eqiad.wmnet, executor 4, partition 8, NODE_LOCAL, 20397 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 16.0 in stage 2.0 (TID 361) (an-worker1143.eqiad.wmnet, executor 16, partition 16, NODE_LOCAL, 28190 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 17.0 in stage 2.0 (TID 362) (an-worker1095.eqiad.wmnet, executor 6, partition 17, NODE_LOCAL, 25421 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 3.0 in stage 2.0 (TID 363) (an-worker1153.eqiad.wmnet, executor 8, partition 3, NODE_LOCAL, 18580 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 10.0 in stage 2.0 (TID 364) (an-worker1079.eqiad.wmnet, executor 1, partition 10, NODE_LOCAL, 18936 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 15.0 in stage 2.0 (TID 365) (an-worker1128.eqiad.wmnet, executor 15, partition 15, NODE_LOCAL, 30400 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 33.0 in stage 2.0 (TID 366) (an-worker1119.eqiad.wmnet, executor 12, partition 33, NODE_LOCAL, 26594 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 19.0 in stage 2.0 (TID 367) (an-worker1116.eqiad.wmnet, executor 10, partition 19, NODE_LOCAL, 28046 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 23.0 in stage 2.0 (TID 368) (an-worker1116.eqiad.wmnet, executor 9, partition 23, NODE_LOCAL, 21106 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 21.0 in stage 2.0 (TID 369) (an-worker1102.eqiad.wmnet, executor 5, partition 21, NODE_LOCAL, 19628 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 44.0 in stage 2.0 (TID 370) (an-worker1119.eqiad.wmnet, executor 13, partition 44, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 52.0 in stage 2.0 (TID 371) (an-worker1119.eqiad.wmnet, executor 14, partition 52, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 20.0 in stage 2.0 (TID 372) (an-worker1083.eqiad.wmnet, executor 2, partition 20, NODE_LOCAL, 19960 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 31.0 in stage 2.0 (TID 373) (an-worker1095.eqiad.wmnet, executor 7, partition 31, NODE_LOCAL, 24614 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 24.0 in stage 2.0 (TID 374) (an-worker1083.eqiad.wmnet, executor 3, partition 24, NODE_LOCAL, 24435 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 88.0 in stage 2.0 (TID 375) (an-worker1119.eqiad.wmnet, executor 11, partition 88, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 30.0 in stage 2.0 (TID 376) (an-worker1085.eqiad.wmnet, executor 4, partition 30, NODE_LOCAL, 21060 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 65.0 in stage 2.0 (TID 377) (an-worker1143.eqiad.wmnet, executor 16, partition 65, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 68.0 in stage 2.0 (TID 378) (an-worker1095.eqiad.wmnet, executor 6, partition 68, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO TaskSetManager: Starting task 22.0 in stage 2.0 (TID 379) (an-worker1153.eqiad.wmnet, executor 8, partition 22, NODE_LOCAL, 17837 bytes) taskResourceAssignments Map()
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1079.eqiad.wmnet:44645 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1119.eqiad.wmnet:45881 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1102.eqiad.wmnet:33061 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1116.eqiad.wmnet:38999 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1085.eqiad.wmnet:43491 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1128.eqiad.wmnet:35005 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1083.eqiad.wmnet:45911 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1143.eqiad.wmnet:42097 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1116.eqiad.wmnet:42965 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1095.eqiad.wmnet:45475 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1119.eqiad.wmnet:34183 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1119.eqiad.wmnet:37683 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1095.eqiad.wmnet:38457 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1083.eqiad.wmnet:40809 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on an-worker1153.eqiad.wmnet:46731 (size: 6.5 KiB, free: 2004.6 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1085.eqiad.wmnet:43491 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1079.eqiad.wmnet:44645 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1095.eqiad.wmnet:45475 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1116.eqiad.wmnet:38999 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1119.eqiad.wmnet:45881 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1102.eqiad.wmnet:33061 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1128.eqiad.wmnet:35005 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1083.eqiad.wmnet:45911 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1083.eqiad.wmnet:40809 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1153.eqiad.wmnet:46731 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1116.eqiad.wmnet:42965 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1143.eqiad.wmnet:42097 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1119.eqiad.wmnet:34183 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1095.eqiad.wmnet:38457 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:24 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on an-worker1119.eqiad.wmnet:37683 (size: 30.9 KiB, free: 2004.5 MiB)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 90.0 in stage 2.0 (TID 380) (an-worker1119.eqiad.wmnet, executor 11, partition 90, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 88.0 in stage 2.0 (TID 375) in 516 ms on an-worker1119.eqiad.wmnet (executor 11) (1/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 110.0 in stage 2.0 (TID 381) (an-worker1119.eqiad.wmnet, executor 13, partition 110, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 44.0 in stage 2.0 (TID 370) in 565 ms on an-worker1119.eqiad.wmnet (executor 13) (2/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 77.0 in stage 2.0 (TID 382) (an-worker1095.eqiad.wmnet, executor 6, partition 77, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 68.0 in stage 2.0 (TID 378) in 597 ms on an-worker1095.eqiad.wmnet (executor 6) (3/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 118.0 in stage 2.0 (TID 383) (an-worker1119.eqiad.wmnet, executor 11, partition 118, NODE_LOCAL, 20808 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 90.0 in stage 2.0 (TID 380) in 95 ms on an-worker1119.eqiad.wmnet (executor 11) (4/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 122.0 in stage 2.0 (TID 384) (an-worker1119.eqiad.wmnet, executor 14, partition 122, NODE_LOCAL, 20026 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 52.0 in stage 2.0 (TID 371) in 632 ms on an-worker1119.eqiad.wmnet (executor 14) (5/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 37.0 in stage 2.0 (TID 385) (an-worker1116.eqiad.wmnet, executor 10, partition 37, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 2.0 in stage 2.0 (TID 351) in 647 ms on an-worker1116.eqiad.wmnet (executor 10) (6/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 123.0 in stage 2.0 (TID 386) (an-worker1119.eqiad.wmnet, executor 13, partition 123, NODE_LOCAL, 18907 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 110.0 in stage 2.0 (TID 381) in 75 ms on an-worker1119.eqiad.wmnet (executor 13) (7/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 27.0 in stage 2.0 (TID 387) (an-worker1153.eqiad.wmnet, executor 8, partition 27, NODE_LOCAL, 18196 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 3.0 in stage 2.0 (TID 363) in 660 ms on an-worker1153.eqiad.wmnet (executor 8) (8/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 70.0 in stage 2.0 (TID 388) (an-worker1143.eqiad.wmnet, executor 16, partition 70, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 65.0 in stage 2.0 (TID 377) in 661 ms on an-worker1143.eqiad.wmnet (executor 16) (9/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 29.0 in stage 2.0 (TID 389) (an-worker1153.eqiad.wmnet, executor 8, partition 29, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 22.0 in stage 2.0 (TID 379) in 717 ms on an-worker1153.eqiad.wmnet (executor 8) (10/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 125.0 in stage 2.0 (TID 390) (an-worker1119.eqiad.wmnet, executor 11, partition 125, NODE_LOCAL, 27897 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 118.0 in stage 2.0 (TID 383) in 172 ms on an-worker1119.eqiad.wmnet (executor 11) (11/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 34.0 in stage 2.0 (TID 391) (an-worker1079.eqiad.wmnet, executor 1, partition 34, NODE_LOCAL, 18555 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 10.0 in stage 2.0 (TID 364) in 791 ms on an-worker1079.eqiad.wmnet (executor 1) (12/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 39.0 in stage 2.0 (TID 392) (an-worker1102.eqiad.wmnet, executor 5, partition 39, NODE_LOCAL, 22618 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 1.0 in stage 2.0 (TID 353) in 808 ms on an-worker1102.eqiad.wmnet (executor 5) (13/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 85.0 in stage 2.0 (TID 393) (an-worker1095.eqiad.wmnet, executor 6, partition 85, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 77.0 in stage 2.0 (TID 382) in 204 ms on an-worker1095.eqiad.wmnet (executor 6) (14/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 45.0 in stage 2.0 (TID 394) (an-worker1083.eqiad.wmnet, executor 3, partition 45, NODE_LOCAL, 18196 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 7.0 in stage 2.0 (TID 358) in 809 ms on an-worker1083.eqiad.wmnet (executor 3) (15/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 43.0 in stage 2.0 (TID 395) (an-worker1116.eqiad.wmnet, executor 10, partition 43, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 37.0 in stage 2.0 (TID 385) in 194 ms on an-worker1116.eqiad.wmnet (executor 10) (16/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 131.0 in stage 2.0 (TID 396) (an-worker1119.eqiad.wmnet, executor 13, partition 131, NODE_LOCAL, 25888 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 123.0 in stage 2.0 (TID 386) in 211 ms on an-worker1119.eqiad.wmnet (executor 13) (17/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 132.0 in stage 2.0 (TID 397) (an-worker1119.eqiad.wmnet, executor 14, partition 132, NODE_LOCAL, 22976 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 122.0 in stage 2.0 (TID 384) in 231 ms on an-worker1119.eqiad.wmnet (executor 14) (18/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 55.0 in stage 2.0 (TID 398) (an-worker1116.eqiad.wmnet, executor 10, partition 55, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 43.0 in stage 2.0 (TID 395) in 70 ms on an-worker1116.eqiad.wmnet (executor 10) (19/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 87.0 in stage 2.0 (TID 399) (an-worker1143.eqiad.wmnet, executor 16, partition 87, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 70.0 in stage 2.0 (TID 388) in 254 ms on an-worker1143.eqiad.wmnet (executor 16) (20/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 61.0 in stage 2.0 (TID 400) (an-worker1116.eqiad.wmnet, executor 9, partition 61, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 23.0 in stage 2.0 (TID 368) in 920 ms on an-worker1116.eqiad.wmnet (executor 9) (21/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 133.0 in stage 2.0 (TID 401) (an-worker1119.eqiad.wmnet, executor 14, partition 133, NODE_LOCAL, 27672 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 12.0 in stage 2.0 (TID 355) in 928 ms on an-worker1119.eqiad.wmnet (executor 14) (22/149)
24/01/12 13:41:25 INFO ExecutorAllocationManager: Requesting 16 new executors because tasks are backlogged (new desired total will be 16 for resource profile id: 0)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 41.0 in stage 2.0 (TID 402) (an-worker1153.eqiad.wmnet, executor 8, partition 41, NODE_LOCAL, 18581 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 27.0 in stage 2.0 (TID 387) in 295 ms on an-worker1153.eqiad.wmnet (executor 8) (23/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 72.0 in stage 2.0 (TID 403) (an-worker1102.eqiad.wmnet, executor 5, partition 72, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 21.0 in stage 2.0 (TID 369) in 987 ms on an-worker1102.eqiad.wmnet (executor 5) (24/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 42.0 in stage 2.0 (TID 404) (an-worker1153.eqiad.wmnet, executor 8, partition 42, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 29.0 in stage 2.0 (TID 389) in 275 ms on an-worker1153.eqiad.wmnet (executor 8) (25/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 32.0 in stage 2.0 (TID 405) (an-worker1128.eqiad.wmnet, executor 15, partition 32, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 349) in 1019 ms on an-worker1128.eqiad.wmnet (executor 15) (26/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 49.0 in stage 2.0 (TID 406) (an-worker1083.eqiad.wmnet, executor 3, partition 49, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 24.0 in stage 2.0 (TID 374) in 1098 ms on an-worker1083.eqiad.wmnet (executor 3) (27/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 53.0 in stage 2.0 (TID 407) (an-worker1083.eqiad.wmnet, executor 3, partition 53, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 45.0 in stage 2.0 (TID 394) in 299 ms on an-worker1083.eqiad.wmnet (executor 3) (28/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 36.0 in stage 2.0 (TID 408) (an-worker1079.eqiad.wmnet, executor 1, partition 36, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 34.0 in stage 2.0 (TID 391) in 315 ms on an-worker1079.eqiad.wmnet (executor 1) (29/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 73.0 in stage 2.0 (TID 409) (an-worker1116.eqiad.wmnet, executor 10, partition 73, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 55.0 in stage 2.0 (TID 398) in 209 ms on an-worker1116.eqiad.wmnet (executor 10) (30/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 89.0 in stage 2.0 (TID 410) (an-worker1116.eqiad.wmnet, executor 9, partition 89, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 6.0 in stage 2.0 (TID 352) in 1117 ms on an-worker1116.eqiad.wmnet (executor 9) (31/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 67.0 in stage 2.0 (TID 411) (an-worker1083.eqiad.wmnet, executor 2, partition 67, NODE_LOCAL, 18196 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 20.0 in stage 2.0 (TID 372) in 1110 ms on an-worker1083.eqiad.wmnet (executor 2) (32/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 121.0 in stage 2.0 (TID 412) (an-worker1095.eqiad.wmnet, executor 6, partition 121, NODE_LOCAL, 26639 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 17.0 in stage 2.0 (TID 362) in 1128 ms on an-worker1095.eqiad.wmnet (executor 6) (33/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 124.0 in stage 2.0 (TID 413) (an-worker1095.eqiad.wmnet, executor 7, partition 124, NODE_LOCAL, 28292 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 13.0 in stage 2.0 (TID 357) in 1137 ms on an-worker1095.eqiad.wmnet (executor 7) (34/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 46.0 in stage 2.0 (TID 414) (an-worker1153.eqiad.wmnet, executor 8, partition 46, NODE_LOCAL, 18528 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 41.0 in stage 2.0 (TID 402) in 185 ms on an-worker1153.eqiad.wmnet (executor 8) (35/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 93.0 in stage 2.0 (TID 415) (an-worker1116.eqiad.wmnet, executor 9, partition 93, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 61.0 in stage 2.0 (TID 400) in 234 ms on an-worker1116.eqiad.wmnet (executor 9) (36/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 126.0 in stage 2.0 (TID 416) (an-worker1095.eqiad.wmnet, executor 6, partition 126, NODE_LOCAL, 30322 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 85.0 in stage 2.0 (TID 393) in 366 ms on an-worker1095.eqiad.wmnet (executor 6) (37/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 119.0 in stage 2.0 (TID 417) (an-worker1143.eqiad.wmnet, executor 16, partition 119, NODE_LOCAL, 20749 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 87.0 in stage 2.0 (TID 399) in 276 ms on an-worker1143.eqiad.wmnet (executor 16) (38/149)
24/01/12 13:41:25 INFO TaskSetManager: Starting task 79.0 in stage 2.0 (TID 418) (an-worker1083.eqiad.wmnet, executor 3, partition 79, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:25 INFO TaskSetManager: Finished task 49.0 in stage 2.0 (TID 406) in 125 ms on an-worker1083.eqiad.wmnet (executor 3) (39/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 48.0 in stage 2.0 (TID 419) (an-worker1128.eqiad.wmnet, executor 15, partition 48, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 32.0 in stage 2.0 (TID 405) in 221 ms on an-worker1128.eqiad.wmnet (executor 15) (40/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 75.0 in stage 2.0 (TID 420) (an-worker1102.eqiad.wmnet, executor 5, partition 75, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 72.0 in stage 2.0 (TID 403) in 266 ms on an-worker1102.eqiad.wmnet (executor 5) (41/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 83.0 in stage 2.0 (TID 421) (an-worker1083.eqiad.wmnet, executor 3, partition 83, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 53.0 in stage 2.0 (TID 407) in 167 ms on an-worker1083.eqiad.wmnet (executor 3) (42/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 109.0 in stage 2.0 (TID 422) (an-worker1116.eqiad.wmnet, executor 9, partition 109, NODE_LOCAL, 16738 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 89.0 in stage 2.0 (TID 410) in 172 ms on an-worker1116.eqiad.wmnet (executor 9) (43/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 134.0 in stage 2.0 (TID 423) (an-worker1119.eqiad.wmnet, executor 13, partition 134, NODE_LOCAL, 26216 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 11.0 in stage 2.0 (TID 354) in 1293 ms on an-worker1119.eqiad.wmnet (executor 13) (44/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 86.0 in stage 2.0 (TID 424) (an-worker1083.eqiad.wmnet, executor 2, partition 86, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 67.0 in stage 2.0 (TID 411) in 190 ms on an-worker1083.eqiad.wmnet (executor 2) (45/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 47.0 in stage 2.0 (TID 425) (an-worker1153.eqiad.wmnet, executor 8, partition 47, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 42.0 in stage 2.0 (TID 404) in 308 ms on an-worker1153.eqiad.wmnet (executor 8) (46/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 111.0 in stage 2.0 (TID 426) (an-worker1116.eqiad.wmnet, executor 9, partition 111, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 109.0 in stage 2.0 (TID 422) in 47 ms on an-worker1116.eqiad.wmnet (executor 9) (47/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 135.0 in stage 2.0 (TID 427) (an-worker1119.eqiad.wmnet, executor 12, partition 135, NODE_LOCAL, 28188 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 33.0 in stage 2.0 (TID 366) in 1329 ms on an-worker1119.eqiad.wmnet (executor 12) (48/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 136.0 in stage 2.0 (TID 428) (an-worker1119.eqiad.wmnet, executor 14, partition 136, NODE_LOCAL, 27551 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 132.0 in stage 2.0 (TID 397) in 484 ms on an-worker1119.eqiad.wmnet (executor 14) (49/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 142.0 in stage 2.0 (TID 429) (an-worker1119.eqiad.wmnet, executor 12, partition 142, NODE_LOCAL, 25531 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 4.0 in stage 2.0 (TID 350) in 1360 ms on an-worker1119.eqiad.wmnet (executor 12) (50/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 80.0 in stage 2.0 (TID 430) (an-worker1102.eqiad.wmnet, executor 5, partition 80, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 39.0 in stage 2.0 (TID 392) in 559 ms on an-worker1102.eqiad.wmnet (executor 5) (51/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 120.0 in stage 2.0 (TID 431) (an-worker1116.eqiad.wmnet, executor 9, partition 120, NODE_LOCAL, 18579 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 93.0 in stage 2.0 (TID 415) in 210 ms on an-worker1116.eqiad.wmnet (executor 9) (52/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 128.0 in stage 2.0 (TID 432) (an-worker1116.eqiad.wmnet, executor 10, partition 128, NODE_LOCAL, 21904 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 19.0 in stage 2.0 (TID 367) in 1390 ms on an-worker1116.eqiad.wmnet (executor 10) (53/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 60.0 in stage 2.0 (TID 433) (an-worker1079.eqiad.wmnet, executor 1, partition 60, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 36.0 in stage 2.0 (TID 408) in 289 ms on an-worker1079.eqiad.wmnet (executor 1) (54/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 130.0 in stage 2.0 (TID 434) (an-worker1116.eqiad.wmnet, executor 9, partition 130, NODE_LOCAL, 29176 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 111.0 in stage 2.0 (TID 426) in 70 ms on an-worker1116.eqiad.wmnet (executor 9) (55/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 96.0 in stage 2.0 (TID 435) (an-worker1102.eqiad.wmnet, executor 5, partition 96, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 75.0 in stage 2.0 (TID 420) in 151 ms on an-worker1102.eqiad.wmnet (executor 5) (56/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 138.0 in stage 2.0 (TID 436) (an-worker1116.eqiad.wmnet, executor 10, partition 138, NODE_LOCAL, 26413 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 73.0 in stage 2.0 (TID 409) in 311 ms on an-worker1116.eqiad.wmnet (executor 10) (57/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 127.0 in stage 2.0 (TID 437) (an-worker1083.eqiad.wmnet, executor 3, partition 127, NODE_LOCAL, 17837 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 79.0 in stage 2.0 (TID 418) in 199 ms on an-worker1083.eqiad.wmnet (executor 3) (58/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 129.0 in stage 2.0 (TID 438) (an-worker1083.eqiad.wmnet, executor 3, partition 129, NODE_LOCAL, 20449 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 83.0 in stage 2.0 (TID 421) in 174 ms on an-worker1083.eqiad.wmnet (executor 3) (59/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 144.0 in stage 2.0 (TID 439) (an-worker1119.eqiad.wmnet, executor 11, partition 144, NODE_LOCAL, 29928 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 14.0 in stage 2.0 (TID 359) in 1449 ms on an-worker1119.eqiad.wmnet (executor 11) (60/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 50.0 in stage 2.0 (TID 440) (an-worker1128.eqiad.wmnet, executor 15, partition 50, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 15.0 in stage 2.0 (TID 365) in 1457 ms on an-worker1128.eqiad.wmnet (executor 15) (61/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 64.0 in stage 2.0 (TID 441) (an-worker1128.eqiad.wmnet, executor 15, partition 64, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 48.0 in stage 2.0 (TID 419) in 247 ms on an-worker1128.eqiad.wmnet (executor 15) (62/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 54.0 in stage 2.0 (TID 442) (an-worker1153.eqiad.wmnet, executor 8, partition 54, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 47.0 in stage 2.0 (TID 425) in 178 ms on an-worker1153.eqiad.wmnet (executor 8) (63/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 147.0 in stage 2.0 (TID 443) (an-worker1083.eqiad.wmnet, executor 2, partition 147, NODE_LOCAL, 29347 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 5.0 in stage 2.0 (TID 356) in 1489 ms on an-worker1083.eqiad.wmnet (executor 2) (64/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 108.0 in stage 2.0 (TID 444) (an-worker1102.eqiad.wmnet, executor 5, partition 108, NODE_LOCAL, 16738 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 80.0 in stage 2.0 (TID 430) in 134 ms on an-worker1102.eqiad.wmnet (executor 5) (65/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 62.0 in stage 2.0 (TID 445) (an-worker1153.eqiad.wmnet, executor 8, partition 62, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 46.0 in stage 2.0 (TID 414) in 365 ms on an-worker1153.eqiad.wmnet (executor 8) (66/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 137.0 in stage 2.0 (TID 446) (an-worker1143.eqiad.wmnet, executor 16, partition 137, NODE_LOCAL, 29408 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 119.0 in stage 2.0 (TID 417) in 356 ms on an-worker1143.eqiad.wmnet (executor 16) (67/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 74.0 in stage 2.0 (TID 447) (an-worker1079.eqiad.wmnet, executor 1, partition 74, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 60.0 in stage 2.0 (TID 433) in 171 ms on an-worker1079.eqiad.wmnet (executor 1) (68/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 86.0 in stage 2.0 (TID 424) in 284 ms on an-worker1083.eqiad.wmnet (executor 2) (69/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 140.0 in stage 2.0 (TID 448) (an-worker1102.eqiad.wmnet, executor 5, partition 140, NODE_LOCAL, 21192 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 108.0 in stage 2.0 (TID 444) in 119 ms on an-worker1102.eqiad.wmnet (executor 5) (70/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 129.0 in stage 2.0 (TID 438) in 189 ms on an-worker1083.eqiad.wmnet (executor 3) (71/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 78.0 in stage 2.0 (TID 449) (an-worker1085.eqiad.wmnet, executor 4, partition 78, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 8.0 in stage 2.0 (TID 360) in 1636 ms on an-worker1085.eqiad.wmnet (executor 4) (72/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 131.0 in stage 2.0 (TID 396) in 787 ms on an-worker1119.eqiad.wmnet (executor 13) (73/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 127.0 in stage 2.0 (TID 437) in 223 ms on an-worker1083.eqiad.wmnet (executor 3) (74/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 96.0 in stage 2.0 (TID 435) in 243 ms on an-worker1102.eqiad.wmnet (executor 5) (75/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 143.0 in stage 2.0 (TID 450) (an-worker1116.eqiad.wmnet, executor 9, partition 143, NODE_LOCAL, 19320 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 120.0 in stage 2.0 (TID 431) in 292 ms on an-worker1116.eqiad.wmnet (executor 9) (76/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 81.0 in stage 2.0 (TID 451) (an-worker1128.eqiad.wmnet, executor 15, partition 81, NODE_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 50.0 in stage 2.0 (TID 440) in 201 ms on an-worker1128.eqiad.wmnet (executor 15) (77/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 145.0 in stage 2.0 (TID 452) (an-worker1095.eqiad.wmnet, executor 7, partition 145, NODE_LOCAL, 24073 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 31.0 in stage 2.0 (TID 373) in 1668 ms on an-worker1095.eqiad.wmnet (executor 7) (78/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 133.0 in stage 2.0 (TID 401) in 755 ms on an-worker1119.eqiad.wmnet (executor 14) (79/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 66.0 in stage 2.0 (TID 453) (an-worker1153.eqiad.wmnet, executor 8, partition 66, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 62.0 in stage 2.0 (TID 445) in 186 ms on an-worker1153.eqiad.wmnet (executor 8) (80/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 16.0 in stage 2.0 (TID 361) in 1688 ms on an-worker1143.eqiad.wmnet (executor 16) (81/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 91.0 in stage 2.0 (TID 454) (an-worker1128.eqiad.wmnet, executor 15, partition 91, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 64.0 in stage 2.0 (TID 441) in 224 ms on an-worker1128.eqiad.wmnet (executor 15) (82/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 71.0 in stage 2.0 (TID 455) (an-worker1153.eqiad.wmnet, executor 8, partition 71, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 54.0 in stage 2.0 (TID 442) in 229 ms on an-worker1153.eqiad.wmnet (executor 8) (83/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 146.0 in stage 2.0 (TID 456) (an-worker1128.eqiad.wmnet, executor 15, partition 146, NODE_LOCAL, 19649 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 91.0 in stage 2.0 (TID 454) in 78 ms on an-worker1128.eqiad.wmnet (executor 15) (84/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 140.0 in stage 2.0 (TID 448) in 182 ms on an-worker1102.eqiad.wmnet (executor 5) (85/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 121.0 in stage 2.0 (TID 412) in 671 ms on an-worker1095.eqiad.wmnet (executor 6) (86/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 128.0 in stage 2.0 (TID 432) in 411 ms on an-worker1116.eqiad.wmnet (executor 10) (87/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 134.0 in stage 2.0 (TID 423) in 527 ms on an-worker1119.eqiad.wmnet (executor 13) (88/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 84.0 in stage 2.0 (TID 457) (an-worker1085.eqiad.wmnet, executor 4, partition 84, NODE_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 78.0 in stage 2.0 (TID 449) in 203 ms on an-worker1085.eqiad.wmnet (executor 4) (89/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 143.0 in stage 2.0 (TID 450) in 210 ms on an-worker1116.eqiad.wmnet (executor 9) (90/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 74.0 in stage 2.0 (TID 447) in 326 ms on an-worker1079.eqiad.wmnet (executor 1) (91/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 9.0 in stage 2.0 (TID 348) in 1900 ms on an-worker1079.eqiad.wmnet (executor 1) (92/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 92.0 in stage 2.0 (TID 458) (an-worker1153.eqiad.wmnet, executor 8, partition 92, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 66.0 in stage 2.0 (TID 453) in 257 ms on an-worker1153.eqiad.wmnet (executor 8) (93/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 138.0 in stage 2.0 (TID 436) in 530 ms on an-worker1116.eqiad.wmnet (executor 10) (94/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 30.0 in stage 2.0 (TID 376) in 1976 ms on an-worker1085.eqiad.wmnet (executor 4) (95/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 94.0 in stage 2.0 (TID 459) (an-worker1153.eqiad.wmnet, executor 8, partition 94, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 71.0 in stage 2.0 (TID 455) in 283 ms on an-worker1153.eqiad.wmnet (executor 8) (96/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 142.0 in stage 2.0 (TID 429) in 676 ms on an-worker1119.eqiad.wmnet (executor 12) (97/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 81.0 in stage 2.0 (TID 451) in 373 ms on an-worker1128.eqiad.wmnet (executor 15) (98/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 135.0 in stage 2.0 (TID 427) in 711 ms on an-worker1119.eqiad.wmnet (executor 12) (99/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 125.0 in stage 2.0 (TID 390) in 1267 ms on an-worker1119.eqiad.wmnet (executor 11) (100/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 114.0 in stage 2.0 (TID 460) (an-worker1153.eqiad.wmnet, executor 8, partition 114, NODE_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 92.0 in stage 2.0 (TID 458) in 151 ms on an-worker1153.eqiad.wmnet (executor 8) (101/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 145.0 in stage 2.0 (TID 452) in 445 ms on an-worker1095.eqiad.wmnet (executor 7) (102/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 146.0 in stage 2.0 (TID 456) in 342 ms on an-worker1128.eqiad.wmnet (executor 15) (103/149)
24/01/12 13:41:26 INFO TaskSetManager: Finished task 84.0 in stage 2.0 (TID 457) in 289 ms on an-worker1085.eqiad.wmnet (executor 4) (104/149)
24/01/12 13:41:26 INFO TaskSetManager: Starting task 116.0 in stage 2.0 (TID 461) (an-worker1153.eqiad.wmnet, executor 8, partition 116, NODE_LOCAL, 17820 bytes) taskResourceAssignments Map()
24/01/12 13:41:26 INFO TaskSetManager: Finished task 114.0 in stage 2.0 (TID 460) in 80 ms on an-worker1153.eqiad.wmnet (executor 8) (105/149)
24/01/12 13:41:27 INFO TaskSetManager: Starting task 117.0 in stage 2.0 (TID 462) (an-worker1153.eqiad.wmnet, executor 8, partition 117, NODE_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:27 INFO TaskSetManager: Finished task 94.0 in stage 2.0 (TID 459) in 243 ms on an-worker1153.eqiad.wmnet (executor 8) (106/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 126.0 in stage 2.0 (TID 416) in 1086 ms on an-worker1095.eqiad.wmnet (executor 6) (107/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 130.0 in stage 2.0 (TID 434) in 860 ms on an-worker1116.eqiad.wmnet (executor 9) (108/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 124.0 in stage 2.0 (TID 413) in 1129 ms on an-worker1095.eqiad.wmnet (executor 7) (109/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 136.0 in stage 2.0 (TID 428) in 1000 ms on an-worker1119.eqiad.wmnet (executor 14) (110/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 116.0 in stage 2.0 (TID 461) in 219 ms on an-worker1153.eqiad.wmnet (executor 8) (111/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 144.0 in stage 2.0 (TID 439) in 959 ms on an-worker1119.eqiad.wmnet (executor 11) (112/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 117.0 in stage 2.0 (TID 462) in 203 ms on an-worker1153.eqiad.wmnet (executor 8) (113/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 147.0 in stage 2.0 (TID 443) in 989 ms on an-worker1083.eqiad.wmnet (executor 2) (114/149)
24/01/12 13:41:27 INFO TaskSetManager: Finished task 137.0 in stage 2.0 (TID 446) in 1389 ms on an-worker1143.eqiad.wmnet (executor 16) (115/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 105.0 in stage 2.0 (TID 463) (an-worker1128.eqiad.wmnet, executor 15, partition 105, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 115.0 in stage 2.0 (TID 464) (an-worker1085.eqiad.wmnet, executor 4, partition 115, RACK_LOCAL, 16738 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 28.0 in stage 2.0 (TID 465) (an-worker1119.eqiad.wmnet, executor 14, partition 28, RACK_LOCAL, 18554 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 25.0 in stage 2.0 (TID 466) (an-worker1153.eqiad.wmnet, executor 8, partition 25, RACK_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 38.0 in stage 2.0 (TID 467) (an-worker1079.eqiad.wmnet, executor 1, partition 38, RACK_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 98.0 in stage 2.0 (TID 468) (an-worker1116.eqiad.wmnet, executor 9, partition 98, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 141.0 in stage 2.0 (TID 469) (an-worker1083.eqiad.wmnet, executor 2, partition 141, RACK_LOCAL, 20777 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 101.0 in stage 2.0 (TID 470) (an-worker1116.eqiad.wmnet, executor 10, partition 101, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 26.0 in stage 2.0 (TID 471) (an-worker1095.eqiad.wmnet, executor 7, partition 26, RACK_LOCAL, 18170 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 148.0 in stage 2.0 (TID 472) (an-worker1119.eqiad.wmnet, executor 11, partition 148, RACK_LOCAL, 17094 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 59.0 in stage 2.0 (TID 473) (an-worker1102.eqiad.wmnet, executor 5, partition 59, RACK_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 40.0 in stage 2.0 (TID 474) (an-worker1119.eqiad.wmnet, executor 12, partition 40, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 99.0 in stage 2.0 (TID 475) (an-worker1095.eqiad.wmnet, executor 6, partition 99, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 56.0 in stage 2.0 (TID 476) (an-worker1128.eqiad.wmnet, executor 15, partition 56, RACK_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 63.0 in stage 2.0 (TID 477) (an-worker1085.eqiad.wmnet, executor 4, partition 63, RACK_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 103.0 in stage 2.0 (TID 478) (an-worker1116.eqiad.wmnet, executor 9, partition 103, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 104.0 in stage 2.0 (TID 479) (an-worker1116.eqiad.wmnet, executor 10, partition 104, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 82.0 in stage 2.0 (TID 480) (an-worker1095.eqiad.wmnet, executor 7, partition 82, RACK_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 106.0 in stage 2.0 (TID 481) (an-worker1102.eqiad.wmnet, executor 5, partition 106, RACK_LOCAL, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Starting task 102.0 in stage 2.0 (TID 482) (an-worker1095.eqiad.wmnet, executor 6, partition 102, RACK_LOCAL, 17820 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 148.0 in stage 2.0 (TID 472) in 69 ms on an-worker1119.eqiad.wmnet (executor 11) (116/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 35.0 in stage 2.0 (TID 483) (an-worker1095.eqiad.wmnet, executor 6, partition 35, RACK_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 99.0 in stage 2.0 (TID 475) in 107 ms on an-worker1095.eqiad.wmnet (executor 6) (117/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 69.0 in stage 2.0 (TID 484) (an-worker1085.eqiad.wmnet, executor 4, partition 69, RACK_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 115.0 in stage 2.0 (TID 464) in 140 ms on an-worker1085.eqiad.wmnet (executor 4) (118/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 104.0 in stage 2.0 (TID 479) in 129 ms on an-worker1116.eqiad.wmnet (executor 10) (119/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 112.0 in stage 2.0 (TID 485) (an-worker1102.eqiad.wmnet, executor 5, partition 112, RACK_LOCAL, 16746 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 106.0 in stage 2.0 (TID 481) in 131 ms on an-worker1102.eqiad.wmnet (executor 5) (120/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 103.0 in stage 2.0 (TID 478) in 135 ms on an-worker1116.eqiad.wmnet (executor 9) (121/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 76.0 in stage 2.0 (TID 486) (an-worker1128.eqiad.wmnet, executor 15, partition 76, RACK_LOCAL, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 105.0 in stage 2.0 (TID 463) in 150 ms on an-worker1128.eqiad.wmnet (executor 15) (122/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 101.0 in stage 2.0 (TID 470) in 148 ms on an-worker1116.eqiad.wmnet (executor 10) (123/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 95.0 in stage 2.0 (TID 487) (an-worker1128.eqiad.wmnet, executor 15, partition 95, RACK_LOCAL, 17454 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 56.0 in stage 2.0 (TID 476) in 168 ms on an-worker1128.eqiad.wmnet (executor 15) (124/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 40.0 in stage 2.0 (TID 474) in 171 ms on an-worker1119.eqiad.wmnet (executor 12) (125/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 82.0 in stage 2.0 (TID 480) in 200 ms on an-worker1095.eqiad.wmnet (executor 7) (126/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 28.0 in stage 2.0 (TID 465) in 216 ms on an-worker1119.eqiad.wmnet (executor 14) (127/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 113.0 in stage 2.0 (TID 488) (an-worker1102.eqiad.wmnet, executor 5, partition 113, RACK_LOCAL, 16738 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 59.0 in stage 2.0 (TID 473) in 219 ms on an-worker1102.eqiad.wmnet (executor 5) (128/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 102.0 in stage 2.0 (TID 482) in 220 ms on an-worker1095.eqiad.wmnet (executor 6) (129/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 63.0 in stage 2.0 (TID 477) in 236 ms on an-worker1085.eqiad.wmnet (executor 4) (130/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 98.0 in stage 2.0 (TID 468) in 249 ms on an-worker1116.eqiad.wmnet (executor 9) (131/149)
24/01/12 13:41:30 INFO TaskSetManager: Starting task 18.0 in stage 2.0 (TID 489) (an-worker1102.eqiad.wmnet, executor 5, partition 18, RACK_LOCAL, 17094 bytes) taskResourceAssignments Map()
24/01/12 13:41:30 INFO TaskSetManager: Finished task 112.0 in stage 2.0 (TID 485) in 114 ms on an-worker1102.eqiad.wmnet (executor 5) (132/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 69.0 in stage 2.0 (TID 484) in 144 ms on an-worker1085.eqiad.wmnet (executor 4) (133/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 25.0 in stage 2.0 (TID 466) in 315 ms on an-worker1153.eqiad.wmnet (executor 8) (134/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 38.0 in stage 2.0 (TID 467) in 314 ms on an-worker1079.eqiad.wmnet (executor 1) (135/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 113.0 in stage 2.0 (TID 488) in 111 ms on an-worker1102.eqiad.wmnet (executor 5) (136/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 26.0 in stage 2.0 (TID 471) in 338 ms on an-worker1095.eqiad.wmnet (executor 7) (137/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 18.0 in stage 2.0 (TID 489) in 90 ms on an-worker1102.eqiad.wmnet (executor 5) (138/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 35.0 in stage 2.0 (TID 483) in 249 ms on an-worker1095.eqiad.wmnet (executor 6) (139/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 141.0 in stage 2.0 (TID 469) in 411 ms on an-worker1083.eqiad.wmnet (executor 2) (140/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 95.0 in stage 2.0 (TID 487) in 241 ms on an-worker1128.eqiad.wmnet (executor 15) (141/149)
24/01/12 13:41:30 INFO TaskSetManager: Finished task 76.0 in stage 2.0 (TID 486) in 334 ms on an-worker1128.eqiad.wmnet (executor 15) (142/149)
24/01/12 13:41:33 INFO TaskSetManager: Starting task 51.0 in stage 2.0 (TID 490) (an-worker1083.eqiad.wmnet, executor 3, partition 51, ANY, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 57.0 in stage 2.0 (TID 491) (an-worker1116.eqiad.wmnet, executor 10, partition 57, ANY, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 58.0 in stage 2.0 (TID 492) (an-worker1085.eqiad.wmnet, executor 4, partition 58, ANY, 17812 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 97.0 in stage 2.0 (TID 493) (an-worker1095.eqiad.wmnet, executor 7, partition 97, ANY, 17096 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 100.0 in stage 2.0 (TID 494) (an-worker1102.eqiad.wmnet, executor 5, partition 100, ANY, 16746 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 107.0 in stage 2.0 (TID 495) (an-worker1083.eqiad.wmnet, executor 2, partition 107, ANY, 16738 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Starting task 139.0 in stage 2.0 (TID 496) (an-worker1143.eqiad.wmnet, executor 16, partition 139, ANY, 17480 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO TaskSetManager: Finished task 100.0 in stage 2.0 (TID 494) in 84 ms on an-worker1102.eqiad.wmnet (executor 5) (143/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 107.0 in stage 2.0 (TID 495) in 91 ms on an-worker1083.eqiad.wmnet (executor 2) (144/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 57.0 in stage 2.0 (TID 491) in 168 ms on an-worker1116.eqiad.wmnet (executor 10) (145/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 97.0 in stage 2.0 (TID 493) in 167 ms on an-worker1095.eqiad.wmnet (executor 7) (146/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 139.0 in stage 2.0 (TID 496) in 289 ms on an-worker1143.eqiad.wmnet (executor 16) (147/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 58.0 in stage 2.0 (TID 492) in 413 ms on an-worker1085.eqiad.wmnet (executor 4) (148/149)
24/01/12 13:41:33 INFO TaskSetManager: Finished task 51.0 in stage 2.0 (TID 490) in 442 ms on an-worker1083.eqiad.wmnet (executor 3) (149/149)
24/01/12 13:41:33 INFO YarnScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool
24/01/12 13:41:33 INFO DAGScheduler: ShuffleMapStage 2 (count at PagesPartitionsDefiner.scala:101) finished in 8.908 s
24/01/12 13:41:33 INFO DAGScheduler: looking for newly runnable stages
24/01/12 13:41:33 INFO DAGScheduler: running: Set()
24/01/12 13:41:33 INFO DAGScheduler: waiting: Set(ResultStage 3)
24/01/12 13:41:33 INFO DAGScheduler: failed: Set()
24/01/12 13:41:33 INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[21] at count at PagesPartitionsDefiner.scala:101), which has no missing parents
24/01/12 13:41:33 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 10.1 KiB, free 8.4 GiB)
24/01/12 13:41:33 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 8.4 GiB)
24/01/12 13:41:33 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on stat1007.eqiad.wmnet:13006 (size: 5.0 KiB, free: 8.4 GiB)
24/01/12 13:41:33 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:1388
24/01/12 13:41:33 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 3 (MapPartitionsRDD[21] at count at PagesPartitionsDefiner.scala:101) (first 15 tasks are for partitions Vector(0))
24/01/12 13:41:33 INFO YarnScheduler: Adding task set 3.0 with 1 tasks resource profile 0
24/01/12 13:41:33 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 497) (an-worker1119.eqiad.wmnet, executor 11, partition 0, PROCESS_LOCAL, 4465 bytes) taskResourceAssignments Map()
24/01/12 13:41:33 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on an-worker1119.eqiad.wmnet:38383 (size: 5.0 KiB, free: 2004.5 MiB)
24/01/12 13:41:33 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 2 to 10.64.5.8:53552
24/01/12 13:41:33 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 497) in 74 ms on an-worker1119.eqiad.wmnet (executor 11) (1/1)
24/01/12 13:41:33 INFO YarnScheduler: Removed TaskSet 3.0, whose tasks have all completed, from pool
24/01/12 13:41:33 INFO DAGScheduler: ResultStage 3 (count at PagesPartitionsDefiner.scala:101) finished in 0.093 s
24/01/12 13:41:33 INFO DAGScheduler: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job
24/01/12 13:41:33 INFO YarnScheduler: Killing all running tasks in stage 3: Stage finished
24/01/12 13:41:33 INFO DAGScheduler: Job 1 finished: count at PagesPartitionsDefiner.scala:101, took 9.019340 s
24/01/12 13:41:33 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
24/01/12 13:41:33 INFO AbstractConnector: Stopped Spark@4b4d124{HTTP/1.1, (http/1.1)}{0.0.0.0:4046}
24/01/12 13:41:33 INFO SparkUI: Stopped Spark web UI at http://stat1007.eqiad.wmnet:4046
24/01/12 13:41:33 INFO YarnClientSchedulerBackend: Interrupting monitor thread
24/01/12 13:41:33 INFO YarnClientSchedulerBackend: Shutting down all executors
24/01/12 13:41:33 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
24/01/12 13:41:33 INFO YarnClientSchedulerBackend: YARN client scheduler backend Stopped
24/01/12 13:41:34 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
24/01/12 13:41:34 INFO MemoryStore: MemoryStore cleared
24/01/12 13:41:34 INFO BlockManager: BlockManager stopped
24/01/12 13:41:34 INFO BlockManagerMaster: BlockManagerMaster stopped
24/01/12 13:41:34 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
24/01/12 13:41:34 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: Task not serializable
at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:416)
at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:406)
at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:162)
at org.apache.spark.SparkContext.clean(SparkContext.scala:2459)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$1(RDD.scala:860)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
at org.apache.spark.rdd.RDD.mapPartitions(RDD.scala:859)
at org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner.definePartitions(PagesPartitionsDefiner.scala:52)
at org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner.<init>(PagesPartitionsDefiner.scala:43)
at org.wikimedia.analytics.refinery.job.mediawikidumper.MediawikiDumper$.apply(MediawikiDumper.scala:54)
at org.wikimedia.analytics.refinery.job.mediawikidumper.MediawikiDumper$.main(MediawikiDumper.scala:357)
at org.wikimedia.analytics.refinery.job.mediawikidumper.MediawikiDumper.main(MediawikiDumper.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.io.NotSerializableException: org.apache.log4j.Logger
Serialization stack:
- object not serializable (class: org.apache.log4j.Logger, value: org.apache.log4j.Logger@799cb334)
- field (class: scala.Some, name: value, type: class java.lang.Object)
- object (class scala.Some, Some(org.apache.log4j.Logger@799cb334))
- field (class: org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner, name: log, type: class scala.Option)
- object (class org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner, org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner@61b2ca5a)
- element of array (index: 0)
- array (class [Ljava.lang.Object;, size 1)
- field (class: java.lang.invoke.SerializedLambda, name: capturedArgs, type: class [Ljava.lang.Object;)
- object (class java.lang.invoke.SerializedLambda, SerializedLambda[capturingClass=class org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner, functionalInterfaceMethod=scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;, implementation=invokeStatic org/wikimedia/analytics/refinery/job/mediawikidumper/PagesPartitionsDefiner.$anonfun$definePartitions$1:(Lorg/wikimedia/analytics/refinery/job/mediawikidumper/PagesPartitionsDefiner;Lscala/collection/Iterator;)Lscala/collection/Iterator;, instantiatedMethodType=(Lscala/collection/Iterator;)Lscala/collection/Iterator;, numCaptured=1])
- writeReplace data (class: java.lang.invoke.SerializedLambda)
- object (class org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner$$Lambda$3620/99015930, org.wikimedia.analytics.refinery.job.mediawikidumper.PagesPartitionsDefiner$$Lambda$3620/99015930@6d7e973b)
at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47)
at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:101)
at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:413)
... 25 more
24/01/12 13:41:34 INFO ShutdownHookManager: Shutdown hook called
24/01/12 13:41:34 INFO ShutdownHookManager: Deleting directory /tmp/spark-ddcc0a01-c64e-4ff4-aef3-bf3006dd5350
24/01/12 13:41:34 INFO ShutdownHookManager: Deleting directory /srv/spark-tmp/spark-3ed69458-8cf6-496f-b5dd-8cd5c38e33ab
jebe@stat1007:~$ hdf dfs -ls /users/jebe/data/archive/content_dump_test/
-bash: hdf: command not found
jebe@stat1007:~$ hdfs dfs -ls /users/jebe/data/archive/content_dump_test/
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
ls: `/users/jebe/data/archive/content_dump_test/': No such file or directory
jebe@stat1007:~$
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits