Page MenuHomePhabricator
Paste P43527

refinery deploy to HDFS step feb1 run 2
ActivePublic

Authored by Stevemunene on Feb 1 2023, 11:42 AM.
Tags
None
Referenced Files
F36571956: refinery deploy to HDFS step feb1 run 2
Feb 1 2023, 11:43 AM
F36571951: refinery deploy to HDFS step feb1 run 2
Feb 1 2023, 11:42 AM
Subscribers
None
stevemunene@an-launcher1002:/srv/deployment/analytics/refinery$ sudo -u hdfs kerberos-run-command hdfs /srv/deployment/analytics/refinery/bin/refinery-deploy-to-hdfs --verbose --no-dry-run
2023-02-01T11:36:36+00:00 Current git description: '2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001'
2023-02-01T11:36:36+00:00
2023-02-01T11:36:36+00:00 * Preparing HDFS ...
2023-02-01T11:36:36+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mkdir -p hdfs:///wmf/refinery
2023-02-01T11:36:41+00:00
2023-02-01T11:36:41+00:00 * Copying local checkout to versioned directory in HDFS ...
2023-02-01T11:36:41+00:00 hdfs dfs -rm -r -f -skipTrash hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001.tmp
2023-02-01T11:36:47+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mkdir hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001.tmp
2023-02-01T11:36:52+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -put -f airflow artifacts bin diagrams druid gobblin HACKING.md hive hql oozie packaged-environments python README.md setup.cfg spark static_data hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001.tmp
23/02/01 11:38:44 WARN hdfs.DataStreamer: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1257)
at java.lang.Thread.join(Thread.java:1331)
at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980)
at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807)
2023-02-01T11:38:47+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -put /dev/fd/63 hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001.tmp/.deployment
2023-02-01T11:38:57+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001.tmp hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001
2023-02-01T11:39:03+00:00
2023-02-01T11:39:03+00:00 * Setting up 'current' version on cluster ...
2023-02-01T11:39:03+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -cp hdfs:///wmf/refinery/2023-02-01T11.36.36+00.00--scap_sync_2023-02-01_0001 hdfs:///wmf/refinery/current.tmp
23/02/01 11:40:04 WARN hdfs.DataStreamer: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1257)
at java.lang.Thread.join(Thread.java:1331)
at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980)
at org.apache.hadoop.hdfs.DataStreamer.closeInternal(DataStreamer.java:844)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:840)
23/02/01 11:40:45 WARN hdfs.DataStreamer: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1257)
at java.lang.Thread.join(Thread.java:1331)
at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980)
at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807)
23/02/01 11:40:54 WARN hdfs.DataStreamer: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1257)
at java.lang.Thread.join(Thread.java:1331)
at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980)
at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807)
2023-02-01T11:41:09+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/current hdfs:///wmf/refinery/current.swap
2023-02-01T11:41:15+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/current.tmp hdfs:///wmf/refinery/current
2023-02-01T11:41:20+00:00 hdfs dfs -rm -r -f -skipTrash hdfs:///wmf/refinery/current.swap
Deleted hdfs:///wmf/refinery/current.swap
2023-02-01T11:41:25+00:00 pass (Used parameters: --verbose --no-dry-run )
stevemunene@an-launcher1002:/srv/deployment/analytics/refinery$

Event Timeline