Page MenuHomePhabricator
Paste P43556

refinery deploy to HDFS step feb1 run 3
ActivePublic
Actions

Authored by Stevemunene on Feb 1 2023, 5:29 PM.
Tags
None
Referenced Files
F36577198: refinery deploy to HDFS step feb1 run 3
Feb 1 2023, 5:29 PM
Subscribers
None
stevemunene@an-launcher1002:/srv/deployment/analytics/refinery$ sudo -u hdfs kerberos-run-command hdfs /srv/deployment/analytics/refinery/bin/refinery-deploy-to-hdfs --verbose --no-dry-run
2023-02-01T17:20:43+00:00 Current git description: '2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001'
2023-02-01T17:20:43+00:00
2023-02-01T17:20:43+00:00 * Preparing HDFS ...
2023-02-01T17:20:43+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mkdir -p hdfs:///wmf/refinery
2023-02-01T17:20:47+00:00
2023-02-01T17:20:47+00:00 * Copying local checkout to versioned directory in HDFS ...
2023-02-01T17:20:47+00:00 hdfs dfs -rm -r -f -skipTrash hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001.tmp
2023-02-01T17:20:50+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mkdir hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001.tmp
2023-02-01T17:20:55+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -put -f airflow artifacts bin diagrams druid gobblin HACKING.md hive hql oozie packaged-environments python README.md setup.cfg spark static_data hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001.tmp
23/02/01 17:22:31 WARN hdfs.DataStreamer: Caught exception
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1257)
at java.lang.Thread.join(Thread.java:1331)
at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980)
at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807)
2023-02-01T17:22:45+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -put /dev/fd/63 hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001.tmp/.deployment
2023-02-01T17:22:53+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001.tmp hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001
2023-02-01T17:22:57+00:00
2023-02-01T17:22:57+00:00 * Setting up 'current' version on cluster ...
2023-02-01T17:22:57+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -cp hdfs:///wmf/refinery/2023-02-01T17.20.43+00.00--scap_sync_2023-02-01_0001 hdfs:///wmf/refinery/current.tmp
2023-02-01T17:26:13+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/current hdfs:///wmf/refinery/current.swap
2023-02-01T17:26:17+00:00 hdfs dfs -D fs.permissions.umask-mode=022 -mv hdfs:///wmf/refinery/current.tmp hdfs:///wmf/refinery/current
2023-02-01T17:26:23+00:00 hdfs dfs -rm -r -f -skipTrash hdfs:///wmf/refinery/current.swap
Deleted hdfs:///wmf/refinery/current.swap
2023-02-01T17:26:29+00:00 pass (Used parameters: --verbose --no-dry-run )
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL