I ran into a failure since last week on stat1005. One of my Jupiter notebook ran by cronjob started to fail at one hive query with below error. I checked file "/tmp/parquet-0.log”. It belongs to another user. I don’t have permission to write. It also happened in interactive mode.
Please suggest any method to fix or work around this issue. Thanks!
[0;31mChildProcessError[0m: The Hive command line client encountered the following error:
Can't load log handler "java.util.logging.FileHandler"
java.io.FileNotFoundException: /tmp/parquet-0.log (Permission denied)
java.io.FileNotFoundException: /tmp/parquet-0.log (Permission denied)
commands where the failure happened:
query_edits_quality_damaging_model_nonbot_registered_user='''
SELECT WEEKOFYEAR(rev_timestamp) AS week_n,
ROUND(SUM(case when scores["damaging"].prediction[0]='true' then 1 else 0 end)/count(rev_id) ,4) AS damaging_rate
FROM event_sanitized.mediawiki_revision_score
WHERE year='{YEAR}' AND (rev_timestamp >= '{START_YYYY_MM_DD}' AND rev_timestamp <= '{END_YYYY_MM_DD}')
AND `database`='fawiki'
-- exclude IP editors in baseline as ORES is biased against anonymous editors
AND performer.user_id IS NOT NULL
-- exclude bot editors
AND not performer.user_is_bot
AND performer.user_text not regexp "^.*bot([^a-z].*$|$)"
GROUP BY WEEKOFYEAR(rev_timestamp)
ORDER BY week_n
LIMIT 10000
‘''
df_edits_quality_damaging_rate_nonbot_registered_user=hive.run(query_edits_quality_damaging_model_nonbot_registered_user.format(YEAR=yr, START_YYYY_MM_DD=start_date_YYYY_MM_DD , END_YYYY_MM_DD=end_date_YYYY_MM_DD))