@ssingh hi! Sorry for the delay. I am fine adding the group permissions but if the directory was marked in that way it was on purpose, maybe due to data protection? As far as I now @Jdcc-berkman is still active (namely holds a valid account / ssh-key), so he could probably chime in and give us some hint.

I could also move all that data to your home directory and chmod it properly, then we could fix the cron and everything should work as expected.

My memory of setting these perms is long gone, but if I did mark it this way, it was probably because there is private data from the stats servers sitting in there. It's WMF's data, so as far as I'm concerned, do whatever you need to with the perms.

@ssingh let's do things properly and move crons and repositories in one place. The dependencies that were discussed in the related task are basically two crons IIRC, so we can move them under your username first, and then possibly add them to puppet?

So to recap:

• move /home/jdcc/project_monitoring and /home/jdcc/heka under your home directory, changing the owner (jddc -> sukhe) of all files/dir to you (so you'll have perms)
• move jdcc's crons under your crontab

0 15 * * * USER=jdcc /home/jdcc/anaconda3/bin/python /home/jdcc/project_monitoring/scripts/check_projects.py > /home/jdcc/project_monitoring/reports/`date -Idate`.html 2>> /home/jdcc/errors.log
0 10 * * * USER=jdcc /home/jdcc/anaconda3/bin/python /home/jdcc/article_monitoring/scripts/check_articles.py > /home/jdcc/article_monitoring/reports/`date -Idate`.html 2>> /home/jdcc/errors.log

• move slaporte's crontabs under your crontab:

46 23 * * * sh /home/slaporte/send_report.s

Does it sound ok?

@elukey Once this is puppetized, how much would it continue to depend on particular users retaining continued access?

@elukey: Sorry for the late reply.

Yes, that's fine with me as I do plan on maintaining this and having control over all the scripts will make it easier.

@ssingh late reply as well, apologies :)

I did the following:

• copied over anaconda3, project_monitoring and heka from /home/jdcc to yours + chmod sukhe:wikidev to all
• copied over send_report.sh from slaporte's home + chmod sukhe:wikidev

I also added to your crontab all the crons mentioned in my previous post, amended accordingly with your username. Moreover I have modified your version of the send_report to only mail things to you for the moment, with a different sender/mail-subject so you can compare it with the one sent by slaporte's user. Basically I left everything as it was for jdcc/slaport, and "cloned" a version of everything in your home dir. Once you'll give me the green light I'll clean up everything!

@ssingh ping :)

Hi @elukey, thanks for the ping.

The reports were not being generated so I have made some more changes today (updating paths); I will update the ticket with the progress but for now, let's keep the jdcc directory.

Thanks for your help.

Hi @elukey. Thanks for your patience on this issue. I can confirm everything is working from my ~.

@ssingh very happy to help! We should be ok to close the task right?

I can see that @Jdcc-berkman is still an active user - shall I keep the content of the /home/jdcc home directory or is it going to be deleted eventually? (trying to avoid leaving data around if not needed :)

If this is successfully running elsewhere now, I don't need the data in my home directory anymore nor my account. I've backed up the various scripts, so I'm good to be removed.

@Jdcc-berkman since you have access to the host, can you please clean up those files? So I'll not accidentally delete anything valuable etc.. Thanks :)

@elukey: Is it also possible to kill jdcc's currently running processes as they are no longer being used? The reason I ask is because I am getting this error when running the scripts:

Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable.

This possibly may be related to jdcc's scripts running in addition to mine (same scripts) and therefore creating too many concurrent connections,

$ ps -u jdcc | wc -l
13

Or perhaps @Jdcc-berkman can do it themselves...

Thanks.

Hi @elukey: ping :)

Killed all the processes on stat1007 and also commented his crontab (so nothing should restart). Let me know if it is ok :)

In T219546#5192433, @elukey wrote:

Killed all the processes on stat1007 and also commented his crontab (so nothing should restart). Let me know if it is ok :)

Thanks very much! It's working now. :)

@ssingh anything left to be done?

Hi @elukey. The transition is working smoothly and there are no issues so we are good to go at least on my end.