Dec 4 2019
That makes sense. I'm attaching the most recent jobs from this morning below - and things seem to be running as normal again. Thanks for all your help here! I think we can close this.
Dec 2 2019
Thanks! New ticket is here: T239635
I can open a new phab task - just confirming that: the contacts in the MSL should not be on the _all_wikimedia database contact source update job, is that correct? So when there’s an increase in the Master Suppression List, a decrease in the main database should be expected?
@Ejegg One thing I would like to ask about: even though the jobs are running smoothly again, the total # rows is still roughly 1 million below what it used to be. Just wondering if this is worth looking into further. Attaching the jobs from 11/21 (before the timing issue started). You can see the total is 17 mil and today's current total is 16 mil.
Nov 28 2019
Thanks - the jobs this morning ran without failure and the numbers look healthy to me. Attaching below.
Nov 27 2019
Hey @Ejegg I manually ran the import job again so it should be out of the way for the new file tomorrow, but would you be willing to confirm that it has been deleted from SFTP?
Thanks @Ejegg I'll update this thread letting you know if the totals look healthy again tomorrow morning :)
Nov 26 2019
We tell IBM to look for “DatabaseUpdate-*” and “Unsubscribes-*” which matches the beginning nomenclature of the FTP file. It then grabs the oldest file first since we don't specify the date. So in the case of multiple days backed up, we manually run the data job via that screenshot I added however many times it takes to catch up.
Sure, let me know the best way I can show you, or I will try writing it out here and just let me know if it makes sense.
Updating that the _all_wikimedia data job runs successfully but the Master Suppression List does not, but I am able to manually run the MSL job with success.
Nov 25 2019
Nov 19 2019
Nov 8 2019
False alarm, the import this morning is back to normal and pulling the correct date file. Sorry for flagging that too early! I think we are all good here. Thanks for your help and have a great weekend!
Nov 7 2019
Thanks, Ejegg ! I checked again and the password expiration countdown banner has gone away for this user 😄 So I think this part is resolved. Thank you!
Nov 6 2019
Bumping this again, the silverpop data upload account is saying the password will expire in 5 days, so I think the password needs to be reset via a direct account log in before then.
Nov 5 2019
We received a response to why the password keeps expiring every 7 days:
Nov 4 2019
Hi all, I was able to run the import jobs manually today and the totals look good.
Nov 1 2019
Quick update: We can't manually run the file because the FTP file isn't there, and my thinking is that it wasn't there for the scheduled 9am GMT time either. Caitlin and I think maybe the file upload must have failed.
Oct 31 2019
Thanks for the heads up @Ejegg ! It definitely did spike this morning :)
Oct 23 2019
Hi @Ejegg - The imports ran successfully this morning 🙌🏼 Thanks for your awesome work and speedy replies!
Oct 22 2019
Thanks for investigating! Ok I will keep the current 9am GMT run time for IBM and will let you know if it is successful tomorrow :)
Thanks, @Ejegg ! If the load time is not fixable, would fr-tech be willing to set the export time 2 hours earlier?
Adding a quick update here that both imports failed this morning. I will wait a couple hours and then run the imports manually. Please let me know if you need me to hold off on doing that for any reason. Attaching screenshot below if it's helpful.
Oct 4 2019
z_birth_date has been added to the recurring import :) Thanks everyone!
Oct 3 2019
Got it - did not see it, so running today's import now and will try again tomorrow.
@CCogdill_WMF ok so I should be seeing birth_date when setting up the import today or do I manually add that in as a date field?
Sep 24 2019
Sep 10 2019
Hi @Eileenmcnaughton! Yes, it worked :) Thanks for the work you put into this!
Jul 17 2019
Jul 16 2019
Jul 5 2019
Update: Looks like mysql pgehres is able to pull the clicks data today. Not sure what has changed, but the query no longer produces an empty set.
Jul 3 2019
Jun 28 2019
@CCogdill_WMF Hello! Yes, I'm attaching a screenshot of yesterday's import next to today's import. Looks like the main database decreased by ~1k and the unsubscribes increased by ~1k also.
Jun 5 2019
This morning, I just saw a slight decrease again. Please see attached screenshot.
May 20 2019
Nov 13 2018
Password received, thank you.