Mon, May 3
@DStrine and @Eileenmcnaughton - Brian found a lot of things in the MSL purge, so I've summarized it a Google Doc for now. Please let me know if this should be broken out in a new task or a series of tasks. https://docs.google.com/document/d/13PCYOVyUpOmm64YCpzgS621VK8bN8j0dmttZm7v1dqM/edit?usp=sharing
@DStrine TY page survey only - no surveys sent through email recently and no TY page survey updates recently. I've checked the es-419 and es TY pages and the survey links are correct and in Spanish for each. I'm seeing new responses come in for the es-419 survey.
Fri, Apr 30
Thanks @Eileenmcnaughton ! With that confirmed, I think we're good to resolve this ticket.
Thu, Apr 29
Ok, confirming that the full update is complete and the other job was just the regular smaller "7-day" file.
Thanks @Eileenmcnaughton and @Dwisehaupt - I'm manually running the job in Acoustic now and it grabbed the DatabaseUpdate-20210429005351.csv file first and it's about halfway through processing now. It also grabbed another file and it's currently "in waiting" in the Acoustic data job queue: DatabaseUpdate-20210429032733.csv, which I'm assuming is the regular 7-day update file?
Ok, great thank you so much. I'll temporarily disable Acoustic's job to process files at 9am UTC just so that I have more control when it's time to try processing this again tomorrow.
@Eileenmcnaughton If you wouldn't mind generating without that field if that's what it usually look like, that would be great! Acoustic has it's own modified date field anyway and re-mapping isn't as quick as it used to be. If you could please give me a ping here when the file is ready, I could kick it off again.
@Eileenmcnaughton No, sorry what I said wasn't clear. To clarify, Acoustic is set to read postal_code as a text field but it's being sent as a date field from Civi. Since the jobs have been running ok everyday with no bad data, I'm assuming that the issue is in the file. If you could confirm that the export file has postal_code as a text data field, then that will point to the issue being in Acoustic and then Brian and I can investigate further tomorrow.
@Eileenmcnaughton Oh it looks like it's attempting to read the postal_code as a date field instead of a text field.
@Eileenmcnaughton I can't remember if Acoustic provides much detail on which fields have dates not in that format, but I'm attempting to download a results log of the issues and maybe I can find more info for you.
Wed, Apr 28
Ok @Eileenmcnaughton some good news and bad news:
- It's running the correct file now which will take a while to finish
- Bad news is that so far it's reporting 6.5 million rows of bad data, the reason for which is "Date provided is not a valid date/format. Valid format: MM/dd/yyyy"
Yes - running it again now
Oh no, looks like it just ran DatabaseUpdate-20210428125329.csv again. I tried manually running it a second time and it failed (which means it didn't find anything on the FTP). Do you know of a way to have your file be first in the queue? I'm not sure why Acoustic isn't picking up both.
Thanks @Eileenmcnaughton - I'm making Acoustic run the job again and it's picked something up so I'll let you know if it completes successfully.
Thanks @Eileenmcnaughton - I just ran the job manually but getting a confusing result:
File Name: DatabaseUpdate-20210428125329.csv
Job ID: 183474622
Job Type: Recurring Contact Source Import @ Wednesday, April 28, 2021 at 7:51:04 PM GMT
Total Rows: 200560
Total Valid Rows: 200521
I'm only getting 200k rows. The File Name matches what already processed earlier this morning so I tried running it again but it's saying there's nothing on the FTP. Would you be willing to confirm the filename for the export you just sent?
Thanks @Eileenmcnaughton ! Let me know when the file is uploaded on FTP and I can manually process it on Acoustic's side.
Fri, Apr 23
Hi @Eileenmcnaughton. Confirming that I received the larger update in the Acoustic data job summary on April 22nd (pasted job below). If I recall correctly, I think the full export size used to be around 19 million rows.
Wed, Apr 21
@DStrine I've let Brian know we'll have a full data file coming soon. Can you confirm if the deployment will be this week or next?
Thu, Apr 15
Cool thanks, @DStrine - heads up that I'm floating those April WMF holidays to this weekend so I'll be around again starting next Tuesday.
Apr 8 2021
Got it! Sounds good to me, thanks!
Hey @DStrine ! We have no emails today or tomorrow, so I'm around to process the full export if that works for your team. I'm also going to be working Monday and we have no emails going out that day either.
Mar 31 2021
Quick update that I received some answers on these definitions:
Mar 23 2021
Following up that my previous comment is still accurate and I can resolve this task now. It seems this was a one-time occurrence and unfortunately there's nothing we can do to ensure that we will get external communications ahead of time from Acoustic.
Mar 22 2021
Hey everyone I'm back! I'm meeting with Brian tomorrow to talk through everything that happened last week, so I'll comment here again after that to wrap up and resolve this ticket.
Mar 13 2021
Mar 10 2021
Feb 24 2021
Hi @Eileenmcnaughton - I've just deleted the 20180828_Advocacy database in Acoustic, so we no longer need privacy erase requests going to ID: 20285807
Feb 16 2021
Hi @Eileenmcnaughton - I've just deleted the 20180208_StrategyEmailList database in Acoustic, so we no longer need privacy erase requests going to ID: 19429406
Feb 8 2021
Feb 4 2021
Updating here that the data for foundation_has_recurred_donation within Acoustic has been altered, so we currently can't use this field in our queries until we get a full Civi file ran through Acoustic. This issue will be fixed by default when the capitalization is fixed, but I'm bumping the priority up a notch since the has_recurred_donation field is now no longer usable in Acoustic.
Hi @Eileenmcnaughton ! Thanks for logging this. Everything you listed in still there in Silverpop, so please keep sending to all 6 privacy erase requests. Here's what each one is going to:
Feb 3 2021
Cool, this is now fixed. Thanks much @jgleeson ! Resolving this phab now :)
Thanks so much, @jgleeson ! Feel free to ping me on IRC when the job's complete. :)
Feb 1 2021
Hello! Just writing to confirm that the export/import process ran smoothly over the weekend with no failures. :) thanks so much for your help on this @jgleeson I'm resolving the ticket now.
Jan 29 2021
@jgleeson - I just finished mapping the fields for _all_Wikimedia. The summary report looks good and I'm posting it below:
Jan 28 2021
Hi @jgleeson ! Thanks for this update. I saw it before logging off today, so I went forward with doing a test run with our fake database.
@jgleeson and I met and discussed a game plan for the Matching Gifts fields. We decided to play things on the safe side and do a test run of importing the fields to a fake database, before running the real job on _all_Wikimedia.
Jan 27 2021
@jgleeson Yeah sounds great - let's do that. I'm free to hop on a call anytime between 1:30pm - 7pm UTC
We can try talking it out here, but I'm happy to jump on a call tomorrow too if you prefer.
hi @jgleeson - email team's last email deployment for this week is tomorrow, Thursday Jan 28th at 16:00 UTC. So, if things are ready on your side, I'll be ready to receive the new civi export file after 16:00 tomorrow. :)
Jan 26 2021
Thanks, I appreciate that. Confirming that resolving this task makes sense and we can move over to email or Slack about any followups. Thanks!
Hi @EYener - thanks for circling back! This was a one-off request that I don't believe we need anymore. However, the main idea of this task was to see if I can learn how to pull a longitudinal report from Civi. If I can learn how to do that, I won't need to loop in multiple stakeholders in order to give Trilogy a report. Is that a possibility?
Jan 20 2021
Thanks @DStrine ! Either option works for me. Thanks for the update :)
Hey @jgleeson ! Thanks for grabbing this task. Just wanted to say that pretty much any Thursday after 4pm UTC will work for me in regards to uploading the CSV file to FTP. I can remap the fields in my Thursday evenings and QA check it on Fridays. Let me know which Thursday sounds good for you and I'll plan for it :) Thanks!
Jan 13 2021
Sure, I can document on this phab. Brian did some digging on his end and found this in GDPR law:
Jan 12 2021
Yeah 450 days sounds correct to me too. Thanks so much for your work on this! @Eileenmcnaughton
Hi folks, I heard from @bsisolak that we can actually cancel this phab task, due to additional info we learned about GDPR law. It's best to keep an email address as opted out even if they request data erasure, to avoid the possibility of said person resubscribing.
Jan 11 2021
Hey all, I'm back and catching up on everything. Are there any new developments on this phab before I remap the _all_wikimedia database?
Dec 22 2020
Dec 21 2020
Talked with fr-tech about this phab on standup. Fr-tech cannot split up the civi export "bundle" (without extra coding work) and there's no real need for fr-tech to stop sending the files. We agreed to keep the exports running as is, and I will have Acoustic catch up when I'm back around on Jan 11th.
@Eileenmcnaughton I don't think we cancel emails very often, so I don't see a blocker from my side. Is it possible to allow cancelled sends in Superset for this specific date range?
Dec 17 2020
Dec 16 2020
Dec 15 2020
@EYener may be able to elaborate more on those ^questions, but some quick context about Superset:
Dec 11 2020
It's the 5th subject line segment from the "Recent" target group send on Dec 7th. It's just one single mailing_id (sp69308812) and the mailing name is R2-5b_20201207_UnitedStates(US)_English(en)_Email3_SL-Copy-DifferentCircumstances_Recent (2)
Thanks for the tag! I would set this as a medium or low priority. I would very much like this to show up in Superset at some point, but it's not a blocker for our processes. I can pull the stats up the old fashioned way for now. The send size for this segment is 15k. Thanks all!
Dec 9 2020
Dec 2 2020
Quickly chiming in here that _all_Wikimedia already has been mapped with employer_id and employer_name from the civi export.
@MSuijkerbuijk_WMF Yes, I sent over a training doc I had via email.
Thanks for that clarification, @Eileenmcnaughton !
Dec 1 2020
Nov 24 2020
Moska is out of office, but I'm pretty sure QA checks are almost complete except one last thing Briana needs Moska for. That's all I know for now, but Moska returns tomorrow so she may be able to chime in then. Thanks!
Nov 12 2020
Thanks @jgleeson ! I checked out the recurring import jobs and they processed yesterday's again but also the correct day's files right after. We're all set today for sending out emails :)
Nov 11 2020
Sure thing, @DStrine ! Apologies, I was under the impression that re-using the same task was preferred, as the issue replicates the same way on the Acoustic side. I'll make new tasks from now on. Shall I resolve this one now and make a new task for tomorrow if needed?
Issue is resolved for today! Big thanks to @Ejegg and @jgleeson. From what I read in the chat and email today, it seems that in the short-term, fr-tech will need to fix data manually for each query before re-running the export job. @jgleeson will kindly check back in tomorrow to see if the export needs redoing then. Thank you, Jack!
Thanks! @Ejegg looked into it and sent over the files again. I processed them on my side and so I think we are good. I do want to quickly note that the Unsubscribes file doesn't seem to be increasing (like it usually does) for the last few days. Dropping in yesterday's job and today's job:
The civi export file failed again this morning, but we already had emails go out at 10:00 UTC. Donor services has been notified. We have another batch of emails scheduled for tomorrow, so it's urgent for us to have the export running today and tomorrow. Is there someone working today who is willing to look into this? Thanks!
Nov 10 2020
This issue is now resolved. We had a bump in the road with manual exports failing a few times, and then a bad data issue with fields including a "b'" before the values.
Thanks for working on this today!
Nov 4 2020
Noting this issue happened again today (Nov 4, 2020). @jgleeson noticed and kicked off a rerun this morning, so we are clear to send emails today.
Oct 30 2020
The conversation about the few un-updated contacts has been moved to this phab T254304 so we can "officially" close this one out. Thanks!
Adding in some new mystery contacts from another phab (T265700) into this one to investigate why a small number of contacts did not get modified by a full database update and are also not on the Master Suppression List.
Acoustic import ran successfully and looks good to me. Thanks for the help! Resolving this task now.
Met with fr-tech team today! @Ejegg found that the "uploader" is being kicked off too soon before the "builder" is finished. He kicked off a new file for me and is working on making sure the uploads wait to deploy until after the building job completes. Let me know if I got that right :)
Whoops, just realized Jack, Eileen, and Maggie are OOO today. @Ejegg - are you available to triage this?
Hey @jgleeson. I'm opening this ticket again as the export sent over yesterday's files again. Luckily, there's no emails going out today, but we pick back up on Monday.
Oct 28 2020
Also, just wanting to realign on the main issue of this task:
yahoo.com is bouncing at 3% now based on the latest enUS send we deployed, which seems alarming since it's the second most popular domain group in every email send. @EYener is it possible to pull a civi report tracking yahoo specifically?
Confirming that the import jobs completed successfully now and we are rescheduling emails. Do we want to keep this phab open for the re-try code or should I resolve this task?