Page MenuHomePhabricator

Bug: "Total Bad Data" rows in IBM import should be in the correct format when exported out of Civi
Closed, ResolvedPublic0 Estimated Story Points

Description

Likely related to T228241

Here are some more details regarding the new fields. We mapped the fields and then ran the data import job as usual and you can see between these two summaries that the "Total Bad Data" row (last item) increased significantly:

screenshot in email "Re: IBM Import details"

I initially thought it may have been mapped incorrectly, however I had our email consultant (and IBM guru) take a look and he did the same thing I did. After we ran a few search queries in IBM for "null" data, we deduced that the bad data could be from IBM not recognizing the data due to either an incorrect date format for the following queries: [endowment_last_donation_date] & [endowment_first_donation_date], and/or incorrect number format (the file contains commas for 4-digit+ numbers) for all the others [total_2018], [total_2019], [total_2020], and [endowment_number_donations].

Our default date format is: mm/DD/YYY and numeric format does not contain commas.

Please let me know if you have any questions. I'll be on IRC :)

Thank you for your help!

Event Timeline

mepps renamed this task from New civi fields possibly imported incorrectly into IBM to Bug: "Total Bad Data" rows in IBM import should be in the correct format when exported out of Civi.Aug 2 2019, 6:03 PM

@MNoorWMF @CCogdill_WMF all the referenced fields are new fields per the deployment earlier this week. They don't seem to be outputting a different format but were they definitely added at the silverpop end?

Also note that @Ejegg has rolled back a different patch that went out this week so we will see what the output from that is

yes, we added them on Thursday. our consultant took a look at the import yesterday and it looks like the import results summary is reading the proper amounts again and i've run some queries this morning - everything looks to be fixed. this weekend's results were all good. thank you everyone!

Closing this for now. @Ejegg we need a task to address what you rolled back.