What I normally do is test doing a UI import to the target smart import mapping & check it's OK (e.g 200 rows) & get Leanne to check.

Then I upload & use the api in org.wikimedia.civicrm.targetsmart to import it (see existing scheduled jobs).

There is one complexity - we generally need a couple of pseudocolumns at the end of the file - e.g 'Supplemental Address 1' so that we don't get the street_address updated & an old supplemental address left in the middle of the new address. When doing a UI import you generally add a couple of columns to achieve this & on the api import there is a parameter

$spec['null_rows_at_end_count'] = [
  'title' => E::ts('Number of "nulls" columns to add at the end to blank out data'),
  'type' => CRM_Utils_Type::T_INT,
  'api.default' => 0,
];

@Eileenmcnaughton I just added the amended file to the server here: Tech/Major Gifts/TargetSmart Update/2020 Update/Amended contacts/June 2020

Re: the supplemental address, they included in the file, but for this segment I don't think we'll need to update address. Could you skip the address fields in the mapping?

Thank you!

I'm looking at this, and trying to update the saved field mappings to ignore address (in staging). For some reason I keep getting an error on the import ui. I'll look at whether there's a better way to do this. I found the api call in the targetsmart cron job.

I uploaded the import file to our civi server in tmp.

and dev

Basically if we're trying to ignore address fields, we'd want to update the saved field mapping to do not import on those but agian, that screen keeps erroring out for me saying it's not secure. I can get to the import screen and select the csv but I hit submit it doesn't go anywhere.

Thanks for working on this, @mepps! Any chance we'll be able to resolve
this during this week? We're hoping to have all the data we need by Monday
so we restart our targeting work.

I was able to upload the first 200 through the UI to this group civicrm/group/search?force=1&context=smog&gid=818 and didn't hit the issues mepps hit. Normally @LeanneS takes a look at the sample import before we queue it up. With no address data it's a bit simpler - not sure if @LeanneS is around...

Ah got it, I didn’t realize how the process worked. We may just have to
move some deadlines set for us to review the targeting strategy with
leadership. Pats said that’s fine so just keep us posted on an ETA.

Le jeu. 2 juil. 2020 à 16:42, Eileenmcnaughton <
no-reply@phabricator.wikimedia.org> a écrit :

Eileenmcnaughton added a comment. View Task
https://phabricator.wikimedia.org/T255805

I was able to upload the first 200 through the UI to this group
civicrm/group/search?force=1&context=smog&gid=818 and didn't hit the issues
mepps hit. Normally @LeanneS
https://phabricator.wikimedia.org/p/LeanneS/ takes a look at the sample
import before we queue it up. With no address data it's a bit simpler - not
sure if @LeanneS https://phabricator.wikimedia.org/p/LeanneS/ is
around...

*TASK DETAIL*
https://phabricator.wikimedia.org/T255805

*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

*To: *Eileenmcnaughton

*Cc: *mepps, KHaggard, MSuijkerbuijk_WMF, CCogdill_WMF, EYener,
Eileenmcnaughton, DStrine, LeanneS, Aklapper, EBjune

@CCogdill_WMF I'm just uploading the file now to try with the script to check that & then it will just be a case of turning it on - it doesn't have to be @LeanneS I don't imagine - another set of eyes is probably enough. It's only 47 minutes since I first pinged Leanne right now though :-)

@Eileenmcnaughton @CCogdill_WMF I just did a spot check of a few in this group and they looked good. Happy to do a more thorough review on Monday, but the first handful seem mapped nicely.

OK - I have run the first 2000 rows in via our command line method - the command works and simply requires the schedule to be enabled in process control job targetsmart_import.yaml (this can be done by any fr-tech team member).

The question now is the balance between doing further checks on the imported contacts vs the fact it will take a while for the csv to be processed - the schedule (from last time) suggests 1000 every 6 minutes = 10000 per hour -which would mean around 80 hours from when it's enabled (which is a 2 minute job to do)

@Eileenmcnaughton Did another review and the contacts look good!

This is now running - I just checked & 18k contacts have been processed

I've left this in pending deployment as I guess 'done' is the job being disabled again

Cool! So are we expecting this will take a few days to complete? Should we
expect it in the email export by EOW, or what do you think?

@CCogdill_WMF I just checked and around ~200k of ~800k are in - we can up the schedule a bit if needed since there is not other heavy traffic at the moment

Awesome, thank you! Let us know your ETA so we can get our ducks in a row
and prepare for the next steps in our targeting projections :)

by Eileen's estimates this should be done by the end of the week.

Okay great, thanks for the confirmation!

Sorry, one more question: Will this data automatically be included in the
Acoustic export once it's imported? Or is there a separate task for that?

@CCogdill_WMF yes - they should be automatically uploaded - the first 400k or so should be in the upload about to finish

Great, thank you!