Lets think through what we want to change to address the following:
- We have some new calculations that need to be added to the export: T252245
- We are also running into upload file sizes and we can't compress the file. **
Approach**
We should switch from a full table refresh to incremental refreshes, using the civicrm_contact.modified_date and the log_date fields on the log tables to find changed contacts.
The goal would be the table would be always in a state where an update could be taken from it. This table is likely to be used for other purposes -searching, Erin's queries so may hold fields beyond just the export