Actually passing filters id = [] doesn't work (you can achieve this by selecting download without selecting any rows, which does work for a lower number of rows, but not when there are as many as there are in the SK in question). So there's something more going on here that would need looking into.

@Lars in nginx client_max_body_size was already increased to 10M in T219191. Also, on the php side we have the following php adjustments from that task :

post_max_size            => '12M',
upload_max_filesize      => '10M',

@Lars this is on the list for CiviCore work consideration, right?

@AKanji-WMF Potentially, I need to dig in a little more to figure out if this is something in core or something on our end.
Thanks @Dwisehaupt, looks like that isn't the issue then.

@MBeat33 also reports failures for this SK with only 24k rows.

@Dwisehaupt Can we increase client_max_body_size beyond 10M in nginx? I seem to be hitting a limit around 8M actual file size, which seems like it wouldn't be a problem even with overhead, but worth trying just in case it is that.

I think ideally we should be able to download relatively large files from SearchKit in CiviCRM, maybe 32M or 64M would make sense?

@Dwisehaupt I think we should increase that file size limit above, but I'm convinced that it isn't the issue, as I'm unable to download even a 2MB xlsx of the same data. Trying with different files makes it clear that it isn't a time issue (for some files it fails in 15s, others it fails in 30s). So I think that leaves memory, could we be hitting a memory limit somehow trying to write a csv or xlsx?

I'm not getting any errors on civi1002, but maybe trying to logging you set up for the import batch issue might give us some info?

@Dwisehaupt Got it:

Mar 19 20:22:19 civi1002 apache2[450077]: [php:error] [pid 450077:tid 450077] [client 127.0.0.1:51834] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 2093224 bytes) in /srv/org.wikimedia.civicrm/vendor/league/csv/src/Writer.php on line 66, referer: https://civicrm.wikimedia.org/civicrm/admin/search

Could we try 512MB? And client_max_body_size of 32MB as above?

In nginx config:

• update client_max_body_size: 10M -> 32M

In php apache config:

• update memory_limit: 256M -> 512M

@MBeat33 We've figured out that it was hitting memory limits and have doubled the limit here. I can now download about 75,000 rows from the fraud SK linked above (when downloading a csv, trying to download a xlsx is significantly worse so I wouldn't recommend it for large files). Hopefully that's enough for your needs, but if this becomes an issue again we could potentially go higher (or investigate why the csv writer is such a memory hog).

@Lars that's plenty for my needs! And yeah, I never use the xlsx option. Thanks for fixing this.

@MBeat33 Great. And I should also mention that since it's a memory issue, the number of columns and the data in them matters too. So if you find yourself not being able to download, you can cut down the number of fields to those you really need and might be able to get it to work.

Thanks, I'm happy to tinker as needed ;)