Errors during core:archive

I hate getting these messages when running core:archive in Matomo.

• WARNING   [2024-04-24 14:47:54] 6414  /var/www/littlefurnace.com/matomo/core/CliMulti/Process.php(314): Warning - shell_exec(): Unable to execute 'ps x 2>/dev/null | awk '! /defunct/ {print $1}' 2>/dev/null' - Matomo 5.0.3 - Please report this message in the Matomo forums: https://forum.matomo.org (please do a search first as it might have been reported already)
• ERROR     [2024-04-24 16:04:11] 9574  Error unserializing the following response from ?module=API&method=CoreAdminHome.archiveReports&idSite=5&period=week&date=2024-02-26&format=json&segment=actionType%3D%3Dpageviews%3Bactions%3E%3D3&plugin=Funnels&trigger=archivephp&pluginOnly=1&requestedReport=Funnels_exits_2: ''
• ERROR     [2024-04-24 16:04:11] 9574  Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=5&period=day&date=2024-02-15&format=json&segment=actionType%3D%3Dpageviews%3Bactions%3E%3D3&plugin=Funnels&trigger=archivephp&pluginOnly=1&requestedReport=Funnels_funnel_sum_entries_2. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file.  For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log

My Matomo server is a $5 shared space hosted on Digital Ocean, the cheapest deployment, the least memory, all minimal RAM, CPUs, everything. 

Frustrated by these errors, I learned to overcome these errors by running console core:archive manually and changing options until it runs well.   Here are the things I do.

1. Incrementally run longer time spans.  I run today and yesterday. Then this month. Then forever.

2. Run it once with no segments. Then run it with segments.

3. Run it with the PHP memory allocation declared at run time.


Case 1:
Got errors during the archiving, in the section before scheduled tasks.
Added the option --skip-all-segments. Still errors.
Deleted the segment from the UI. Success!  (Not sure why the option didn't do the trick.)

​Case 2 (September 2024 update) 
1. Tons of errors: sudo su www-data -s /bin/bash -c  "php   -d memory_limit=8G  /var/www/matomo/console core:archive" 
2. Zero errors: sudo su www-data -s /bin/bash -c  "php   -d memory_limit=8G  /var/www/littlefurnace.com/matomo/console core:archive --skip-all-segments --force-idsites=2"
Conclusion: On underpowered servers, you can eliminate errors by doing one site at a time and by skipping segments."

Then go back and do the segments, still doing only one site at a time
3.  sudo su www-data -s /bin/bash -c  "php   -d memory_limit=8G  /var/www/littlefurnace.com/matomo/console core:archive  --force-idsites=1"
​