Queued Tracking Troubleshooting in Matomo

So, you're facing an issue with the Matomo Queued Tracking plugin, where large request sets are failing to process. Maybe your error indicates a rollback occurred because the process could not acquire a lock. This situation can arise due to configuration issues or contention for resources.
Here’s how you can troubleshoot and resolve the issue:
---
### 1. **Check Lock Status**
The command provided in the error message helps verify whether a lock is active:
```bash
php /var/www/html/matomo/console queuedtracking:lock-status
```
Run this command to check if any other processes are holding a lock. If a lock is stuck, you may need to clear it manually.
---
### 2. **Manually Clear the Lock**
If a stuck lock is detected, you can clear it by running:
```bash
php /var/www/html/matomo/console queuedtracking:unlock
```
This command ensures no stale locks block the process.
---
### 3. **Optimize the Request Set Size**
Having over 200,000 requests in a single set can cause memory and processing issues. Reduce the batch size by configuring the `maxRequestsPerQueue` setting in `config/config.ini.php`:
```ini
[QueuedTracking]
maxRequestsPerQueue = 50000
```
Restart Matomo after making this change to ensure it takes effect.
---
### 4. **Increase Resources**
Your log indicates memory usage peaking at 15.64 GB. If this is close to your system’s limit, the process might fail due to insufficient resources. Consider increasing:
- PHP memory limit in `php.ini`:
```ini
memory_limit = 32G
```
- Worker processes (use with caution, especially if resources are constrained):
```bash
php /var/www/html/matomo/console queuedtracking:process --workers=4
```
---
### 5. **Process Smaller Batches**
Process fewer request sets at a time by adding a limit to the processing command:
```bash
php /var/www/html/matomo/console queuedtracking:process --queue-id=0 --force-num-requests-process-at-once=10000
```
This ensures that requests are handled in smaller chunks, reducing the load on your system.
---
### 6. **Monitor Logs**
Review Matomo logs for detailed error messages:
```bash
tail -f /var/www/html/matomo/tmp/logs/matomo.log
```
Identify if there are issues related to specific request sets or database performance.
---
### 7. **Database Tuning**
Since queued tracking relies heavily on the database, ensure your database is optimized:
- Increase `innodb_buffer_pool_size` in your MySQL configuration.
- Ensure indexes on the `log_*` tables are optimized.
- Regularly archive old data using Matomo’s cron jobs.
---
### 8. **Restart Services**
After making changes, restart necessary services:
```bash
sudo service php8.3-fpm restart
sudo service mysql restart
```
---
### Final Step
After applying these changes, re-run the processing command:
```bash
php /var/www/html/matomo/console queuedtracking:process
```
---
These steps should help resolve your issue. Let me know if you need further assistance!