I am getting a bunch of warning like the following
.... Warning, skipped writing 5844 log lines due to limit set by monitor_log_write_rate option...
Will that lead to logs lost?
This issue typically occurred for customers who have so many filesystems mounted, because for each filesystem the system metrics monitor prints a bunch of statistics, and multiplied together that is more than the default allowed write rate for monitors.
You can use the following agent configuration to resolve the issue.
First, you disable the implicit metric monitor, because we need to create it explicitly with different values for the write rate.
Then, create the Linux system metrics monitor with a higher write rate. If log lines are still being skipped, increase the write rate to 128000 and the write burst to 256000. Just make sure the monitor_log_write_rate is nice and high, and that the monitor_log_max_write_burst is something higher.