Why I am seeing a bunch of "monitor_log_write" warnings from my metrics logs?

I am getting a bunch of warning like the following

.... Warning, skipped writing 5844 log lines due to limit set by monitor_log_write_rate option...

Will that lead to logs lost?

Comments

  • Wei LiWei Li admin

    This issue typically occurred for customers who have so many filesystems mounted, because for each filesystem the system metrics monitor prints a bunch of statistics, and multiplied together that is more than the default allowed write rate for monitors.

    You can use the following agent configuration to resolve the issue.

    { 
       ....
       implicit_metric_monitor: false,
    
       monitors: [ 
              { 
                  module: "scalyr_agent.builtin_monitors.linux_system_metrics",
                  monitor_log_write_rate: 64000,
                  monitor_log_max_write_burst: 100000 
             } 
       ]
    }
    

    First, you disable the implicit metric monitor, because we need to create it explicitly with different values for the write rate.

    Then, create the Linux system metrics monitor with a higher write rate. If log lines are still being skipped, increase the write rate to 128000 and the write burst to 256000. Just make sure the monitor_log_write_rate is nice and high, and that the monitor_log_max_write_burst is something higher.

Sign In or Register to comment.