How to Monitor and Reduce Log Volume

  • Creating Alerts To Monitor Log Volume

Set alert configuration to get notified when log volume exceeds a certain threshold. For example, alert if there is more than 120gb ingested in a 24 hour period.

      alerts: [
        {
          trigger: "sumPerSecond:1d(tag='logVolume' metric='logBytes') * 0.00008046627 > 120",   
          alertAddress: "enter webhook or email here",
          description: "Exceeding 120 GB/Day"
        }
      ]

Set alert configuration to get notified when log volume is X percent greater than it was the same day the previous week. For example, alert if there is an increase of 10 percent week over week.

    alerts: [
        {
          trigger: "sumPerSecond:1d(tag='logVolume' metric='logBytes') >= (sumPerSecond:8d(tag='logVolume' metric='logBytes') - sumPerSecond:7d(tag='logVolume' metric='logBytes')) * 1.1",
          alertAddress: "enter webhook or email here",
          description: "Week Over Week Log Volume Increase"
        },
      ]
  • Turn off Metrics and Agent Logs to Reduce Log Volume

Disable metrics log and agent log due to cost concern, you can disable them by setting the following plugins in the agent configuration

    implicit_metric_monitor: false,
    implicit_agent_process_metrics_monitor: false,
    implicit_agent_log_collection: false
Sign In or Register to comment.