How is Scalyr log volume calculated?
The log volume ingested by Scalyr is the basis for billing, so it's important for customers to have an exact understanding how this volume is calculated, in order to optimize their logs and agent configurations to get the best value out of the service.
I can think of a number of different types of data that could contribute to log volume:
- The raw log messages that are read by the
scalyr-agentfrom a log file.
server_attributesdefined globally in the
agent.jsonconfiguration file (which are sent by the agent with every HTTP request to the Scalyr backend, I believe).
attributesdefined for each monitored log file in the
logssection of the
agent.jsonfile (which are added to each log event, I believe).
- For both
server_attributesthere are standard keys (e.g.
serverHost, some Kubernetes metadata) as well as custom keys defined by the user.
Could you document how these different data sources (and potentially others?) play into the log volume calculation? Are attributes included in the volume? How is this different for server and log attributes? Do both keys and values count, or only values? Are standard and custom attributes handled differently? Does compression play a role?