{"id":381632,"date":"2024-06-29T03:46:52","date_gmt":"2024-06-29T03:46:52","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=381632"},"modified":"-0001-11-30T00:00:00","modified_gmt":"2024-06-29T03:46:52","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=381632","title":{"rendered":"Monitoring CPU\/RAM\/disk metrics with OpenTelemetry and Uptrace<\/span>"},"content":{"rendered":"
<\/div>\n
\n
\n
\n
\n

OpenTeleletry Collector is an open source data collection pipeline that allows you to monitor CPU, RAM, disk, network metrics, and many more.<\/p>\n

Collector itself does not include built-in storage or analysis capabilities, but you can export the data to Uptrace and ClickHouse, using them as a replacement for Grafana and Prometheus.<\/p>\n

When compared to Prometheus, ClickHouse can offer smaller on-disk data size and better query performance when analyzing millions of timeseries.<\/p>\n

What is OpenTelemetry?<\/h3>\n

OpenTelemetry<\/a> is an open-source observability framework hosted by Cloud Native Computing Foundation. It is a merger of OpenCensus and OpenTracing projects.<\/p>\n

OpenTelemetry provides a standardized way to capture and transmit metrics, traces, and logs from various software components in a distributed system.<\/p>\n

OpenTelemetry is designed to be vendor-agnostic and supports multiple programming languages, making it suitable for a wide range of applications and environments.<\/p>\n

OpenTelemetry Collector<\/h3>\n

OpenTelemetry Collector is a middleware between instrumented applications and various backends or observability platforms.<\/p>\n

OpenTelemetry Collector can also act as an agent that pulls telemetry data from systems you want to monitor and sends it to tracing tools using the OpenTelemetry protocol.<\/p>\n

For example, Collector can monitor Redis<\/a> by periodically running the INFO<\/code> command to collect telemetry data and send it to your observability pipeline for analysis and monitoring.<\/p>\n

\"Otel<\/p>\n
Otel Collector Pipeline<\/figcaption><\/div>\n<\/figure>\n

Host metrics<\/h3>\n

hostmetricsreceiver<\/a> is an OpenTelemetry Collector plugin that gathers various metrics about the host system, for example, CPU, RAM, disk metrics and other system-level metrics.<\/p>\n

However, OpenTelemetry itself does not include built-in storage or analysis capabilities for the collected data. Instead, you can export the data to an OpenTelemetry backend<\/a> of your choice such as Prometheus or Uptrace.<\/p>\n

To start collecting host metrics, you need to install Otel Collector<\/a> on each system you want to monitor and add the following lines to the Collector config:<\/p>\n

receivers:   hostmetrics:     collection_interval: 10s     scrapers:       # CPU utilization metrics       cpu:       # Disk I\/O metrics       disk:       # File System utilization metrics       filesystem:       # CPU load metrics       load:       # Memory utilization metrics       memory:       # Network interface I\/O metrics & TCP connection metrics       network:       # Paging\/Swap space utilization and I\/O metrics       paging: <\/code><\/pre>\n
See OpenTelemetry Collector host metrics<\/a> documentation for details.<\/p>\n
What is Uptrace?<\/h3>\n
Uptrace is an open source APM tool<\/a> that supports distributed tracing, metrics, and logs. You can use it to monitor applications and set up automatic alerts to receive notifications via email, Slack, Telegram, and more.<\/p>\n
Uptrace uses OpenTelelemetry to collect data and ClickHouse database to store it. Uptrace also requires PostgreSQL database to store metadata such as metric names and alerts.<\/p>\n
You can install Uptrace<\/a> binary or use the Docker example<\/a> to run the backend with a single command.<\/p>\n
After starting Uptrace, you will receive a data source name (DSN) that contains connection details for Uptrace.<\/p>\n
You can then export the data from Collector to Uptrace using the OTLP exporter and passing the DSN in headers:<\/p>\n
exporters:   otlp\/uptrace:     endpoint: localhost:14317     tls: { insecure: true }     headers: { 'uptrace-dsn': 'http:\/\/project1_secret_token@localhost:14317\/1' } <\/code><\/pre>\n
Dashboards<\/h3>\n
Uptrace maintains dashboards templates for monitoring system metrics, Redis, PostgreSQL, MySQL, Kafka, JVM, and many more. When the relevant metrics start arriving to Uptrace, it automatically creates dashboards from templates saving your time.<\/p>\n
Uptrace supports 2 types of dashboards:<\/p>\n