Prometheus Exporter

CAUTION

High cardinality metric names and labels are discouraged by Prometheus as they can provide performance and reliability problems. You should consider alternative strategies to reduce the cardinality. Vector offers a tag_cardinality_limit transform as a way to protect against this.

Example Configuration

Counter

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"
3default_namespace = "service"

1{
2  "metric": {
3    "kind": "incremental",
4    "name": "logins",
5    "counter": {
6      "value": 1.5
7    },
8    "tags": {
9      "host": "my-host.local"
10    }
11  }
12}

1"# HELP service_logins logins\n# TYPE service_logins counter\nservice_logins{host=\"my-host.local\"} 1.5"

Gauge

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"

1{
2  "metric": {
3    "kind": "absolute",
4    "name": "memory_rss",
5    "namespace": "app",
6    "gauge": {
7      "value": 1.5
8    },
9    "tags": {
10      "host": "my-host.local"
11    }
12  }
13}

1"# HELP app_memory_rss memory_rss\n# TYPE app_memory_rss gauge\napp_memory_rss{host=\"my-host.local\"} 1.5"

Histogram

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"

1{
2  "metric": {
3    "kind": "absolute",
4    "name": "response_time_s",
5    "tags": {},
6    "histogram": {
7      "buckets": [
8        {
9          "upper_limit": 0.005,
10          "count": 0
11        },
12        {
13          "upper_limit": 0.01,
14          "count": 1
15        },
16        {
17          "upper_limit": 0.025,
18          "count": 0
19        },
20        {
21          "upper_limit": 0.05,
22          "count": 1
23        },
24        {
25          "upper_limit": 0.1,
26          "count": 0
27        },
28        {
29          "upper_limit": 0.25,
30          "count": 0
31        },
32        {
33          "upper_limit": 0.5,
34          "count": 0
35        },
36        {
37          "upper_limit": 1,
38          "count": 0
39        },
40        {
41          "upper_limit": 2.5,
42          "count": 0
43        },
44        {
45          "upper_limit": 5,
46          "count": 0
47        },
48        {
49          "upper_limit": 10,
50          "count": 0
51        }
52      ],
53      "count": 2,
54      "sum": 0.789
55    }
56  }
57}

1"# HELP response_time_s response_time_s\n# TYPE response_time_s histogram\nresponse_time_s_bucket{le=\"0.005\"} 0\nresponse_time_s_bucket{le=\"0.01\"} 1\nresponse_time_s_bucket{le=\"0.025\"} 0\nresponse_time_s_bucket{le=\"0.05\"} 1\nresponse_time_s_bucket{le=\"0.1\"} 0\nresponse_time_s_bucket{le=\"0.25\"} 0\nresponse_time_s_bucket{le=\"0.5\"} 0\nresponse_time_s_bucket{le=\"1.0\"} 0\nresponse_time_s_bucket{le=\"2.5\"} 0\nresponse_time_s_bucket{le=\"5.0\"} 0\nresponse_time_s_bucket{le=\"10.0\"} 0\nresponse_time_s_bucket{le=\"+Inf\"} 0\nresponse_time_s_sum 0.789\nresponse_time_s_count 2"

Distribution to histogram

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"
3buckets = [ 0, 1, 3 ]

1{
2  "metric": {
3    "name": "request_retries",
4    "kind": "incremental",
5    "distribution": {
6      "samples": [
7        {
8          "value": 0,
9          "rate": 4
10        },
11        {
12          "value": 1,
13          "rate": 2
14        },
15        {
16          "value": 4,
17          "rate": 1
18        }
19      ],
20      "statistic": "histogram"
21    },
22    "tags": {
23      "host": "my-host.local"
24    }
25  }
26}

1"# HELP request_retries request_retries\n# TYPE request_retries histogram\nrequest_retries_bucket{host=\"my-host.local\",le=\"0\"} 4\nrequest_retries_bucket{host=\"my-host.local\",le=\"1\"} 6\nrequest_retries_bucket{host=\"my-host.local\",le=\"3\"} 6\nrequest_retries_bucket{host=\"my-host.local\",le=\"+Inf\"} 7\nrequest_retries_sum{host=\"my-host.local\"} 6\nrequest_retries_count{host=\"my-host.local\"} 7"

Distribution to summary

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"
3quantiles = [ 0.5, 0.75, 0.95 ]

1{
2  "metric": {
3    "name": "request_retries",
4    "kind": "incremental",
5    "tags": {},
6    "distribution": {
7      "samples": [
8        {
9          "value": 0,
10          "rate": 3
11        },
12        {
13          "value": 1,
14          "rate": 2
15        },
16        {
17          "value": 4,
18          "rate": 1
19        }
20      ],
21      "statistic": "summary"
22    }
23  }
24}

1"# HELP request_retries request_retries\n# TYPE request_retries summary\nrequest_retries{quantile=\"0.5\"} 0\nrequest_retries{quantile=\"0.75\"} 1\nrequest_retries{quantile=\"0.95\"} 4\nrequest_retries_sum 6\nrequest_retries_count 6\nrequest_retries_min 0\nrequest_retries_max 4\nrequest_retries_avg 1"

Summary

Config

Input

Output

1[sinks.my_sink_id]
2type = "prometheus_exporter"

1{
2  "metric": {
3    "name": "requests",
4    "kind": "absolute",
5    "summary": {
6      "quantiles": [
7        {
8          "upper_limit": 0.01,
9          "value": 1.5
10        },
11        {
12          "upper_limit": 0.5,
13          "value": 2
14        },
15        {
16          "upper_limit": 0.99,
17          "value": 3
18        }
19      ],
20      "count": 6,
21      "sum": 12
22    },
23    "tags": {
24      "host": "my-host.local"
25    }
26  }
27}

1"# HELP requests requests\n# TYPE requests summary\nrequests{host=\"my-host.local\",quantile=\"0.01\"} 1.5\nrequests{host=\"my-host.local\",quantile=\"0.5\"} 2\nrequests{host=\"my-host.local\",quantile=\"0.99\"} 3\nrequests_sum{host=\"my-host.local\"} 12\nrequests_count{host=\"my-host.local\"} 6"

Configuration Options

Required Options

address(required)

The address to expose for scraping.

Type	Syntax	Default	Example
string	literal		["0.0.0.0:9598"]

inputs(required)

A list of upstream source or transform IDs. Wildcards (*) are supported.

See configuration for more info.

Type	Syntax	Default	Example
array	literal		["my-source-or-transform-id","prefix-*"]

type(required)

The component type. This is a required field for all components and tells Vector which component to use.

Type	Syntax	Default	Example
string	literal		["prometheus_exporter"]

Advanced Options

buckets(optional)

Default buckets to use for aggregating distribution metrics into histograms.

Type	Syntax	Default	Example
array		0.0050.010.0250.050.10.250.512.5510	[0.005,0.01]

flush_period_secs(optional)

Time interval between set values are reset.

Type	Syntax	Default	Example
uint		60

default_namespace(optional)

Used as a namespace for metrics that don't have it. Typically namespaces are set during ingestion (sources), but it is optional and when missing, we'll use this value. It should follow Prometheus naming conventions.

Type	Syntax	Default	Example
string	literal		["service"]

quantiles(optional)

Quantiles to use for aggregating distribution metrics into a summary.

Type	Syntax	Default	Example
array		0.50.750.90.950.99	[0.5,0.75,0.9,0.95,0.99]

How it Works

Histogram Buckets

Choosing the appropriate buckets for Prometheus histograms is a complicated point of discussion. The Histograms and Summaries Prometheus guide provides a good overview of histograms, buckets, summaries, and how you should think about configuring them. The buckets you choose should align with your known range and distribution of values as well as how you plan to report on them. The aforementioned guide provides examples on how you should align them.

State

This component is stateful, meaning its behavior changes based on previous inputs (events). State is not preserved across restarts, therefore state-dependent behavior will reset between restarts and depend on the inputs (events) received since the most recent restart.

Memory Usage

Like other Prometheus instances, the prometheus sink aggregates metrics in memory which keeps the memory footprint to a minimum if Prometheus fails to scrape the Vector instance over an extended period of time. The downside is that data will be lost if Vector is restarted. This is by design of Prometheus' pull model approach, but is worth noting if restart Vector frequently.