Vector Remap Language (VRL)

Vector Remap Language (VRL) is an expression-oriented language designed for transforming observability data (logs and metrics) in a safe and performant manner. It features a simple syntax and a rich set of built-in functions tailored specifically to observability use cases.

You can use VRL in Vector via the remap transform. For a more in-depth picture, see the announcement blog post.

linkQuickstart

VRL programs act on a single observability event and can be used to:

• Transform observability events
• Specify conditions for routing and filtering events

Those programs are specified as part of your Vector configuration. Here's an example remap transform that contains a VRL program in the source field:

1[transforms.modify]
2type = "remap"
3inputs = ["logs"]
4source = '''
5  del(.user_info)
6  .timestamp = now()
7'''

This program changes the contents of each event that passes through this transform, deleting the user_info field and adding a timestamp to the event.

linkExample: parsing JSON

Let's have a look at a more complex example. Imagine that you're working with HTTP log events that look like this:

1"{\"status\":200,\"timestamp\":\"2021-03-01T19:19:24.646170Z\",\"message\":\"SUCCESS\",\"username\":\"ub40fan4life\"}"

You want to apply these changes to each event:

• Parse the raw string into JSON
• Reformat the time into a UNIX timestamp
• Remove the username field
• Convert the message to lowercase

This VRL program would accomplish all of that:

1. = parse_json!(string!(.message))
2.timestamp = to_unix_timestamp(to_timestamp!(.timestamp))
3del(.username)
4.message = downcase(string!(.message))

Finally, the resulting event:

1{
2  "message": "success",
3  "status": 200,
4  "timestamp": 1614626364
5}

linkExample: filtering events

The JSON parsing program in the example above modifies the contents of each event. But you can also use VRL to specify conditions, which convert events into a single Boolean expression. Here's an example filter transform that filters out all messages for which the severity field equals "info":

1[transforms.filter_out_info]
2type = "filter"
3inputs = ["logs"]
4condition = '.severity != "info"'

Conditions can also be more multifaceted. This condition would filter out all events for which the severity field is "info", the status_code field is greater than or equal to 400, and the host field isn't set:

1condition = '.severity != "info" && .status_code < 400 && exists(.host)

{{< info title="More VRL examples" >}} You can find more VRL examples further down on this page or in the VRL example reference. {{< /info >}}

linkReference

All language constructs are contained in the following reference pages. Use these references as you write your VRL programs:

{{< pages >}}

linkLearn

VRL is designed to minimize the learning curve. These resources can help you get acquainted with Vector and VRL:

{{< jump "/docs/setup/quickstart" >}} {{< jump "/guides/level-up/transformation" >}}

linkThe goals of VRL

VRL is built by the Vector team and its development is guided by two core goals, safety and performance, without compromising on flexibility. This makes VRL ideal for critical, performance-sensitive infrastructure, like observabiity pipelines. To illustrate how we achieve these, below is a VRL feature matrix across these principles:

Feature	Safety	Performance
Compilation	✅	✅
Ergonomic safety	✅	✅
Fail safety	✅
Memory safety	✅
Vector and Rust native	✅	✅
Statelessness	✅	✅

linkConcepts

VRL has some core concepts that you should be aware of as you dive in.

{{< vrl/concepts >}}

linkFeatures

{{< vrl/features >}}

linkPrinciples

{{< vrl/principles >}}

linkOther examples

{{< vrl/real-world-examples >}}