Split

CAUTION

This transform has been deprecated in favor of the remap transform, which enables you to use Vector Remap Language (VRL for short) to create transform logic of any degree of complexity. The examples below show how you can use VRL to replace this transform's functionality.

.message = split(.message)

Example Configuration

Split log message

Config

Input

Output

1[transforms.my_transform_id]
2type = "split"
3field_names = [
4  "remote_addr",
5  "user_id",
6  "timestamp",
7  "message",
8  "status",
9  "bytes"
10]
11separator = ","
12field = "message"
13
14  [transforms.my_transform_id.types]
15  status = "int"
16  bytes = "int"

1{
2  "log": {
3    "message": "5.86.210.12,zieme4647,19/06/2019:17:20:49 -0400,GET /embrace/supply-chains/dynamic/vertical,201,20574"
4  }
5}

1{
2  "log": {
3    "remote_addr": "5.86.210.12",
4    "user_id": "zieme4647",
5    "timestamp": "19/06/2019:17:20:49 -0400",
6    "message": "GET /embrace/supply-chains/dynamic/vertical",
7    "status": 201,
8    "bytes": 20574
9  }
10}

Configuration Options

Required Options

field_names(required)

The field names assigned to the resulting tokens, in order.

Type	Syntax	Default	Example
array	literal		["timestamp","level","message","parent.child"]

inputs(required)

A list of upstream source or transform IDs. Wildcards (*) are supported.

See configuration for more info.

Type	Syntax	Default	Example
array	literal		["my-source-or-transform-id","prefix-*"]

type(required)

The component type. This is a required field for all components and tells Vector which component to use.

Type	Syntax	Default	Example
string	literal		["split"]

Advanced Options

drop_field(optional)

If true the field will be dropped after parsing.

Type	Syntax	Default	Example
bool

field(optional)

The field to apply the split on.

Type	Syntax	Default	Example
string	literal	message	["message","parent.child"]

separator(optional)

The separator to split the field on. If no separator is given, it will split on all whitespace. 'Whitespace' is defined according to the terms of the Unicode Derived Core Property White_Space.

Type	Syntax	Default	Example
string	literal	[whitespace]	[","]

timezone(optional)

The name of the time zone to apply to timestamp conversions that do not contain an explicit time zone. This overrides the global timezone option. The time zone name may be any name in the TZ database, or local to indicate system local time.

Type	Syntax	Default	Example
string	literal	local	["local","America/NewYork","EST5EDT"]

types(optional)

Key/value pairs representing mapped log field names and types. This is used to coerce log fields from strings into their proper types. The available types are listed in the Types list below.

Timestamp coercions need to be prefaced with timestamp|, for example "timestamp|%F". Timestamp specifiers can use either of the following:

One of the built-in-formats listed in the Timestamp Formats table below.
The time format specifiers from Rust's chrono library.

Types

array
bool
bytes
float
int
map
null
timestamp (see the table below for formats)

Timestamp Formats

Format	Description	Example
`%F %T`	`YYYY-MM-DD HH:MM:SS`	`2020-12-01 02:37:54`
`%v %T`	`DD-Mmm-YYYY HH:MM:SS`	`01-Dec-2020 02:37:54`
`%FT%T`	ISO 8601[RFC 3339](https://tools.ietf.org/html/rfc3339) format without time zone	`2020-12-01T02:37:54`
`%a, %d %b %Y %T`	RFC 822/2822 without time zone	`Tue, 01 Dec 2020 02:37:54`
`%a %d %b %T %Y`	`date` command output without time zone	`Tue 01 Dec 02:37:54 2020`
`%a %b %e %T %Y`	ctime format	`Tue Dec 1 02:37:54 2020`
`%s`	UNIX timestamp	`1606790274`
`%FT%TZ`	ISO 8601/RFC 3339 UTC	`2020-12-01T09:37:54Z`
`%+`	ISO 8601/RFC 3339 UTC with time zone	`2020-12-01T02:37:54-07:00`
`%a %d %b %T %Z %Y`	`date` command output with time zone	`Tue 01 Dec 02:37:54 PST 2020`
`%a %d %b %T %z %Y`	`date` command output with numeric time zone	`Tue 01 Dec 02:37:54 -0700 2020`
`%a %d %b %T %#z %Y`	`date` command output with numeric time zone (minutes can be missing or present)	`Tue 01 Dec 02:37:54 -07 2020`

Note: the examples in this table are for 54 seconds after 2:37 am on December 1st, 2020 in Pacific Standard Time.

Type	Syntax	Default	Example
hash			[{"status":"int","duration":"float","success":"bool","timestamp_iso8601":"timestamp\|%F","timestamp_custom":"timestamp\|%a %b %e %T %Y","timestamp_unix":"timestamp\|%F %T","parent":{"child":"int"}}]

How it Works

State

This component is stateless, meaning its behavior is consistent across each input.