Reference

Table of Contents

Classes

Defined types

Classes

fluentbit

Installs and configures fluentbit

Examples

include fluentbit

Parameters

The following parameters are available in the fluentbit class.

manage_package_repo

Data type: Boolean

Installs the package repositories

Default value: true

create_defaults

Data type: Boolean

create some fluetbit example files (syslog, es, filers)

Default value: false

service_name

Data type: String

the td-agent-bit service name

Default value: 'td-agent-bit'

input_plugins

Data type: Hash

Hash of the INPUT plugins to be configured

Default value: $create_defaults

output_plugins

Data type: Hash

Hash of the OUTPUT plugins to be configured

Default value: $create_defaults

filters

Data type: Hash

Hash of the filter to be configured

Default value: $create_defaults

fluentbit::config

Creates a puppet resource for every input, ouptut config Doesn't support filters (yet) Includes all [input] and [output] configs. (@include) Sets global variables (@set) Configures global [service] section

Parameters

The following parameters are available in the fluentbit::config class.

flush

Data type: Integer

Set the flush time in seconds. Everytime it timeouts, the engine will flush the records to the output plugin.

Default value: 5

daemon

Data type: Enum['on', 'off']

Boolean value to set if Fluent Bit should run as a Daemon (background) or not. Allowed values are: yes, no, on and off.

Default value: off

log_file

Data type: Optional[String]

Absolute path for an optional log file.

Default value: undef

log_level

Data type: String

Set the logging verbosity level. Allowed values are: error, info, debug and trace. Values are accumulative, e.g: if 'debug' is set, it will include error, info and debug. Note that trace mode is only available if Fluent Bit was built with the WITH_TRACE option enabled.

Default value: info

parsers_file

Data type: Optional[String]

Path for a parsers configuration file. Multiple Parsers_File entries can be used.

Default value: undef

plugins_file

Data type: Optional[String]

Path for a plugins configuration file. A plugins configuration file allows to define paths for external plugins, for an example see here.

Default value: undef

streams_file

Data type: Optional[String]

Path for the Stream Processor configuration file.

Default value: undef

http_server

Data type: Enum['on', 'off']

Enable built-in HTTP Server

Default value: off

http_listen

Data type: String

Set listening interface for HTTP Server when it's enabled

Default value: '0.0.0.0'

http_port

Data type: String

Set TCP Port for the HTTP Server

Default value: '2020'

coro_stack_size

Data type: String

Set the coroutines stack size in bytes. The value must be greater than the page size of the running system.

Default value: '24576'

configfile

Data type: String

Path to the td-agent-bit config file.

Default value: '/etc/td-agent-bit/td-agent-bit.conf'

fluentbit::install

Installs fluentbit package

fluentbit::repo

configures the fluentbit repo

Parameters

The following parameters are available in the fluentbit::repo class.

ensure

Data type: Enum['present', 'absent']

Default value: 'present'

fluentbit::service

Manages the td-agent-bit service

Defined types

fluentbit::filter::modify

Filter to modify records using rules and conditions

Examples

fluentbit::filter::modify { 'namevar': }

Parameters

The following parameters are available in the fluentbit::filter::modify defined type.

configfile

Data type: String

Path to the filter configfile. Naming should be filter_*.conf to make sure it's getting included by the main config.

Default value: '/etc/td-agent-bit/filter_modify.conf'

set

Data type: Optional

Add a key/value pair with key KEY and value VALUE. If KEY already exists, this field is overwritten

Default value: undef

add

Data type: Optional

Add a key/value pair with key KEY and value VALUE if KEY does not exist

Default value: undef

remove

Data type: Optional

Remove a key/value pair with key KEY if it exists

Default value: undef

remove_wildcard

Data type: Optional

Remove all key/value pairs with key matching wildcard KEY

Default value: undef

remove_regex

Data type: Optional

Remove all key/value pairs with key matching regexp KEY

Default value: undef

rename

Data type: Optional

Rename a key/value pair with key KEY to RENAMED_KEY if KEY exists AND RENAMED_KEY does not exist

Default value: undef

hard_rename

Data type: Optional

Rename a key/value pair with key KEY to RENAMED_KEY if KEY exists. If RENAMED_KEY already exists, this field is overwritten

Default value: undef

copy

Data type: Optional

Copy a key/value pair with key KEY to COPIED_KEY if KEY exists AND COPIED_KEY does not exist

Default value: undef

hard_copy

Data type: Optional

Copy a key/value pair with key KEY to COPIED_KEY if KEY exists. If COPIED_KEY already exists, this field is overwritten

Default value: undef

match

Data type: String

Default value: '*'

fluentbit::input::forward

Forward is the protocol used by Fluent Bit and Fluentd to route messages between peers. This plugin implements the input service to listen for Forward messages.

Parameters

The following parameters are available in the fluentbit::input::forward defined type.

configfile

Data type: String

Path to the input configfile. Naming should be input_*.conf to make sure it's getting included by the main config.

Default value: '/etc/td-agent-bit/input_forward.conf'

listen

Data type: String

Listener network interface

Default value: '0.0.0.0'

port

Data type: Integer

TCP port to listen for incoming connections.

Default value: '24224'

buffer_max_size

Data type: Optional[String]

Specify the maximum buffer memory size used to receive a Forward message. The value must be according to the Unit Size specification.

Default value: undef

buffer_chunk_size

Data type: Optional[String]

By default the buffer to store the incoming Forward messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by Buffer_Chunk_Size. The value must be according to the Unit Size specification.

Default value: undef

fluentbit::input::syslog

Setups config file for fluentbit and configures listen on rsyslog. Reloads services if something changes.

Parameters

The following parameters are available in the fluentbit::input::syslog defined type.

mode

Data type: String

Defines transport protocol mode: unix_udp (UDP over Unix socket), unix_tcp (TCP over Unix socket), tcp or udp

Default value: 'unix_udp'

listen

Data type: Optional[String]

If Mode is set to tcp, specify the network interface to bind.

Default value: $mode

port

Data type: Optional[String]

If Mode is set to tcp, specify the TCP port to listen for incoming connections.

Default value: $mode

path

Data type: Optional[String]

If Mode is set to unix_tcp or unix_udp, set the absolute path to the Unix socket file.

Default value: $mode

parser

Data type: String

Specify an alternative parser for the message. By default, the plugin uses the parser syslog-rfc3164. If your syslog messages have fractional seconds set this Parser value to syslog-rfc5424 instead.

Default value: 'syslog-rfc3164'

rsyslog_config

Data type: String

Specify the path to the rsyslog config file for fluentbit to enable listening.

Default value: '/etc/rsyslog.d/60-fluent-bit.conf'

configfile

Data type: String

Path to the input configfile. Naming should be input_*.conf to make sure it's getting included by the main config.

Default value: '/etc/td-agent-bit/input_syslog.conf'

fluentbit::output::es

A description of what this class does

Examples

include fluentbit::output::es

Parameters

The following parameters are available in the fluentbit::output::es defined type.

configfile

Data type: String

Path to the output configfile. Naming should be output_*.conf to make sure it's getting included by the main config.

Default value: '/etc/td-agent-bit/output_es.conf'

match

Data type: String

Tag to route the output.

Default value: '*'

tls

Data type: Enum['on', 'off']

Turn TLS encrypted communication with Elasticsearch on / off.

Default value: 'off'

host

Data type: String

IP address or hostname of the target Elasticsearch instance

Default value: '127.0.0.1'

port

Data type: Integer

TCP port of the target Elasticsearch instance

Default value: 9200

path

Elasticsearch accepts new data on HTTP query path "/_bulk". But it is also possible to serve Elasticsearch behind a reverse proxy on a subpath. This option defines such path on the fluent-bit side. It simply adds a path prefix in the indexing HTTP POST URI.

buffer_size

Specify the buffer size used to read the response from the Elasticsearch HTTP service. This option is useful for debugging purposes where is required to read full responses, note that response size grows depending of the number of records inserted. To set an unlimited amount of memory set this value to False, otherwise the value must be according to the Unit Size specification.

pipeline

Newer versions of Elasticsearch allows to setup filters called pipelines. This option allows to define which pipeline the database should use. For performance reasons is strongly suggested to do parsing and filtering on Fluent Bit side, avoid pipelines.

http_user

Data type: Optional[String]

Optional username credential for Elastic X-Pack access

Default value: undef

http_passwd

Data type: Optional[String]

Password for user defined in HTTP_User

Default value: undef

index

Data type: String

Index name

Default value: 'fluentbit'

type

Data type: String

Type name

Default value: 'flb_type'

logstash_format

Enable Logstash format compatibility.

logstash_prefix

When Logstash_Format is enabled, the Index name is composed using a prefix and the date, e.g: If Logstash_Prefix is equals to 'mydata' your index will become 'mydata-YYYY.MM.DD'. The last string appended belongs to the date when the data is being generated.

logstash_dateformat

Time format (based on strftime) to generate the second part of the Index name.

time_key

When Logstash_Format is enabled, each record will get a new timestamp field. The Time_Key property defines the name of that field.

time_key_format

When Logstash_Format is enabled, this property defines the format of the timestamp.

include_tag_key

When enabled, it append the Tag name to the record.

tag_key

When Include_Tag_Key is enabled, this property defines the key name for the tag.

generate_id

When enabled, generate _id for outgoing records. This prevents duplicate records when retrying ES.

replace_dots

When enabled, replace field name dots with underscore, required by Elasticsearch 2.0-2.3.

trace_output

When enabled print the elasticsearch API calls to stdout (for diag only)

current_time_index

Use current time for index generation instead of message record

logstash_prefix_key

Prefix keys with this string