Skip to main content

Filebeat

Ingesting logs into Siglens using Filebeat

1. Install Filebeat​

Debian and Ubuntu

Install Filebeat on Debian and Ubuntu:

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.9.3-amd64.deb
sudo dpkg -i filebeat-oss-7.9.3-amd64.deb
CentOS, Redhat, and Amazon Linux

Install Filebeat on CentOS, Redhat, and Amazon Linux:

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.9.3-x86_64.rpm
sudo rpm -ivh filebeat-oss-7.9.3-x86_64.rpm

2. Configure Filebeat​

Download the sample events file using the following command:

curl -s -L https://github.com/siglens/pub-datasets/releases/download/v1.0.0/2kevents.json.tar.gz -o 2kevents.json.tar.gz && tar -xvf 2kevents.json.tar.gz

Create a config file:

filebeat.yml
filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /Users/username/logstash/2kevents.json # Path to the log file
    json.keys_under_root: true
    json.add_error_key: true
    processors:
      - drop_event: # Drop events missing first_name
          when:
            not:
              has_fields: ['first_name']

output.elasticsearch:
  hosts: ['http://localhost:8081/elastic/']
  index: 'filebeat-ind-0'

setup.template.enabled: false
setup.ilm.enabled: false

For more information on customizing your filebeat.yml file according to your logs, refer to the Filebeat documentation

3. Run Filebeat​

sudo ./filebeat -e -c $(pwd)/filebeat.yml

Navigate to the directory where Filebeat is installed and run the above command, make sure to set the correct path to the config file.