How to Send Logs to Loki Using Fluent Bit

In this blog, we help you guide how to send logs into Loki using Fluent Bit and its official Loki output plugin. Fluent Bit is a lightweight log collector, processor, and forwarder, capable of ingesting logs from applications and delivering them to destinations like Loki. We’ll use the “forward” input plugin to receive logs and then configure Fluent Bit to send them into Loki.

Dependencies

Before starting, make sure you have:

• Docker
• Docker Compose
  These tools are required to bring up the demo environment.

$dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
$dnf install docker-ce docker-ce-cli containerd.io
$systemctl start docker
$systemctl enable docker
$docker --version
$curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$chmod +x /usr/local/bin/docker-compose
$docker-compose --version

Scenario

For more understanding, we use a sample application (the “Carnivorous Greenhouse”) with micro-services like a user service, plant service, simulation service, websocket service, bug service (that triggers failures randomly), main app and database. All of these services are instrumented with Fluent Bit so they send logs via the Fluent Bit forward protocol. 

Step 1: Environment Setup

First, clone the repository that holds our demo application and observability stack:

git clone -b fluentbit-official  https://github.com/grafana/loki-fundamentals.git

Next, spin up the stack using Docker Compose:

docker compose -f loki-fundamentals/docker-compose.yml up -d

Running this will bring up containers including Grafana, Loki and Fluent Bit. You can then go to http://localhost:3000 to check Grafana is up. 

Step 2: Configure Fluent Bit to Send Logs to Loki

Open the fluent-bit.conf file in the loki-fundamentals directory; this file is where we define inputs, filters, and outputs for Fluent Bit. 

Receiving logs via the forward plugin

Add the following snippet into fluent-bit.conf to allow Fluent Bit to listen for incoming logs:

[INPUT]
    Name             forward
    Listen            0.0.0.0
    Port              24224

Here, we’re using the forward input plugin, listening on all IPs (0.0.0.0) at port 24224.

Exporting logs to Loki using the Loki output plugin

Then add this block to send matching logs to Loki:

[OUTPUT]
    name   loki
    match  service.**
    host   loki
    port   3100
    labels agent=fluent-bit
    label_map_path /fluent-bit/etc/conf/logmap.json

Let’s unpack it:

• name: Specifies the output plugin (here “loki”)
  
• match: Uses the tag pattern service. ** to select logs
  
• host & port: Direct logs to the Loki service (host = loki, port = 3100)
  
• labels: Adds an extra label (agent=fluent-bit) to each log entry
  
• label_map_path: Provides a mapping file (logmap.json) specifying how log fields map to Loki labels

 And the logmap.json file looks like this:

{
  "service": "service_name",
  "instance_id": "instance_id"
}

This points the log field service to the label service_name, and instance_id to instance_id.

 After updating the configuration, restart the Fluent Bit container:

docker restart loki-fundamentals-fluent-bit-1

Then check its logs to confirm the configuration loaded properly:

docker logs loki-fundamentals-fluent-bit-1

Step 3: Start the Application  

Now launch the Carnivorous Greenhouse demo application:

docker compose -f loki-fundamentals/greenhouse/docker-compose-micro.yml up -d --build

This sets up all microservices (db, websocket, bug service, user service, plant service, simulation service, main app). You can access the app at http://localhost:5005. Then perform actions (create user, log in, create plants, enable bug mode) so logs get generated. Finally, head to Loki’s Explore view in Grafana: http://localhost:3000/a/grafana-lokiexplore-app/explore to view your logs.

Conclusion

In this blog we demonstrated how to send logs from Fluent Bit into Loki. We configured Fluent Bit to receive logs from the microservices application, set up the Loki output plugin, including labels and mapping, and then launched the application to generate and view logs via Grafana.

If you’re looking to streamline your logging and monitoring setup or need help integrating tools like Fluent Bit and Loki into your infrastructure, the experts at Skynats are here to assist. Our DevOps Support Services cover everything from configuration and automation to full-scale observability solutions, ensuring your systems run smoothly and efficiently.