1 of 61

Kafbat UI

Overview

About

About Kafbat UI (UI for Apache Kafka)

Kafbat UI is a versatile, fast, lightweight, and flexible web interface designed to monitor and manage Apache Kafka® clusters. Created by developers for developers, it offers an intuitive way to gain visibility into your data flows, quickly identify and resolve issues, and maintain peak performance.

Kafbat UI is completely free and open source.

The streamlined dashboard provides clear insights into vital Kafka metrics—Brokers, Topics, Partitions, Production, and Consumption—making it easier to keep your systems running smoothly.

Developed by Kafbat, the team includes original contributors to the Apache Kafka UI project, continuing its legacy with a renewed vision. We're committed to evolving the project while staying true to its roots. Special thanks to Provectus for their foundational work, which remains a pillar of our progress and innovation.

Features

Topic Insights – View essential topic details including partition count, replication status, and custom configurations.
Configuration Wizard – Set up and configure your Kafka clusters directly through the UI.
Multi-Cluster Management – Monitor and manage all your Kafka clusters in one unified interface.
Metrics Dashboard – Track key Kafka metrics in real time with a streamlined, lightweight dashboard.
Kafka Brokers Overview – Inspect brokers, including partition assignments and controller status.
Consumer Group Details – Analyze parked offsets per partition, and monitor both combined and partition-specific lag.
Message Browser – Explore messages in JSON, plain text, or Avro encoding formats. Live view is supported, enriched with user-defined CEL message filters.
Dynamic Topic Management – Create and configure new topics with flexible, real-time settings.
Pluggable Authentication – Secure your UI using OAuth 2.0 (GitHub, GitLab, Google), LDAP, or basic authentication.
Cloud IAM Support – Integrate with GCP IAM, Azure IAM, and AWS IAM for cloud-native identity and access management.
Managed Kafka Service Support – Full support for Azure EventHub, Google Cloud Managed Service for Apache Kafka, and AWS Managed Streaming for Apache Kafka (MSK)—both server-based and serverless.
Custom SerDe Plugin Support – Use built-in serializers/deserializers like AWS Glue and Smile, or create your own custom plugins.
Role-Based Access Control – with RBAC.
Data Masking – in topic messages to enhance privacy and compliance.
MCP Server - Server

Project

Code of Conduct

Contributor Covenant Code of Conduct

Our Pledge

We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

Roadmap

Kafbat-UI Project Roadmap

Roadmap exists in a form of a GitHub project board and is located .

How to use this document

The roadmap provides a list of features we decided to prioritize in project development. It should serve as a reference point to understand projects' goals.

We do prioritize them based on the feedback from the community, our own vision, and other conditions and circumstances.

The roadmap sets the general way of development. The roadmap is mostly about long-term features. All the features could be re-prioritized, rescheduled, or canceled.

Development

Setting up git

Set your git credentials:

git config --global user.name "Mona Lisa"
git config --global user.email "[email protected]"

More info on setting git credentials:

Setting your username in Git

Setting your commit email address

Building

With Docker

Build & Run

Once you installed the prerequisites and cloned the repository, run the following steps in your project directory:

Step 1 : Build

Without Docker

Build & Run Without Docker

Once you installed the prerequisites and cloned the repository, run the following steps in your project directory:

Running Without Docker Quickly

Download the latest kafbat-ui jar file

Execute the jar

java -Dspring.config.additional-location=<path-to-application-local.yml> --add-opens java.rmi/javax.rmi.ssl=ALL-UNNAMED -jar <path-to-kafbat-ui-jar>

Example of how to configure clusters in the configuration file.

Building And Running Without Docker

NOTE: If you want to get kafbat-ui up and running locally quickly without building the jar file manually, then just follow Running Without Docker Quickly

Proceed with building instructions from [[with-docker]] and disable/remove build-docker-images flag.

Command to build the jar

Once your build is successful and the jar file named kafbat-ui-api-0.0.1-SNAPSHOT.jar is generated inside kafbat-ui-api/target.

Execute the jar

WIP: Testing

TODO :)

Quick Start

Prerequisites

Kafka Permissions

Standalone Kafka ACLs

ACLs required to run the app

ACLs for standalone kafka

This list is enough to run the app in r/o mode

 Permission |    Operation     | ResourceType | ResourceName  | PatternType
------------+------------------+--------------+---------------+--------------
 ALLOW      | READ             | TOPIC        | *             | LITERAL
 ALLOW      | DESCRIBE_CONFIGS | TOPIC        | *             | LITERAL
 ALLOW      | DESCRIBE         | GROUP        | *             | LITERAL
 ALLOW      | DESCRIBE         | CLUSTER      | kafka-cluster | LITERAL
 ALLOW      | DESCRIBE_CONFIGS | CLUSTER      | kafka-cluster | LITERAL

Demo run

Quick start (demo run)

Ensure you have docker installed
Ensure your kafka cluster is available from the machine you're planning to run the app on
Run the following:

docker run -it -p 8080:8080 -e DYNAMIC_CONFIG_ENABLED=true ghcr.io/kafbat/kafka-ui

Go to `` and configure your first cluster by pressing on "Configure new cluster" button.

When you're done with testing, you can refer to the next articles to persist your config & deploy the app wherever you need to.

Persisting config

Please ensure the target volume (~/kui/config.yml) of your config file does exist. If it doesn't, docker will create it as a directory instead of a file, and everything will explode.

Create a YAML file with the following contents:

Run the compose via:

K8s / Helm

To install the app via Helm please refer to .

Configuration

Configuration wizard

Dynamic application configuration

By default, kafbat-ui does not allow to change of its configuration in runtime. When the application is started it reads configuration from system env, config files (ex. application.yaml), and JVM arguments (set by -D). Once the configuration was read it was treated as immutable and won't be refreshed even if the config source (ex. file) was changed.

Since version 0.6 we added an ability to change cluster configs in runtime. This option is disabled by default and should be implicitly enabled. To enable it, you should set DYNAMIC_CONFIG_ENABLED env property to true or add dynamic.config.enabled: true property to your yaml config file.

Sample docker compose configuration:

You can even omit all vars other than DYNAMIC_CONFIG_ENABLED to start the application with empty configs and setup it up after startup.

When the dynamic config feature is enabled you will see additional buttons that will take you to "Wizard" for editing existing cluster configuration or adding new clusters:

Dynamic config files

Kafbat-UI is a stateless application by its nature, so, when you edit configuration during runtime, it will store configuration additions on the container's filesystem (in dynamic_config.yaml file). Dynamic config file will be overridden on each configuration submission.

During the configuration process, you can also upload configuration-related files (like truststore and keystores). They will be stored in etc/kafkaui/uploads a folder with a unique timestamp suffix to prevent name collision. In the wizard, you can also use files that were mounted to the container's filesystem, without uploading them directly.

Note, that if the container is recreated, your edited (and uploaded) files won't be present and the app will be started with static configuration only. If you want to be able to keep the configuration created by wizard, you have to mount/copy the same files into newly created kafbat-ui containers (whole /etc/kafkaui/ folder, by default).

Properties, specified where dynamic config files will be persisted:

Env variable name

Yaml property

Default

Description

Implementation notes:

Currently, the new configuration submission leads to a full application restart. So, if your kafbat-ui app is starting slow (not a usual case, but may happen when you have a slow connection to kafka clusters) you may notice UI inaccessibility during restart time.

Setup example configs

A list of ready-to-go docker compose files for various setup scenarios

Check our basic config examples: DOCKER_COMPOSE.MD

And our repo with more advanced examples: https://github.com/kafbat/ui-config-examples

Helm charts

Kafbat UI is also available as a helm chart. See the underlying articles to learn more about it.

Quick start

Quick Start with Helm Chart

General

Clone/Copy Chart to your working directory
Execute command

Configuration

Kafbat UI parameters

Please find specific parameters description in helm chart

Setup

SSL example

Implement ssl for kafbat-ui

To implement SSL for kafbat-ui you need to provide JKS files into the pod. Here is the instruction on how to do that.

Sticky sessions

if you're running more than one pod and have authentication enabled you will encounter issues with sessions, as we store them in cookies and other instances are not aware of your sessions.

The solution for this would be using sticky session/session affinity.

An example:

Complex configuration examples

Kafka secured with SSL

Connecting to a Secure Broker

The app supports TLS (SSL) and SASL connections for .

Running From Docker-compose file

Authentication

For the UI

Authentication methods for authenticating your users into the UI itself

Basic Authentication

Basic username+password authentication

Basic authentication supports only one local user. If you need multiple, running a Keycloak instance is the easiest way.

To enable basic username+password authentication, add these properties:

YAML config counterpart:

Please note that basic auth is incompatible with any other auth method or RBAC.

LDAP / Active Directory

If you're running AD rather than LDAP, also add these:

For Kafka

List of authentication methods to kafka itself

AWS IAM

How to configure AWS IAM Authentication

Kafbat UI comes with a built-in library.

You could pass SASL configs in the properties section for each cluster.

More details could be found here:

More about permissions:

Authentication Options

SASL_SCRAM

How to configure SASL SCRAM Authentication

You could pass sasl configs in properties section for each cluster.

Examples:

Please replace

<KAFKA_NAME> with cluster name
<KAFKA_URL> with broker list
<KAFKA_USERNAME> with username
<KAFKA_PASSWORD> with password

Running From Docker Image

Running From Docker-compose file

Configuring by application.yaml

Audit log

Kafbat-UI allows you to log all operations to your kafka clusters done within kafbat-ui itself.

Logging can be done to either kafka topic and/or console.

See all the available configuration properties:

Note

If you create the audit topic manually, it should not be compacted, as the audit documents do not have a key, which is required for Kafka topic compaction.

Other than compaction, the Kafka UI does not expect specific requirements for additional topic properties. However, as always, consider configuring the following:

Serialization / SerDe

Serialization, deserialization and custom plugins

Kafka-ui supports multiple ways to serialize/deserialize data.

Setting serdes for specific topics

You can specify a preferred serde for the topics key/value. This serde will be chosen by default in UI on topic's view/produce pages. To do so, set topicValuesPattern/topicValuesPattern properties for the selected serde. Kafka-ui will choose the first serde that matches the specified pattern.

Pluggable SerDes

This page contains links for non-built-in SerDes.

Also see

Official non-built-in SerDes

These SerDes are developed and maintained by kafbat, providing the original quality and support.

Building your own pluggable SerDe

Custom pluggable serde registration

You can implement your serde and register it in the UI. To do so:

Add serde-api dependency,

OpenDataDiscovery Integration

Kafka-ui has integration with the (ODD).

ODD Platform allows you to monitor and navigate kafka data streams and see how they embed into your data platform.

This integration allows you to use kafbat-ui as an ODD "Collector" for kafka clusters.

Currently, kafbat-ui exports:

kafka topics as ODD Datasets with topic's metadata, configs, and schemas

FAQ

Common problems

If you are running against confluent cloud and you have specified correctly the jass config and still continue getting these errors look to see if you are passing confluent. the license in the connector, the absence of a license returns a number of bogus errors like "Login module control flag not specified in JAAS config".

https://docs.confluent.io/platform/current/connect/license.html

A good resource for what properties are needed is here: https://gist.github.com/rmoff/49526672990f1b4f7935b62609f6f567

MCP Server

To enable the MCP server functionality, set the mcp.enabled flag to true.

Once enabled, you can connect to the Kafbat UI instance using an MCP client at the following SSE endpoint:http://hostname/mcp/sse.

Visual Studio Code

To configure MCP in VS Code:

FAQ

Basic (username password) authentication

Role-based access control

Authentication Issues

LDAP authentication doesn't work

Add the following env/config property and check your logs upon authentication:

LOGGING_LEVEL_ORG_SPRINGFRAMEWORK_SECURITY_LDAP_USERDETAILS: "TRACE"

Contributing

This guide aims to walk you through the process of working on issues and Pull Requests (PRs).

Bear in mind that you will not be able to complete some steps on your own if you do not have “write” permission. Feel free to reach out to the maintainers to help you unlock these activities.

General recommendations

Please note that we have a code of conduct. Make sure that you follow it in all of your interactions with the project.

Issues

Choosing an issue

There are two options to look for the issues to contribute to. The first is our board. There the issues are sorted by the required experience level (beginner, intermediate, expert).

The second option is to search for -labeled issues. Some of them might not be displayed on the aforementioned board or vice versa.

You also need to consider labels. You can sort the issues by scope labels, such as scope/backend, scope/frontend or even scope/k8s. If any issue covers several specific areas, and you do not have the required expertise for one of them, just do your part of the work — others will do the rest.

Grabbing the issue

There is a bunch of criteria that make an issue feasible for development. The implementation of any features and/or their enhancements should be reasonable and must be backed by justified requirements (demanded by the community, roadmap plans, etc.). The final decision is left to the maintainers' discretion.

All bugs should be confirmed as such (i.e. the behavior is unintended).

Any issue should be properly triaged by the maintainers beforehand, which includes:

Having a proper milestone set
Having required labels assigned: "accepted" label, scope labels, etc.

Formally, if these triage conditions are met, you can start to work on the issue.

With all these requirements met, feel free to pick the issue you want. Reach out to the maintainers if you have any questions.

Working on the issue

Every issue “in progress” needs to be assigned to a corresponding person. To keep the status of the issue clear to everyone, please keep the card's status updated ("project" card to the right of the issue should match the milestone’s name).

Setting up a local development environment

Please refer to this guide.

Pull Requests

Branch naming

In order to keep branch names uniform and easy to understand, please use the following conventions for branch naming.

Generally speaking, it is a good idea to add a group/type prefix to a branch; e.g., if you are working on a specific branch, you could name it issues/xxx.

Here is a list of good examples: issues/123 feature/feature_name bugfix/fix_thing\

Code style

Java: There is a file called checkstyle.xml in project root under etc directory. You can import it into IntelliJ IDEA via Checkstyle plugin.

Naming conventions

REST paths should be written in lowercase and consist of plural nouns only. Also, multiple words that are placed in a single path segment should be divided by a hyphen (-).\

Query variable names should be formatted in camelCase.

Model names should consist of plural nouns only and should be formatted in camelCase as well.

Creating a PR

When creating a PR please do the following:

In commit messages use these .
Link an issue(-s) via "linked issues" block.
Set the PR labels. Ensure that you set only the same set of labels that is present in the issue, and ignore yellow status/ labels.

Pull Request checklist

When composing a build, ensure that any install or build dependencies have been removed before the end of the layer.
Update the README.md with the details of changes made to the interface. This includes new environment variables, exposed ports, useful file locations, and container parameters.

Reviewing a PR

WIP

Pull Request reviewer checklist

WIP

AWS Marketplace

How to Deploy Kafka UI from AWS Marketplace

Step 1: Go to AWS Marketplace

Go to the AWS Marketplace website and sign in to your account.

Step 2: Find Kafbat UI

Either use the search bar to find "Kafbat UI" or go to .

Click "Continue to Subscribe" and accept the terms and conditions. Click "Continue to Configuration".

Step 4: Choose the Software Version and Region

Choose your desired software version and region. Click "Continue to Launch".

Step 5: Launch the Instance

Choose "Launch from Website" and select your desired EC2 instance type. You can choose a free tier instance or choose a larger instance depending on your needs. We recommend having at least 512 RAM for an instant.

Next, select the VPC and subnet where you want the instance to be launched. If you don't have an existing VPC or subnet, you can create one by clicking "Create New VPC" or "Create New Subnet".

Choose your security group. A security group acts as a virtual firewall that controls traffic to and from your instance. If you don't have an existing security group, you can create a new one based on the seller settings by clicking "Create New Based on Seller Settings".

Give your security group a name and description. The seller settings will automatically populate the inbound and outbound rules for the security group based on best practices. You can review and modify the rules if necessary.

Click "Save" to create your new security group.

Select your key pair or create a new one. A key pair is used to securely connect to your instance via SSH. If you choose to create a new key pair, give it a name and click "Create". Your private key will be downloaded to your computer, so make sure to keep it in a safe place.

Finally, click "Launch" to deploy your instance. AWS will create the instance and install the Kafka UI software for you.

Step 6: Check EC2 Status

To check the EC2 state please click on "EC2 console".

Step 7: Access the Kafka UI

After the instance is launched, you can check its status on the EC2 dashboard. Once it's running, you can access the Kafka UI by copying the public DNS name or IP address provided by AWS and adding after the IP address or DNS name port 8080. Example: ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com:8080

Step 8: Configure Kafka UI to Communicate with Brokers

If your broker is deployed in AWS then allow incoming from Kafbat UI EC2 by adding an ingress rule in the security group which is used for a broker. If your broker is not in AWS then be sure that your broker can handle requests from Kafka-ui EC2 IP address.

More about permissions:

That's it! You've successfully deployed the Kafka UI from AWS Marketplace.

---
version: '2'
services:
  kafbat-ui:
    container_name: kafbat-ui
    image: ghcr.io/kafbat/kafka-ui
    ports:
      - 8080:8080
    depends_on:
      - kafka0
      - kafka1
      - kafka2
      - schema-registry0
      - kafka-connect0
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka0:29092,kafka1:29092,kafka2:29092
      KAFKA_CLUSTERS_0_METRICS_PORT: 9997
      KAFKA_CLUSTERS_0_SCHEMAREGISTRY: http://schema-registry0:8085
      KAFKA_CLUSTERS_0_KAFKACONNECT_0_NAME: first
      KAFKA_CLUSTERS_0_KAFKACONNECT_0_ADDRESS: http://kafka-connect0:8083

  kafka0:
    image: confluentinc/cp-kafka:7.2.1
    hostname: kafka0
    container_name: kafka0
    ports:
      - 9092:9092
      - 9997:9997
    environment:
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka0:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_CLUSTER_ID:
      KAFKA_NODE_ID: 1
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka0:29093,2@kafka1:29093,3@kafka2:29093'
      KAFKA_LISTENERS: 'PLAINTEXT://kafka0:29092,CONTROLLER://kafka0:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
      KAFKA_JMX_PORT: 9997
      KAFKA_JMX_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=kafka0 -Dcom.sun.management.jmxremote.rmi.port=9997
    volumes:
      - ./scripts/update_run_cluster.sh:/tmp/update_run.sh
      - ./scripts/clusterID:/tmp/clusterID
    command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"

  kafka1:
    image: confluentinc/cp-kafka:7.2.1
    hostname: kafka1
    container_name: kafka1
    environment:
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_NODE_ID: 2
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka0:29093,2@kafka1:29093,3@kafka2:29093'
      KAFKA_LISTENERS: 'PLAINTEXT://kafka1:29092,CONTROLLER://kafka1:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
      KAFKA_JMX_PORT: 9997
      KAFKA_JMX_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=kafka1 -Dcom.sun.management.jmxremote.rmi.port=9997
    volumes:
      - ./scripts/update_run_cluster.sh:/tmp/update_run.sh
      - ./scripts/clusterID:/tmp/clusterID
    command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"

  kafka2:
    image: confluentinc/cp-kafka:7.2.1
    hostname: kafka2
    container_name: kafka2
    environment:
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_NODE_ID: 3
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka0:29093,2@kafka1:29093,3@kafka2:29093'
      KAFKA_LISTENERS: 'PLAINTEXT://kafka2:29092,CONTROLLER://kafka2:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
      KAFKA_JMX_PORT: 9997
      KAFKA_JMX_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=kafka1 -Dcom.sun.management.jmxremote.rmi.port=9997
    volumes:
      - ./scripts/update_run_cluster.sh:/tmp/update_run.sh
      - ./scripts/clusterID:/tmp/clusterID
    command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"


  schema-registry0:
    image: confluentinc/cp-schema-registry:7.2.1
    ports:
      - 8085:8085
    depends_on:
      - kafka0
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: PLAINTEXT://kafka0:29092
      SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL: PLAINTEXT
      SCHEMA_REGISTRY_HOST_NAME: schema-registry0
      SCHEMA_REGISTRY_LISTENERS: http://schema-registry0:8085

      SCHEMA_REGISTRY_SCHEMA_REGISTRY_INTER_INSTANCE_PROTOCOL: "http"
      SCHEMA_REGISTRY_LOG4J_ROOT_LOGLEVEL: INFO
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC: _schemas

  kafka-connect0:
    image: confluentinc/cp-kafka-connect:7.2.1
    ports:
      - 8083:8083
    depends_on:
      - kafka0
      - schema-registry0
    environment:
      CONNECT_BOOTSTRAP_SERVERS: kafka0:29092
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: _connect_configs
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_STORAGE_TOPIC: _connect_offset
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_TOPIC: _connect_status
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry0:8085
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry0:8085
      CONNECT_INTERNAL_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_REST_ADVERTISED_HOST_NAME: kafka-connect0
      CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components,/usr/share/filestream-connectors,/tmp/kfk"
    volumes:
      - /tmp/kfk:/tmp/kfk:ro
      - /tmp/kfk/test.txt:/tmp/kfk/test.txt

  kafka-init-topics:
    image: confluentinc/cp-kafka:7.2.1
    volumes:
      - ./message.json:/data/message.json
    depends_on:
      - kafka0
    command: "bash -c 'echo Waiting for Kafka to be ready... && \
               cub kafka-ready -b kafka0:29092 1 30 && \
               kafka-topics --create --topic second.users --partitions 3 --replication-factor 1 --if-not-exists --bootstrap-server kafka0:29092 && \
               kafka-topics --create --topic second.messages --partitions 2 --replication-factor 1 --if-not-exists --bootstrap-server kafka0:29092 && \
               kafka-topics --create --topic first.messages --partitions 2 --replication-factor 1 --if-not-exists --bootstrap-server kafka0:29092 && \
               kafka-console-producer --bootstrap-server kafka0:29092 --topic second.users < /data/message.json'"

# ======================================================
# Kafka Clusters Configuration
# Define all Kafka cluster-specific settings and related integrations.
# ======================================================
kafka:
  clusters:
    - name: local  # Unique name identifier for the Kafka cluster
      bootstrap-servers: kafka1:9092,kafka2:9092  # List of Kafka broker addresses

      # SSL configuration for secure connection to Kafka brokers
      ssl:
        truststore-location: path/to/truststore/file.jks  # Path to truststore JKS file
        truststore-password: password  # Password to access the truststore
        verify-ssl: true  # Enable SSL certificate verification

      # Schema Registry connection and authentication details
      schemaRegistry: http://schema-registry:8085
      schema-registry-auth:
        username: schema registry username
        password: schema registry password
      schema-registry-ssl:
        keystore-location: path/to/keystore/file.jks
        keystore-password: password

      # ksqlDB connection and security settings
      ksqldb-server: http://ksqldb-host:8088
      ksqldb-server-auth:
        username: ksqldb-username
        password: ksqdb-passsword
      ksqldb-server-ssl:
        keystore-location: path/to/keystore/file.jks
        keystore-password: password

      # Kafka Connect REST endpoint and optional authentication
      kafka-connect:
        - name: first  # Identifier for the Connect cluster
          address: http://kafka-connect-host:8083
          username: Auth username
          password: Auth password
          keystore-location: path/to/keystore/file.jks
          keystore-password: keystore password

      # Custom SerDe (Serializer/Deserializer) for interpreting topic data
      serde:
        - name: CustomeHexWithEditedDelimiter
          class-name: io.kafbat.ui.serdes.builtin.HexSerde
          file-path: /var/lib/kui-serde/my-kui-serde.jar
          topic-keys-pattern: ".*-events"  # Regex to match applicable topic keys
          topic-values-pattern: ".*-events"  # Regex to match applicable topic values
          properties:
            uppercase: "false"  # Output lowercase hex
            delimiter: ":"  # Byte delimiter

      default-key-serde: String  # Default SerDe for message keys
      default-value-serde: SchemaRegistry  # Default SerDe for message values

      # Monitoring and metrics collection from the cluster
      metrics:
        type: JMX  # Available options: JMX or PROMETHEUS
        port: 9997  # Port to collect JMX metrics
        username: Auth username
        password: Auth password
        ssl: false  # Whether to use SSL for metrics endpoint
        keystore-location: path/to/keystore/file.jks
        keystore-password: password

      # Global Kafka client properties
      properties:
        "security.protocol": SASL_SSL
        "sasl.mechanism": SCRAM-SHA-512
        "sasl.jaas.config": org.apache.kafka.common.security.scram.ScramLoginModule required username="your-username" password="your-password";

      # Kafka Consumer configuration overrides
      consumer-properties:
        "max.partition.fetch.bytes": 1048576

      # Kafka Producer configuration overrides
      producer-properties:
        "enable.idempotence": false

      read-only: true  # Prevent write operations from the UI

      polling-throttle-rate: 0  # Delay (seconds) between polling intervals in UI; 0 = no throttling

      # Data masking rules for messages shown in the UI
      masking:
        - type: MASK  # Action: MASK, REMOVE, or REPLACE
          fields: [ "id", "name" ]  # Specific fields to apply masking to
          fields-name-pattern: "id.*"  # Regex pattern to match field names
          masking-chars-replacement: ["A", "a", "N", "_"]  # Optional: override default mask characters
          replacement: "***MASKED_FIELD_NAME***"  # Replacement string for masked field names
          topic-values-pattern: events-with-ids-.*  # Apply on topic values matching this pattern
          topic-keys-pattern: events-with-ids-.*  # Apply on topic keys matching this pattern

      # Audit logging configuration
      audit:
        topic-audit-enabled: false  # Enable/disable Kafka topic audit logs
        console-audit-enabled: false  # Enable/disable console audit output
        topic: audit-topic-name  # Kafka topic for audit entries
        audit-topics-partitions: 3  # Number of partitions for the audit topic
        level: ALL  # ALL = log all actions, ALTER_ONLY = only mutating actions
        audit-topic-properties:
          "retention.ms": 43200000  # Audit log retention in ms (12 hours)

  # Prefix to identify internal Kafka topics used by system tools
  internalTopicPrefix: "__"

  # Timeout for admin Kafka operations (in milliseconds)
  admin-client-timeout: 30000

  # Polling behavior for consumer previews in the UI
  polling:
    poll-timeout-ms: 1000  # How long to wait for new records (ms)
    max-page-size: 500  # Maximum number of records per poll
    default-page-size: 100  # Default number of records per poll

# ======================================================
# Model Context Protocol config
# ======================================================
mcp:
  enabled: true # Enable MCP (ModelContextProtocol) endpoint

# ======================================================
# OAuth2 Authentication Configuration
# Used for authenticating users via external identity providers.
# ======================================================
auth:
  oauth2:
    client:
      client_name:
        client-id: xxx
        client-secret: yyy
        scope: openid
        client-name: cognito  # Display name on the login screen
        provider: <provider>  # Path alias used in redirect URI
        redirect-uri: http://localhost:8080/login/oauth2/code/<provider>
        authorization-grant-type: authorization_code
        issuer-uri: https://xxx  # Identity provider URL
        jwk-set-uri: https://yyy/.well-known/jwks.json  # Public key set URL
        user-name-attribute: <zzz>  # JWT field used as the user's name
        custom-params:
          type: <provider_type>  # Optional RBAC integration type (e.g. cognito, google, github, oauth)
          roles-field: groups  # Field containing user roles/groups

    resource-server:
      jwt:
        jwk-set-uri: http://jwk/uri
        jws-algorithms: ["RS256"]
        issuer-uri: http://issuer/uri
        public-key-location:  # Optional path to JWT public key
        audiences: []  # List of accepted audience values in JWT
        authority_prefix:  # Optional prefix added to authorities
        authorities-claim-delimiter:  # Regex delimiter for splitting roles
        authorities-claim-name:  # Claim name holding roles/permissions
        principal-claim-name:  # Claim used for determining the username

      opaque-token:
        client-id:  # Client ID used for introspection
        client-secret:  # Secret used for introspection
        introspection-uri:  # URL to the introspection endpoint

# ======================================================
# LDAP Configuration
# Used for user and group resolution in RBAC when using LDAP.
# ======================================================
spring:
  ldap:
    urls: ldap://localhost:10389
    base: "cn={0},ou=people,dc=planetexpress,dc=com"
    admin-user: "cn=admin,dc=planetexpress,dc=com"
    admin-password: "GoodNewsEveryone"
    user-filter-search-base: "dc=planetexpress,dc=com"
    user-filter-search-filter: "(&(uid={0})(objectClass=inetOrgPerson))"
    group-filter-search-base: "ou=people,dc=planetexpress,dc=com"

# ======================================================
# Role-Based Access Control (RBAC)
# Configure roles, user/group bindings, and permissions.
# ======================================================
rbac:
  roles:
    - name: "memelords"  # Role name
      clusters:
        - local  # Clusters this role applies to
      subjects:
        - provider: oauth_google  # e.g. oauth_{client_name}, ldap
          type: domain  # domain, organization, user, group
          value: "kafbat.dev"
          regex: false
      permissions:
        - resource: applicationconfig  # Resource types: applicationconfig, clusterconfig, topic, consumer, schema, connect, ksql, acl, audit
          value: ".*"
          actions: all  # Allowed actions: read, write, all

# ======================================================
# WebClient Settings
# Configuration for HTTP clients used by the app.
# ======================================================
webclient:
  response-timeout-ms: 20000  # Timeout for all outgoing HTTP requests
  max-in-memory-buffer-size: 20MB  # Maximum buffer size for handling responses

# ======================================================
# Spring Boot Actuator
# Enables health, info, and Prometheus endpoints for monitoring.
# ======================================================
management:
  endpoint:
    info:
      enabled: true
    health:
      enabled: true
  endpoints:
    web:
      exposure:
        include: "info,health,prometheus"

# ======================================================
# Application Logging Configuration
# Configure logging levels for specific packages.
# ======================================================
logging:
  level:
    root: INFO
    io.kafbat.ui: DEBUG
    reactor.netty.http.server.AccessLog: INFO
    org.hibernate.validator: WARN

RBAC (Role based access control)

Role-based access control

In this article, we'll guide how to set up Kafbat-UI with role-based access control.

Authentication methods

First of all, you'd need to set up authentication method(s). Refer to article for setup.

Config placement

First of all, you have to decide if either:

You wish to store all roles in a separate config file
Or within a main config file

This is how you include one more file to start with a docker-compose example:

Alternatively, you can append the roles file contents to your main config file.

Roles file structure

Clusters

In the roles file we define roles, duh. Every role has access to defined clusters:

Subjects

A role also has a list of subjects which are the entities we will use to assign roles to. They are provider-dependant, in general, they can be users, groups, or some other entities (github orgs, google domains, LDAP queries, etc.) In this example we define a role memelords that will contain all the users within the Google domain memelord.lol and, additionally, a GitHub user Haarolean. You can combine as many subjects as you want within a role.

The subject value is either a fixed string or a regular expression identifying a subject. To use regular expression, you must set regex: true. Regular expression works for any combination of provider and type. In this example, we define a role admins that will contain all the users with an oauth role ending with -ADMIN.

Providers

A list of supported providers and corresponding subject fetch mechanism:

oauth: user, role
oauth_google: user, domain
oauth_github: user, organization

Permissions

The next thing which is present in your roles file is, surprisingly, permissions. They consist of:

Resource Can be one of the: CLUSTERCONFIG, TOPIC, CONSUMER, SCHEMA, CONNECT, KSQL, ACL.
The resource value is either a fixed string or a regular expression identifying a resource. Value is not applicable to clusterconfig

Actions

A list of all the actions for the corresponding resources (please note neither resource nor action names are case-sensitive):

applicationconfig: view, edit
clusterconfig: view, edit

Example file

A complete file example:

A read-only setup:

An admin-group setup example:

rbac: roles: - name: "memelords" clusters: - local - dev - staging - prod subjects: - provider: oauth_google type: domain value: "memelord.lol" - provider: oauth_google type: user value: "[email protected]" - provider: oauth_github type: organization value: "memelords_org" - provider: oauth_github type: team value: "memelords_org/memelords_team" - provider: oauth_github type: user value: "memelord" - provider: oauth_cognito type: user value: "username" - provider: oauth_cognito type: group value: "memelords" # AzureAD with OAUTH2 - provider: role type: user value: "[email protected]" - provider: oauth type: role value: "admin" - provider: ldap type: user value: "username" - provider: ldap type: group value: "admin_staff" - provider: ldap_ad type: user value: "username" - provider: ldap_ad type: group value: "admin_staff" permissions: - resource: applicationconfig # value not applicable for applicationconfig actions: [ "view", "edit" ] # can be with or without quotes - resource: clusterconfig # value not applicable for clusterconfig actions: [ "view", "edit" ] - resource: topic value: "ololo.*" actions: # can be a multiline list - VIEW # can be upper or lowercase - CREATE - EDIT - DELETE - MESSAGES_READ - MESSAGES_PRODUCE - MESSAGES_DELETE - ANALYSIS_RUN - ANALYSIS_VIEW - resource: consumer value: "\_confluent-ksql.*" actions: [ VIEW, DELETE, RESET_OFFSETS ] - resource: schema value: "blah.*" actions: [ VIEW, CREATE, DELETE, EDIT, MODIFY_GLOBAL_COMPATIBILITY ] - resource: connect value: "local" actions: [ view, edit, create, delete, operate, reset_offsets ] # connectors selector not implemented yet, use connects # selector: # connector: # name: ".*" # class: 'io.kafbat.connectorName' - resource: ksql # value not applicable for ksql actions: [ execute ] - resource: acl # value not applicable for acl actions: [ view, edit ]