Setting Up A kcat Config

If you're using Kafka, kcat is an essential tool to have in your toolbox. As the name suggests, it's cat for Kafka. It gives you fast, command-line access to simple tasks. Like, say you want to tail a topic called orders:

kcat -Ct orders

(-C is 'consume'. -t sets the topic name. This is unix, so you can collapse those flags into -Ct.)

Easy huh? Well, not quite. Because there's usually more to type than that. For a local Kafka instance you need to set the broker:

kcat -Ct orders -b localhost:9092

That's a pain. And if you want to access a remote machine it gets much worse:

kcat -Ct orders -b localhost:9092 -X security.protocol=sasl_ssl -X sasl.mechanism=PLAIN ...ugh...

But you don't have to do that. You can put it all in a kcat config file. It's a little bit of a tricky one to get right though, so for your reference and mine, here's how you set that up once and for all.

Example kcat Config Files

The file you need to edit is ~/.config/kcat.conf.

For a local instance, it should contain something like:

For a remote server, you'll need to include the authentication details, so your kcat.conf will look more like:<Bootstrap Server>
sasl.username=<API Key>
sasl.password=<API Secret>

With a kcat.conf correctly set up, all the connection information gets picked up every time you call kcat, so that first, simpler command kcat -Ct orders should Just Work.

Supporting Schema Registry

Update: After I wrote this up I found myself wishing I could do the same trick to connect to a Schema Registry. I'd like to be able decode some Avro values by simply saying:

kcat -Ct orders -s value=avro

Again, that's not too hard. You "just" need to cram the location of the schema registry, and your authentication details, in the right format under the right key. Here's the magic incantation to add to your ~/.config/kcat.conf:

schema.registry.url=https://<SR API Key>:<SR API Key>@<SR URL>

With that, kcat will attempt to fetch schemas from the registry whenever it needs a codec.