AVI networks proxy for Kafka Streams on Openshift

Today I want to show you a real scenario that I configured a few years ago, I was using Strimzi 0.15.0 and Kafka Steams 2.2.x. which could be something you will be facing as well.

In this scenario, I am using Openshift 4.x (community version known as OKD.oi) and deploying Kafka using the Strimzi operator.

For this use case, all traffic goes through AVI, therefore it’s necessary to configure the AMQ Stream (Kafka) cluster for the external router via AVI proxy.

Problem

After your RH AMQ Stream cluster has deployed the cluster operator uses Strimizi with the AVI annotation as you can see below.

template:
  externalBootstrapRoute:
    metadata:
      annotations:
        avi_proxy: '{"dedicated_route": true}'
        openshift.io/host.generated: 'true'
  externalBootstrapService:
    metadata:
      annotations:
        avi_proxy: >-
          {"virtualservice":{"services": [{"port": 443,
          "enable_ssl":true}],"auto_allocate_ip":true,"east_west_placement":false}}
  perPodRoute:
    metadata:
      annotations:
        avi_proxy: '{"dedicated_route": true}'
        openshift.io/host.generated: 'true'

Then AVI will configure the service route using the service port 9094 by default.

When you try to send messages using the consumer like this command below for testing propose:

./kafka-console-producer.sh --broker-list <external bootstap route hostname>:443 --topic <your topic name> --producer.config sslconfig.properties
 

You will get this following ERROR:

ERROR Error when sending message to topic topic1 with key: null, value: 8 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic test not present in metadata after 60000 ms.
 

Solution

RH AMQ Stream cluster/Kafka Stream cluster talks with every single member of the cluster via TLS port 443 when it’s exposed as mentioned before AVI picked up the Openshift service port in every cluster member which is 9094, therefore this needs to be changed in AVI in every single service port it to 443.

At the AVI network dashboard click on service go to > edit virtual service in Service Port then change it from 9094 to 443 in every cluster member including the external bootstrap service route shown in the AVI configuration.

Some useful links:

• Openshift community

• Strimzi operator

• AVI networks proxy support page

• AVI documentation

This page was last update at 2024-06-07 11:01