Skip to content

Need to use the same processing function for multiple Kafka topics with different consumer groups in Spring Cloud Stream #3143

@Haibarapink

Description

@Haibarapink

Need to use the same processing function for multiple Kafka topics with different consumer groups in Spring Cloud Stream
Here is my application.yml:

spring:
  cloud:
    function:
      definition: streamProcess  
    stream:
      management:
        endpoint:
          health:
            show-details: always
            binders:
              enabled: true
      
      bindings:

        streamProcess-in-0:
          destination: TOPIC_A
          group: GROUP_A
          binder: kafkaBinder
          consumer:
            batch-mode: false
            concurrency: 1
            #client-id: consumer-group-a


        streamProcess-in-1:
          destination: TOPIC_B
          group: GROUP_B
          binder: kafkaBinder
          consumer:
            batch-mode: false
            concurrency: 1
            #client-id: consumer-group-b

      binders:
        kafkaBinder:
          type: kafka
          environment:
            spring:
              kafka:
                bootstrap-servers: kafka-server1:9092,kafka-server2:9092,kafka-server3:9092
                properties:
                  security.protocol: SASL_PLAINTEXT
                  sasl.mechanism: PLAIN
                  sasl.jaas.config: >-
                    org.apache.kafka.common.security.plain.PlainLoginModule required
                    username="YOUR_USERNAME"
                    password="YOUR_PASSWORD";

                  enable.auto.commit: true
                  max.poll.records: 30000
                  max.poll.interval.ms: 600000  
                  session.timeout.ms: 800000    
                  heartbeat.interval.ms: 200000 
            cloud:
              stream:
                kafka:
                  binder:
                    auto-create-topics: false

But Spring app could process the message from TOPIC_B. Thanks a lot!!!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions