Architecture

Getting Started

Prerequisites

• Java 11
• Docker
• Kafka
• Schema Registry

Installation

Local

• Start Kafka broker and Schema Registry using Confluent Community or Confluent Platform components.

  • Confluent community:

      docker-compose -f ./docker/local/docker-compose.yml up -d
    

  • Confluent Platform:

    docker-compose -f ./docker/confluent-platform/docker-compose.yml up -d
    

• Check if all containers started, using docker-compose ps command.

  # Sample output for Confluent Platform
      Name                  Command            State                       Ports                     
  ----------------------------------------------------------------------------------------------------
  broker            /etc/confluent/docker/run   Up      0.0.0.0:9092->9092/tcp, 0.0.0.0:9101->9101/tcp
  control-center    /etc/confluent/docker/run   Up      0.0.0.0:9021->9021/tcp                        
  rest-proxy        /etc/confluent/docker/run   Up      0.0.0.0:8082->8082/tcp                        
  schema-registry   /etc/confluent/docker/run   Up      0.0.0.0:8081->8081/tcp                        
  zookeeper         /etc/confluent/docker/run   Up      0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp    
  

• Start Account Service

  ./gradlew bootRun
  

Usage

• Check Create Account flow

  • Import proto files into BloomRPC client.
  • Choose 6565 as gRPC server port.
  • Create new account using AccountService.Create rpc call.

    # Sample request body
    {
      "email": "newAccount@test.com",
      "countryCode": "GB",
      "type": 2
    }
    

  • Get account id from response

    # Sample response body
    {
      "id": "519faeb7-ccdf-49dd-95a4-3e2a6ac8ad13",
      "email": "newAccount@test.com",
      "countryCode": "GB",
      "type": "PREMIUM",
      "createdDate": {
      "seconds": "1616860904",
      "nanos": 644947000
      }
    }
    

  • Test interactive queries using REST endpoints.
    • Get account details from accounts.store state store (use account id retrieved before):

        curl -X GET --location "http://localhost:8080/api/v1/accounts/519faeb7-ccdf-49dd-95a4-3e2a6ac8ad13"
      

    • Get account statistics for 'GB' countryCode:

      curl -X GET --location "http://localhost:8080/api/v1/statistics/countries/GB/accounts"
      

• Analise kafka streams topology:

  • Get all available topologies.

    curl -X GET --location "http://localhost:8080/topologies"
    

  • Print topology for accounts-per-country kstream.

    curl -X GET --location "http://localhost:8080/topologies/accounts-per-country"
    

  • Print topology diagram, visit https://zz85.github.io/kafka-streams-viz/ page.

Testing Kafka Streams

• Unit test (TopologyTestDriver) examples may be found under src/test/.
• Integration test (Testcontainers) examples may be found under src/integrationTest/.

Deployment

Docker

Docker compose file with instructions are kept in docker subdirectory.

References

• Kafka Streams Topology Visualizer
• Spring for Apache Kafka
• Quick Start for Apache Kafka using Confluent Platform (Docker)
• DynamicPropertySource in Spring Framework
• Testcontainers: singleton-containers
• Testcontainers: Kafka Modules Reference Guide
• Testing a streams application
• BloomRPC
• Java Faker
• Gradle Test Logger Plugin
• Protobuf Plugin for Gradle

License

Distributed under the MIT License. See LICENSE for more information.