Debugging and Monitoring

Logging

Logging in Simple Sourcing, as in Kafka Streams is done through the SLF4J abstraction layer. A common choice for logging implementations that works with SLF4J is Logback.

If you use Logback, the logback.xml configuration file can be used to control the logging verbosity of messages generated by both Kafka streams and Simple Sourcing.

Add the following logback.xml file to your application resources folder:

<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>
    <logger name="org.apache.kafka" level="WARN"/>
    <logger name="org.apache.kafka.common.metrics" level="WARN"/>
    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>
</configuration>

Broker access

Docker manages connections between all the Docker containers. However if you want to access the Kafka broker from outside a container, there’s one extra step required. Kafka is very specific about the hostname it listens on - it must match the value specified via the KAFKA_ADVERTISED_LISTENERS environment variable.

If you are using Docker for Mac >= 1.12, Docker for Linux, or Docker for Windows 10, then please add the following lines to /etc/hosts or C:\Windows\System32\Drivers\etc\hosts:

127.0.0.1	broker

Once that’s added you can then run and debug your Kafka Streams app in your favorite IDE connecting to the Kafka broker running within docker.

Clearing state

Often during development you’ll make incompatible changes to your domain model schemas. When running locally, the easiest approach to handle these changes is to blow away the old environment.

docker-compose down
rm -r /tmp/kafka-streams
docker-compose up -d