I want to see the retention period set for a particular topic.Is there any command? I tried with
bin/kafka-topics.sh --zookeeper hostname:2181 --alter --config retention.ms=172800000 --topic <topic_name>for deleting 2 days old data. But I want to see how many days are set in the retention period for all the topics.
Best Answer
If you have altered a topic and want to view the topic configuration the following command will be helpful
kafka-topics.sh --zookeeper localhost:2181 --describe --topics-with-overridesThis will describe only the topics along with configurations that have configurations set that differ from the cluster defaults.
If you want to view the configurations for all topicEither you can view these properties log.retention.hours or log.retention.ms in server.properties in kafka config directory.
See config 'log.retention.hours' in Kafka Doc:
The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property
Default value is 168 hours which is 7 days.
Alternative ways:
- use call sh-command through a container
docker run --rm -it confluentinc/cp-kafka:latest sh -c "kafka-topics --zookeeper 11.22.33.44:5555 --describe --topic topic-name"
- use Kafka Tool
Take into account that it would be displayed just params different from default values.
At the example above, the retention.ms is 14 days, when default one is 7 days.
Simply the below command will help
kafka-topics.sh --bootstrap-server server_ip:9092 --describe --topic topic_nameAlong with other infos , it will print delete.retention.ms.
A sample output will be
Topic:TOPIC NAME PartitionCount:6 ReplicationFactor:1 Configs:compression.type=gzip,segment.bytes=1073741824,retention.ms=100,max.message.bytes=100001200,delete.retention.ms=100000Topic: TOPIC NAME Partition: 0 Leader: 2 Replicas: 2 Isr: 2Topic: TOPIC NAME Partition: 1 Leader: 3 Replicas: 3 Isr: 3Topic: TOPIC NAME Partition: 2 Leader: 1 Replicas: 1 Isr: 1Topic: TOPIC NAME Partition: 3 Leader: 2 Replicas: 2 Isr: 2Topic: TOPIC NAME Partition: 4 Leader: 3 Replicas: 3 Isr: 3Topic: TOPIC NAME Partition: 5 Leader: 1 Replicas: 1 Isr: 1If you want to get the retention.ms or retention.bytes or any other config for any topic, you can simply use this command:
kafka-configs.sh --bootstrap-server <BOOTSTRAP_SERVER> --describe --topic <TOPIC_NAME> --allMake sure to put here the address of the broker and the desired topic. This will output all configs defined for a particular topic, including the retention. Hope it helped :)