CDH中Kafka基本指令总结——topic使用与测试producer产生数据、consu

2024年2月6日发(作者：)

message formatter. Default

properties include:

amp=true|false

=true|false

=true|false

tor=

tor=

alizer=

alizer=

deserializer>

Users can also pass in customized

properties for their formatter; more

specifically, users can pass in

properties keyed with 'key.

deserializer.' and 'value.

deserializer.' prefixes to configure

their deserializers.

--skip-message-on-error If there is an error when processing a

message, skip it instead of halt.

--timeout-ms If specified, exit if no message is

available for consumption for the

specified interval.

--topic The topic id to consume on.

--value-deserializer

deserializer for values>

--whitelist Regular expression specifying

whitelist of topics to include for

consumption.

查看消费数据后的偏移量 kafka-run-class(1)查看每个Partition的最新偏移量kafka-run-class setShell --broker-list hadoop:9092 --topic yourTopic --time -1

(2)查看每个Partition的最早的偏移量kafka-run-class setShell --broker-list hadoop:9092 --topic yourTopic --time -2

(3)查看consumer组内消费的offsetkafka-run-class erOffsetChecker --zookeeper hadoop:2181 --topic yourTopic获取topic消费组的偏移量kafka-consumer-offset-checker --zookeeper=localhost:2181 --topic=mytopic --group=my_consumer_group