Flink 踩坑汇总

最新推荐文章于 2026-04-29 15:14:23 发布

原创最新推荐文章于 2026-04-29 15:14:23 发布 · 6.3k 阅读

3 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

flink

这篇博客汇总了作者在学习Flink过程中遇到的两个主要问题：1）在使用field positions指定keys时出现InvalidProgramException，原因是尝试在Java程序中使用Scala的Tuple2；2）由于jar包冲突，KafkaConsumer调用assign方法时抛出NoSuchMethodError，解决方法是确保所有相关依赖版本一致。

Flink 踩坑汇总

- 1、Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: Specifying keys via field positions is only valid for tuple data types. Type: GenericType

最近在学习Flink，踩坑汇总。（后续会持续添加）

1、Exception in thread “main” org.apache.flink.api.common.InvalidProgramException: Specifying keys via field positions is only valid for tuple data types. Type: GenericType<scala.Tuple2>

17:23:53,861 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - class scala.Tuple2 is missing a default constructor so it cannot be used as a POJO type and must be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance.
Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: Specifying keys via field positions is only valid for tuple data types. Type: GenericType<scala.Tuple2>
	at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:232)
	at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:223)
	at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:319)
	at com.xiaojian.StreamingJob.main(StreamingJob.java:76)

运行Flink的时候一直报这个错。

原因：我写的是Java程序，但是引用的Tuple2包是scala的，改成Java的就行。

2、java.lang.NoSuchMethodError: org.apache.kafka.clients.consumer.KafkaConsumer.assign(Ljava/util/List;)V

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
	at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:647)
	at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
	at com.xiaojian.mode.kafka.KafkaStream.main(KafkaStream.java:30)
Caused by: java.lang.NoSuchMethodError: org.apache.kafka.clients.consumer.KafkaConsumer.assign(Ljava/util/List;)V
	at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerCallBridge09.assignPartitions(KafkaConsumerCallBridge09.java:42)
	at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.reassignPartitions(KafkaConsumerThread.java:405)
	at org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:243)

原因：我使用的 flink-connector-kafka-0.10_2.11中的kafka是 0.10 版本的，但是 kafka-clients 是 2.3.1 的，jar包冲突。此处将 kafka-clients 改成 0.10 版本就可以了。

标签

#flink