INFO mapreduce.Job: Task Id : attempt_1610003035850_0014_r_000002_2, Status : FAILED

最新推荐文章于 2025-02-09 21:43:35 发布

原创最新推荐文章于 2025-02-09 21:43:35 发布 · 2.1k 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

hadoop

在运行Hadoop MapReduce作业时遇到了任务失败，具体表现为Task Id : attempt_1610003035850_0014_r_000002_2状态为FAILED。错误信息为java.io.EOFException，根源在于自定义的Writable接口实现中write方法使用错误，应将DataOutput.write()更正为DataOutput.writeInt()。

运行hadoop报错

INFO mapreduce.Job: Task Id : attempt_1610003035850_0014_r_000002_2, Status : FAILED
Error: java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340)
at java.io.DataInputStream.readUTF(DataInputStream.java:589)
at java.io.DataInputStream.readUTF(DataInputStream.java:564)
at ynet.partscore.test.FlowScore.readFields(FlowScore.java:58)
at org.apache.hadoop.io.serializer.WritableSerialization $W r i t a b l e D e s e r i a l i z e r . d e s e r i a l i z e (W r i t a b l e S e r i a l i z a t i o n . j a v a : 71) a t o r g . a p a c h e . h a d o o p . i o . s e r i a l i z e r . W r i t a b l e S e r i a l i z a t i o n$ WritableDeserializer.deserialize(WritableSerialization.java:42)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:146)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:302)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)