Hadoop 0.20.2 réducteur lance ArrayIndexOutOfBoundsException lors de l'itération des valeurs

Question

Je suis assez nouveau pour hadoop, cependant, j'ai lu "Hadoop: Le guide définitif", donc je pense que j'ai une compréhension des concepts de base.

J'utilise Hadoop 0.20.2 pour exécuter un travail assez simple, mais je reçois l'exception suivante:

java.lang.ArrayIndexOutOfBoundsException: 4096 
     at java.io.ByteArrayInputStream.read(ByteArrayInputStream.java:127) 
     at java.io.DataInputStream.readInt(DataInputStream.java:373) 
     at com.convertro.mapreduce.WritableHit.readFields(Unknown Source) 
     at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeseria 
lizer.deserialize(WritableSerialization.java:67) 
     at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeseria 
lizer.deserialize(WritableSerialization.java:40) 
     at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext. 
java:116) 
     at org.apache.hadoop.mapreduce.ReduceContext$ValueIterator.next(ReduceCo 
ntext.java:163) 
     at com.convertro.mapreduce.HitConvertingIterable$HitConvertingIterator.n 
ext(HitConvertingIterable.java:35) 
     at com.convertro.mapreduce.HitConvertingIterable$HitConvertingIterator.n 
ext(HitConvertingIterable.java:1) 
     at com.convertro.naive.NaiveHitReducer.reduce(Unknown Source) 
     at com.convertro.mapreduce.HitReducer.reduce(Unknown Source) 
     at com.convertro.mapreduce.HitReducer.reduce(Unknown Source) 
     at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) 
     at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566 
) 
     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) 
     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:2 

Cela se produit lors de la lecture de la classe WritableHit (la valeur d'entrée à la phase réduire). Voici le code de classe WritableHit:

public class WritableHit implements WritableComparable<WritableHit> { 

    private Hit hit; 

    public WritableHit() { 
     this(null); 
    } 

    public WritableHit(Hit hit) { 
     this.hit = hit; 
    } 

    @Override 
    public void readFields(DataInput input) throws IOException { 
     String clientName = input.readUTF(); 
     String clientSiteId = input.readUTF(); 
     String eventUniqueId = input.readUTF(); 
     String eventValue = input.readUTF(); 
     String pageRequested = input.readUTF(); 
     String refererUrl = input.readUTF(); 
     String uniqueHitId = input.readUTF(); 
     String userAgent = input.readUTF(); 
     String userIdentifier = input.readUTF(); 
     String userIp = input.readUTF(); 
     int timestamp = input.readInt(); 
     int version = input.readInt(); 

     hit = new Hit(version, uniqueHitId, clientName, clientSiteId, timestamp, userIdentifier, 
       userIp, pageRequested, refererUrl, userAgent, eventUniqueId, eventValue); 
    } 

    @Override 
    public void write(DataOutput output) throws IOException { 
     output.writeUTF(hit.getClientName()); 
     output.writeUTF(hit.getClientSiteId()); 
     output.writeUTF(hit.getEventUniqueId()); 
     output.writeUTF(hit.getEventValue()); 
     output.writeUTF(hit.getPageRequested()); 
     output.writeUTF(hit.getRefererUrl()); 
     output.writeUTF(hit.getUniqueHitId()); 
     output.writeUTF(hit.getUserAgent()); 
     output.writeUTF(hit.getUserIdentifier()); 
     output.writeUTF(hit.getUserIp()); 
     output.write(hit.getTimestamp()); 
     output.write(hit.getVersion()); 
    } 

    public Hit getHit() { 
     return hit; 
    } 

    @Override 
    public int compareTo(WritableHit o) { 
     return hit.getUniqueHitId().compareTo(o.getHit().getUniqueHitId()); 
    } 
    } 

Toute aide serait grandement appréciée.

Merci

Answer 1

Je l'ai compris.

Apparemment, lorsque vous implémentez un objet inscriptible, vous devez utiliser la méthode writeInt, et non la méthode écrire.

Une fois cela fait, ça a marché comme un charme.