Lorsque j'utilise mahout et Hadoop pour faire des recommandations, j'ai rencontré un problème.mauvaise classe de valeurs: org.apache.mahout.math.VarLongWritable n'est pas class org.apache.mahout.math.VectorWritable
Le massage d'erreur est:
Error: java.io.IOException: wrong value class: org.apache.mahout.math.VarLongWritable is not class org.apache.mahout.math.VectorWritable
at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1378)
at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:558)
at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Et, La fonction principale est:
job.setInputFormatClass (TextInputFormat.class);
job.setMapperClass(FilesToItemPrefsMapper.class);
job.setMapOutputKeyClass(VarLongWritable.class);
job.setMapOutputValueClass(VarLongWritable.class);
job.setReducerClass(FileToUserVectorReducer.class);
job.setOutputKeyClass(VarLongWritable.class);
job.setOutputValueClass(VectorWritable.class);
job.setOutputFormatClass(SequenceFileOutputFormat.class);
SequenceFileOutputFormat.setOutputCompressionType(job,CompressionType.NONE);
Le mappeur est:
public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
String line = value.toString();
Matcher m = NUMBERS.matcher(line);
m.find();
VarLongWritable userID = new VarLongWritable(Long.parseLong(m.group()));
VarLongWritable itemID = new VarLongWritable();
while (m.find()){
itemID.set(Long.parseLong(m.group()));
context.write(userID, itemID);
}
Le réducteur est:
public class FileToUserVectorReducer
extends Reducer<VarLongWritable, VarLongWritable, VarLongWritable, VectorWritable> {
public void reducer(VarLongWritable userID, Iterable<VarLongWritable> itemPrefs, Context context)
throws IOException, InterruptedException{
Vector userVector = new RandomAccessSparseVector(Integer.MAX_VALUE, 100);
for(VarLongWritable itemPref : itemPrefs){
userVector.set((int)itemPref.get(), 1.0f);
}
context.write(userID, new VectorWritable(userVector));
}
}
Je pense que la valeur de réducteur qui est VectorWritable est situé dans le job.setOutputValueClass (VectorWritable.class) . Si oui, pourquoi donne-t-il un tel message d'erreur?