ERREUR Hadoop SUR LE PROGRAMME wordcount

public static void main(String[] args) throws Exception 
{ 
    Configuration conf = new Configuration(); 
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml")); 
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml")); 
    Job job = new Job(); 
    //job.setJarByClass(WordCount.class); 
    job.setJobName("WordCounter"); 

    job.setJarByClass(WordCount.class); 
    job.setMapperClass(TokenizerMapper.class); 
    job.setCombinerClass(IntSumReducer.class); 
    job.setReducerClass(IntSumReducer.class); 
    job.setOutputKeyClass(Text.class); 
    job.setOutputValueClass(IntWritable.class); 
    FileInputFormat.addInputPath(job, new Path("/home/pramukh/eclipse/eclipseproj/hadoop/input/input.txt")); 
    FileOutputFormat.setOutputPath(job, new Path("/home/pramukh/eclipse/eclipseproj/hadoop/output.txt")); 
//System.exit(0); 
    System.exit(job.waitForCompletion(true) ? 0 : 1); 
    }

Ceci est mon Hadoop java wordcount exemple, ce qui donne l'erreur suivante:ERREUR Hadoop SUR LE PROGRAMME wordcount

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/io/DatumReader at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1074) at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:69) at org.apache.hadoop.io.serializer.SerializationFactory.(SerializationFactory.java:62) at org.apache.hadoop.mapreduce.split.JobSplitWriter.writeNewSplits(JobSplitWriter.java:117) at org.apache.hadoop.mapreduce.split.JobSplitWriter.createSplitFiles(JobSplitWriter.java:74) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1030) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1041) at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapreduce.Job.submit(Job.java:500) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530) at hadoop.WordCount.main(WordCount.java:66) Caused by: java.lang.ClassNotFoundException: org.apache.avro.io.DatumReader at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Merci à l'avance.

Source

2017-01-17 user7431472

Tout d'abord, vous n'avez pas besoin de crier (caps) :) – ppasler

Voici une solution possible: http://stackoverflow.com/questions/30675014/hive-error-cdh-5-error-java-lang-classnotfoundexception-org-apache-avro-io-da – ppasler

Si le reste de votre code dans ce programme est corrigé; Je vous suggère de vérifier le chemin Le dossier OutputPath ne peut pas être existe
J'ai essayé votre code modifie le chemin du fichier, le code peut fonctionner normalement

FileInputFormat.addInputPath(job, new Path("hdfs://localhost:9000/wordcount/test.txt")); 
    FileOutputFormat.setOutputPath(job, new Path("hdfs://localhost:9000/wordcount/out"));

Source

2017-05-08 17:21:06

ERREUR Hadoop SUR LE PROGRAMME wordcount

Répondre

Questions connexes