Hive on spark. Lire le fichier parquet

Question

J'essaie de lire le fichier parquet dans Hive on Spark.

J'ai trouvé que je devais faire quelque chose sorte de que:

CREATE TABLE avro_test ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED 
AS AVRO TBLPROPERTIES ('avro.schema.url'='/files/events/avro_events_scheme.avsc'); 

CREATE EXTERNAL TABLE parquet_test LIKE avro_test STORED AS PARQUET LOCATION '/files/events/parquet_events/'; 

où mon schéma Avro est:

{ 
"type" : "parquet_file", 
    "namespace" : "events", 
    "name" : "events", 
    "fields" : [ 
      { "name" : "category" , "type" : "string" }, 
      { "name" : "duration" , "type" : "long" }, 
      { "name" : "name" , "type" : "string" }, 
      { "name" : "user_id" , "type" : "string"}, 
      { "name" : "value" , "type" : "long" } 
    ] 
} 

En conséquence, je reçois une erreur:

org.apache.spark.sql.catalyst.parser.ParseException: 
Operation not allowed: ROW FORMAT SERDE is incompatible with format 'avro', 
which also specifies a serde(line 1, pos 0) 

Answer 1

I think we have to add inputforamt and outputformat classes. 

CREATE TABLE parquet_test 
ROW FORMAT SERDE 
    'org.apache.hadoop.hive.serde2.avro.AvroSerDe' 
STORED AS INPUTFORMAT 
    'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' 
OUTPUTFORMAT 
    'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' 
TBLPROPERTIES (
    'avro.schema.url''avro.schema.url'='/hadoop/avro_events_scheme.avsc'); 

I hope above would work.