Comment créer Index et de cartographie en ES de LOGSTASH

Question

J'ai suivi ce tutoriel pour les données impport d'un DB dans LOGSTASh et créer un Idex et de cartographie en élastique Recherche INSERT INTO LOGSTASH SELECT DATA FROM DATABASE

Ceci est ma sortie en fonction de mon Fichier de configuration:

[2017-10-12T11:50:45,807][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"C:/Users/Bruno/Downloads/logstash-5.6.2/logstash-5.6.2/modules/fb_apache/configuration"} 
[2017-10-12T11:50:45,812][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"C:/Users/Bruno/Downloads/logstash-5.6.2/logstash-5.6.2/modules/netflow/configuration"} 
[2017-10-12T11:50:46,518][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} 
[2017-10-12T11:50:46,521][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"} 
[2017-10-12T11:50:46,652][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"} 
[2017-10-12T11:50:46,654][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil} 
[2017-10-12T11:50:46,716][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}} 
[2017-10-12T11:50:46,734][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} 
[2017-10-12T11:50:46,749][INFO ][logstash.pipeline  ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500} 
[2017-10-12T11:50:47,053][INFO ][logstash.pipeline  ] Pipeline main started 
[2017-10-12T11:50:47,196][INFO ][logstash.agent   ] Successfully started Logstash API endpoint {:port=>9600} 
[2017-10-12T11:50:47,817][INFO ][logstash.inputs.jdbc  ] (0.130000s) SELECT * from EP_RDA_STRING 
[2017-10-12T11:50:53,095][WARN ][logstash.agent   ] stopping pipeline {:id=>"main"} 

Tout semble OK, au moins je pense. Sauf le fait que l'interrogation du serveur ES pour les index OUTPUT et Mappings, je l'ai vide.

http://localhost:9200/_all/_mapping 

{} 

http://localhost:9200/_cat/indices?v 

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size 

ceci est mon fichier de configuration:

input { 
    jdbc { 
     # sqlserver jdbc connection string to our database, mydb   
     jdbc_connection_string => "jdbc:sqlserver://localhost:1433;databaseName=RDA; integratedSecurity=true;" 
     # The user we wish to execute our statement as 
     jdbc_user => "" 
     # The path to our downloaded jdbc driver 
     jdbc_driver_library => "C:\mypath\sqljdbc_6.2\enu\mssql-jdbc-6.2.1.jre8.jar" 
     # The name of the driver class for Postgresql 
     jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver" 
     # our query 
     statement => "SELECT * from EP_RDA_STRING" 
    } 
} 
output { 
    elasticsearch { 

     index => "RDA" 
     document_type => "RDA_string_view" 
     document_id => "%{ndb_no}" 
     hosts => "localhost:9200" 
    } 
} 

Answer 1

Quelle version de logstash utilisez-vous? Quelle est la commande que vous utilisez pour démarrer le logstash? Assurez-vous que les blocs d'entrée et de sortie ressemblent à celle qui est donnée ci-dessous

input { 
    beats { 
     port => "29600" 
     type => "weblogic-server" 
    } 
} 
filter { 
} 

output { 
    elasticsearch { 
     hosts => ["127.0.0.1:9200"] 
     index => "logstash-%{+YYYY.MM.dd}" 
    } 
    stdout { codec => rubydebug } 
}