1. Accueil
  2. Question et réponse
  3. Hortonworks Data Platform 2.5: Arrêt des services HBase en raison d'une connexion défaillante

Hortonworks Data Platform 2.5: Arrêt des services HBase en raison d'une connexion défaillante

2016-11-23 3 views
0

J'ai mis en place un nouveau cluster Hadoop (contenant 6 machines) avec HDP 2.5. L'installation a bien fonctionné et les premières minutes tout a semblé fonctionner correctement. Mais après quelques minutes, deux des services Hbase a cessé de travailler:Hortonworks Data Platform 2.5: Arrêt des services HBase en raison d'une connexion défaillante

  • HBase/host1.mydomain.de
    • HBase Maître processus: Échec de la connexion [Errno 111] Connexion refuced à host1.mydomain.de
  • HBase/host6.mydomain.de
    • HBase RegionServer processus: Echec de la connexion [Errno 111] Connexion refusée à host6.mydomain.de

Comme je l'ai googlé autour de cette question, je trouve ces conseils:

  • contrôle et activer ntpd (activé avant l'installation, toujours désactivé)
  • contrôle et désactiver le pare-feu (désactivé avant l'installation , encore désactivé)
  • vérifier et désactiver SELinux (désactivé avant l'installation, toujours désactivé)

Le fait est que tous les services fonctionnaient au début, donc les services listés ci-dessus devraient être configurés correctement!

je peux dire ceci à ma configuration du cluster:

  • l'hôte Ambari-Server (host1) peut atteindre tous les hôtes de oher par ping et peut se connecter sans mot de passe par SSH
  • composants installés sont HDFS, YARN, MR2, Tez, ruche, HBase, porc, ZooKeeper, AmbariMetrics, Knox, Spark, Slider
  • Je suis parti tous les paramètres par défaut lors de l'installation, et je l'ai ignoré l'avertissement suivant:

enter image description here

Le fichier journal /var/log/hbase/hbase-hbase-master-host1.domain.de.log contient l'extrait followng (adresses IP sont noircis par AAAA/bbbb/xxxx/yyyy/zzzz):

2016-11-22 18:50:53,007 INFO [master/host1.xxx.de/xxx.xxx.xxx.xxx:16000] client.ZooKeeperRegistry: ClusterId read in ZooKeeper is null 
2016-11-22 18:51:59,581 INFO [Thread-70] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as bbb.bbb.bbb.bbb:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:51:59,584 INFO [Thread-70] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741825_1001 
2016-11-22 18:51:59,597 INFO [Thread-70] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[bbb.bbb.bbb.bbb:50010,DS-7691d8f6-0c76-4780-9836-85f20f935dd6,DISK] 
2016-11-22 18:52:33,674 INFO [Thread-70] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as zzz.zzz.zzz.zzz:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:33,675 INFO [Thread-70] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741837_1013 
2016-11-22 18:52:33,683 INFO [Thread-70] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[zzz.zzz.zzz.zzz:50010,DS-15d2586e-09a9-41ed-898d-689e15cd6596,DISK] 
2016-11-22 18:52:33,771 WARN [host1:16000.activeMasterManager] hdfs.DFSClient: Slow waitForAckedSeqno took 100584ms (threshold=30000ms) 
2016-11-22 18:52:33,797 INFO [host1:16000.activeMasterManager] util.FSUtils: Created version file at hdfs://host1.xxx.de:8020/apps/hbase/data with version=8 
2016-11-22 18:52:36,820 INFO [Thread-76] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as yyy.yyy.yyy.yyy:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:36,821 INFO [Thread-76] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741843_1019 
2016-11-22 18:52:36,828 INFO [Thread-76] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[yyy.yyy.yyy.yyy:50010,DS-439b87d1-f08d-464c-b0e2-728987cd211d,DISK] 
2016-11-22 18:52:37,567 INFO [Thread-76] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as zzz.zzz.zzz.zzz:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:37,567 INFO [Thread-76] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741845_1021 
2016-11-22 18:52:37,575 INFO [Thread-76] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[zzz.zzz.zzz.zzz:50010,DS-15d2586e-09a9-41ed-898d-689e15cd6596,DISK] 
2016-11-22 18:52:40,589 INFO [Thread-76] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as aaa.aaa.aaa.aaa:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:40,589 INFO [Thread-76] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741846_1022 
2016-11-22 18:52:40,593 INFO [Thread-76] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[aaa.aaa.aaa.aaa:50010,DS-15ea9223-2b1b-4f86-8797-ff0e2aaa6787,DISK] 
2016-11-22 18:52:40,694 INFO [host1:16000.activeMasterManager] master.MasterFileSystem: BOOTSTRAP: creating hbase:meta region 
2016-11-22 18:52:40,699 INFO [host1:16000.activeMasterManager] regionserver.HRegion: creating HRegion hbase:meta HTD == 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.Mul$ 
2016-11-22 18:52:43,741 INFO [Thread-79] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as yyy.yyy.yyy.yyy:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:43,742 INFO [Thread-79] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741848_1024 
2016-11-22 18:52:43,744 INFO [Thread-79] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[yyy.yyy.yyy.yyy:50010,DS-fc167096-246b-4215-b344-be786d98c472,DISK] 
2016-11-22 18:52:46,760 INFO [Thread-79] hdfs.DFSClient: Exception in createBlockOutputStream 
java.io.IOException: Got error, status message , ack with firstBadLink as zzz.zzz.zzz.zzz:50010 
     at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1393) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295) 
     at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463) 
2016-11-22 18:52:46,760 INFO [Thread-79] hdfs.DFSClient: Abandoning BP-90489822-xxx.xxx.xxx.xxx-1479836232259:blk_1073741849_1025 
2016-11-22 18:52:46,766 INFO [Thread-79] hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[zzz.zzz.zzz.zzz:50010,DS-c1dc059d-8f0b-4971-88cc-ebc76dd8659a,DISK]

Quelqu'un peut-il donner une idée de pourquoi ces (et seulement ceux-ci) deux services s'arrêtent après quelques minutes?

Source

2016-11-23 D. Müller

+0

double possible de [HDP 2.5: Spark Historique du serveur UI ne montrera pas les demandes incomplètes] (http://stackoverflow.com/questions/40844893/hdp-2-5-spark-history-server-ui-wont-show-incomplete-applications) –

Répondre

0

Je suis même question avant ZooKeeperRegistry: ClusterId read in ZooKeeper is null

Dans Hbase -> config -> HBase-site.xml changé la valeur de zookeeper.znode.parent à /hbase-unsecure et remis en marche les services

Source

2016-11-23 08:28:45

+0

Merci pour le conseil, mais Ma config cette propriété est déjà définie.Vous avez raison, dans le journal est apparu le problème ClusterId. Étrange ... –

+0

pouvez-vous poster le journal complet –

 Questions connexes

  • Aucun problème connexe^_^