Keras/HDF5 TypeError: PointSelection __getitem__ fonctionne uniquement avec des tableaux booléens

Question

Lorsque je tente charge les données de formation à partir d'un fichier HDF5 en utilisant les fonctions de fit_generator et générateur, je reçois un ValueError, qui résulte d'un PointSelectionError avec HDF5:

Epoch 1/10 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
    File "/usr/lib/python2.7/threading.py", line 763, in run 
    self.__target(*self.__args, **self.__kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 429, in data_generator_task 
    generator_output = next(self._generator) 
    File "osr.py", line 108, in generate_training_sequences 
    X = training_save_file["X"][batch_idxs] 
    File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684) 
    File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642) 
    File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/dataset.py", line 462, in __getitem__ 
    selection = sel.select(self.shape, args, dsid=self.id) 
    File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/selections.py", line 72, in select 
    sel[arg] 
    File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/selections.py", line 210, in __getitem__ 
    raise TypeError("PointSelection __getitem__ only works with bool arrays") 
TypeError: PointSelection __getitem__ only works with bool arrays 

Traceback (most recent call last): 
    File "osr.py", line 359, in <module> 
    osr.train_osr_model() 
    File "osr.py", line 89, in train_osr_model 
    nb_worker=1) 
    File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1532, in fit_generator 
    str(generator_output)) 
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None 

J'ai recherché l'erreur et il a été mentionné que cela pourrait être dû à des indices en double, mais cela ne semble pas être vrai dans mon cas. Voici les indices de ligne qui ont été consultés:

[581 305 67 510 631 832 340 663 689 801 579 701 831 879 382 844 15 798 
342 329 118 657 503 129 602 2 528 157 341 299 731 539] 

Voici les fonctions de formation et de génération:

def train_osr_model(self): 
     """ Train the optical speech recognizer 
     """ 
     print "\nTraining OSR" 
     validation_ratio = 0.3 
     batch_size = 32 
     with h5py.File(self.training_save_fn, "r") as training_save_file: 
      sample_count = int(training_save_file.attrs["sample_count"]) 
      sample_idxs = range(0, sample_count) 
      sample_idxs = np.random.permutation(sample_idxs) 
      training_sample_idxs = sample_idxs[0:int((1-validation_ratio)*sample_count)] 
      validation_sample_idxs = sample_idxs[int((1-validation_ratio)*sample_count):] 
      training_sequence_generator = self.generate_training_sequences(batch_size=batch_size, 
                      training_save_file=training_save_file, 
                      training_sample_idxs=training_sample_idxs) 
      validation_sequence_generator = self.generate_validation_sequences(batch_size=batch_size, 
                       training_save_file=training_save_file, 
                       validation_sample_idxs=validation_sample_idxs) 

      print "Sample Idxs: {0}\n".format(sample_idxs) # FOR DEBUG ONLY 
      print "Training Idxs: {0}\n".format(training_sample_idxs) # FOR DEBUG ONLY 
      print "Validation Idxs: {0}\n".format(validation_sample_idxs) # FOR DEBUG ONLY 

      pbi = ProgressDisplay() 
      self.osr.fit_generator(generator=training_sequence_generator, 
            validation_data=validation_sequence_generator, 
            samples_per_epoch=len(training_sample_idxs), 
            nb_val_samples=len(validation_sample_idxs), 
            nb_epoch=10, 
            max_q_size=1, 
            verbose=2, 
            callbacks=[pbi], 
            class_weight=None, 
            nb_worker=1) 

    def generate_training_sequences(self, batch_size, training_save_file, training_sample_idxs): 
     """ Generates training sequences from HDF5 file on demand 
     """ 
     while True: 
      # generate sequences for training 
      training_sample_count = len(training_sample_idxs) 
      batches = int(training_sample_count/batch_size) 
      remainder_samples = training_sample_count%batch_size 
      if remainder_samples: 
       batches = batches + 1 
      # generate batches of samples 
      for idx in xrange(0, batches): 
       if idx == batches - 1: 
        batch_idxs = training_sample_idxs[idx*batch_size:] 
       else: 
        batch_idxs = training_sample_idxs[idx*batch_size:idx*batch_size+batch_size] 

       print batch_idxs # FOR DEBUG ONLY 

       X = training_save_file["X"][batch_idxs] 
       Y = training_save_file["Y"][batch_idxs] 



       yield (np.array(X), np.array(Y)) 

    def generate_validation_sequences(self, batch_size, training_save_file, validation_sample_idxs): 
     while True: 
      # generate sequences for validation 
      validation_sample_count = len(validation_sample_idxs) 
      batches = int(validation_sample_count/batch_size) 
      remainder_samples = validation_sample_count%batch_size 
      if remainder_samples: 
       batches = batches + 1 
      # generate batches of samples 
      for idx in xrange(0, batches): 
       if idx == batches - 1: 
        batch_idxs = validation_sample_idxs[idx*batch_size:] 
       else: 
        batch_idxs = validation_sample_idxs[idx*batch_size:idx*batch_size+batch_size] 

       print batch_idxs # FOR DEBUG ONLY 

       X = training_save_file["X"][batch_idxs] 
       Y = training_save_file["Y"][batch_idxs] 

       yield (np.array(X), np.array(Y)) 

Voici les fonctions qui prétraitements et enregistre les données de formation dans un fichier HDF5:

def process_training_data(self): 
    """ Preprocesses training data and saves them into an HDF5 file 
    """ 
    # load training metadata from config file 
    training_metadata = {} 
    training_classes = [] 
    with open(self.config_file) as training_config: 
     training_metadata = json.load(training_config) 
     training_classes = sorted(list(training_metadata.keys())) 

     print "".join(["\n", 
         "Found {0} training classes!\n".format(len(training_classes)), 
         "-"*40]) 
     for class_label, training_class in enumerate(training_classes): 
      print "{0:<4d} {1:<10s} {2:<30s}".format(class_label, training_class, training_metadata[training_class]) 
     print "" 

    # count number of samples 
    sample_count = 0 
    sample_count_by_class = [0]*len(training_classes) 
    for class_label, training_class in enumerate(training_classes): 
     # get training class sequeunce paths 
     training_class_data_path = training_metadata[training_class] 
     training_class_sequence_paths = [os.path.join(training_class_data_path, file_name) 
             for file_name in os.listdir(training_class_data_path) 
             if (os.path.isfile(os.path.join(training_class_data_path, file_name)) 
              and ".mov" in file_name)] 
     # update sample count 
     sample_count += len(training_class_sequence_paths) 
     sample_count_by_class[class_label] = len(training_class_sequence_paths) 

    print "".join(["\n", 
        "Found {0} training samples!\n".format(sample_count), 
        "-"*40]) 
    for class_label, training_class in enumerate(training_classes): 
     print "{0:<4d} {1:<10s} {2:<6d}".format(class_label, training_class, sample_count_by_class[class_label]) 
    print "" 

    # initialize HDF5 save file, but clear older duplicate first if it exists 
    try: 
     print "Saved file \"{0}\" already exists! Overwriting previous saved file.\n".format(self.training_save_fn) 
     os.remove(self.training_save_fn) 
    except OSError: 
     pass 

    # process and save training data into HDF5 file 
    print "Generating {0} samples from {1} samples via data augmentation\n".format(sample_count*self.samples_generated_per_sample, 
                        sample_count) 
    sample_count = sample_count*self.samples_generated_per_sample 
    with h5py.File(self.training_save_fn, "w") as training_save_file: 
     training_save_file.attrs["training_classes"] = np.string_(",".join(training_classes)) 
     training_save_file.attrs["sample_count"] = sample_count 
     x_training_dataset = training_save_file.create_dataset("X", 
                   shape=(sample_count, self.frames_per_sequence, 3, self.rows, self.columns), 
                   dtype="f") 
     y_training_dataset = training_save_file.create_dataset("Y", 
                   shape=(sample_count, len(training_classes)), 
                   dtype="i") 

     # iterate through each class data 
     sample_idx = 0 
     for class_label, training_class in enumerate(training_classes): 
      # get training class sequeunce paths 
      training_class_data_path = training_metadata[training_class] 
      training_class_sequence_paths = [os.path.join(training_class_data_path, file_name) 
              for file_name in os.listdir(training_class_data_path) 
              if (os.path.isfile(os.path.join(training_class_data_path, file_name)) 
               and ".mov" in file_name)] 
      # iterate through each sequence 
      for idx, training_class_sequence_path in enumerate(training_class_sequence_paths): 
       sys.stdout.write("Processing training data for class \"{0}\": {1}/{2} sequences\r" 
           .format(training_class, idx+1, len(training_class_sequence_paths))) 
       sys.stdout.flush() 

       # accumulate samples and labels 
       samples_batch = self.process_frames(training_class_sequence_path) 
       label = [0]*len(training_classes) 
       label[class_label] = 1 
       label = np.array(label).astype("int32") 

       for sample in samples_batch: 
        x_training_dataset[sample_idx] = sample 
        y_training_dataset[sample_idx] = label 

        # update sample index 
        sample_idx += 1 

      print "\n" 

     training_save_file.close() 

     print "Training data processed and saved to {0}".format(self.training_save_fn) 

def process_frames(self, video_file_path): 
    """ Preprocesses sequence frames 
    """ 
    # haar cascades for localizing oral region 
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml') 
    mouth_cascade = cv2.CascadeClassifier('haarcascade_mcs_mouth.xml') 

    video = cv2.VideoCapture(video_file_path) 
    success, frame = video.read() 

    frames = [] 
    success = True 

    # convert to grayscale, localize oral region, equalize frame dimensions, and accumulate valid frames 
    while success: 
     success, frame = video.read() 
     if success: 
     # convert to grayscale 
     frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) 

     # localize single facial region 
     faces_coords = face_cascade.detectMultiScale(frame, 1.3, 5) 
     if len(faces_coords) == 1: 
      face_x, face_y, face_w, face_h = faces_coords[0] 
      frame = frame[face_y:face_y + face_h, face_x:face_x + face_w] 

      # localize oral region 
      mouth_coords = mouth_cascade.detectMultiScale(frame, 1.3, 5) 
      threshold = 0 
      for (mouth_x, mouth_y, mouth_w, mouth_h) in mouth_coords: 
      if (mouth_y > threshold): 
       threshold = mouth_y 
       valid_mouth_coords = (mouth_x, mouth_y, mouth_w, mouth_h) 
      else: 
       pass 
      mouth_x, mouth_y, mouth_w, mouth_h = valid_mouth_coords 
      frame = frame[mouth_y:mouth_y + mouth_h, mouth_x:mouth_x + mouth_w] 

      # equalize frame dimensions 
      frame = cv2.resize(frame, (self.columns, self.rows)).astype('float32') 

      # accumulate frames 
      frames.append(frame) 

     # ignore multiple facial region detections 
     else: 
      pass 

    # equalize sequence lengths 
    if len(frames) < self.frames_per_sequence: 
     frames = [frames[0]]*(self.frames_per_sequence - len(frames)) + frames 
    frames = np.array(frames[-self.frames_per_sequence:]) 

    # function to normalize and add channel dimension to each frame 
    proc_frame = lambda frame: np.array([frame/255.0]*3) 

    samples_batch = [np.array(map(proc_frame, frames))] 
    # random transformations for data augmentation 
    for _ in xrange(0, self.samples_generated_per_sample-1): 
     rotated_frames = random_rotation(frames, rg=4.5) 
     shifted_frames = random_shift(rotated_frames, wrg=0.05, hrg=0.05) 
     sheared_frames = random_shear(shifted_frames, intensity=0.08) 
     zoomed_frames = random_zoom(sheared_frames, zoom_range=(1.05, 1.05)) 
     samples_batch.append(np.array(map(proc_frame, zoomed_frames))) 

    return samples_batch 

Answer 1

L'erreur vient de deux choses:

• Celui que vous lisez vient du fait que batch_idxs est un tableau, pas une liste. Les objets h5py acceptent l'indexation avec des listes. Mais même si vous changez

  X = training_save_file["X"][list(batch_idxs)] 
  

  vous obtiendrez toujours une erreur. Cela vient de certaines restrictions sur l'indexation de la liste. Cela nous amène au deuxième point.

• Si vous lisez le doc you sent me, cela est écrit:

  The following restrictions exist:

  • List selections may not be empty
  • Selection coordinates must be given in increasing order
  • Duplicate selections are ignored
  • Very long lists (> 1000 elements) may produce poor performance

  Le deuxième point est notre numéro: votre mélange aléatoire lors de la création training_sample_idxs rend aléatoire l'ordre d'index et l'ensemble de données les attend pour ecreasing. C'est une limitation à laquelle vous devrez faire face, mais ce n'est pas trop contraignant car l'ordre dans un batch n'a pas d'importance, le modèle sera quand même optimisé sur tout le lot.

Cela aide-t-il?