recherche élastique à de nombreuses agrégations: Impossible de se connecter au serveur

Question

Quand je en elasticsearch 5.2.0 ai plus de 64 agrégations il échoue avec:

{ 
    "ok": false, 
    "message": "Unable to connect to the server." 
} 

Ce problème est lié à Elastic Cloud - Je l'ai testé sur une instance élastique locale et il n'a pas cette restriction.

Est-ce que ce comportement est attendu?
J'ai essayé de supprimer différentes des agrégations, mais tout semble descendre au nombre d'agrégations. Cela fonctionne très bien avec 65 agrégations ou moins.

Il est agrégations simples comme les suivantes:

"aggs": { 
    "wheelSize": { 
     "filter": { ... }, 
     "aggs": { 
     "facet_terms": { 
      "terms": { 
      "field": "facets.wheelSize", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "tireSize": { 
     "filter": { ... }, 
     "aggs": { 
     "facet_terms": { 
      "terms": { 
      "field": "facets.tireSize", 
      "size": 1000 
      } 
     } 
     } 
    } 
    } 

Edit:
Code pour reproduire:

Créer un index:

PUT product_test 
{ 
    "mappings": { 
    "product": { 
     "properties": { 
     "prop1": { 
      "type": "long" 
     }, 
     "prop2": { 
      "type": "long" 
     }, 
     "prop3": { 
      "type": "long" 
     }, 
     "prop4": { 
      "type": "long" 
     }, 
     "prop5": { 
      "type": "long" 
     }, 
     "prop6": { 
      "type": "long" 
     }, 
     "prop7": { 
      "type": "long" 
     }, 
     "prop8": { 
      "type": "long" 
     }, 
     "prop9": { 
      "type": "long" 
     }, 
     "prop10": { 
      "type": "long" 
     }, 
     "prop11": { 
      "type": "long" 
     }, 
     "prop12": { 
      "type": "long" 
     }, 
     "prop13": { 
      "type": "long" 
     }, 
     "prop14": { 
      "type": "long" 
     }, 
     "prop15": { 
      "type": "long" 
     }, 
     "prop16": { 
      "type": "long" 
     }, 
     "prop17": { 
      "type": "long" 
     }, 
     "prop18": { 
      "type": "long" 
     }, 
     "prop19": { 
      "type": "long" 
     }, 
     "prop20": { 
      "type": "long" 
     }, 
     "prop21": { 
      "type": "long" 
     }, 
     "prop22": { 
      "type": "long" 
     }, 
     "prop23": { 
      "type": "long" 
     }, 
     "prop24": { 
      "type": "long" 
     }, 
     "prop25": { 
      "type": "long" 
     }, 
     "prop26": { 
      "type": "long" 
     }, 
     "prop27": { 
      "type": "long" 
     }, 
     "prop28": { 
      "type": "long" 
     }, 
     "prop29": { 
      "type": "long" 
     }, 
     "prop30": { 
      "type": "long" 
     }, 
     "prop31": { 
      "type": "long" 
     }, 
     "prop32": { 
      "type": "long" 
     }, 
     "prop33": { 
      "type": "long" 
     }, 
     "prop34": { 
      "type": "long" 
     }, 
     "prop35": { 
      "type": "long" 
     }, 
     "prop36": { 
      "type": "long" 
     }, 
     "prop37": { 
      "type": "long" 
     }, 
     "prop38": { 
      "type": "long" 
     }, 
     "prop39": { 
      "type": "long" 
     }, 
     "prop40": { 
      "type": "long" 
     }, 
     "prop41": { 
      "type": "long" 
     }, 
     "prop42": { 
      "type": "long" 
     }, 
     "prop43": { 
      "type": "long" 
     }, 
     "prop44": { 
      "type": "long" 
     }, 
     "prop45": { 
      "type": "long" 
     }, 
     "prop46": { 
      "type": "long" 
     }, 
     "prop47": { 
      "type": "long" 
     }, 
     "prop48": { 
      "type": "long" 
     }, 
     "prop49": { 
      "type": "long" 
     }, 
     "prop50": { 
      "type": "long" 
     }, 
     "prop51": { 
      "type": "long" 
     }, 
     "prop52": { 
      "type": "long" 
     }, 
     "prop53": { 
      "type": "long" 
     }, 
     "prop54": { 
      "type": "long" 
     }, 
     "prop55": { 
      "type": "long" 
     }, 
     "prop56": { 
      "type": "long" 
     }, 
     "prop57": { 
      "type": "long" 
     }, 
     "prop58": { 
      "type": "long" 
     }, 
     "prop59": { 
      "type": "long" 
     }, 
     "prop60": { 
      "type": "long" 
     }, 
     "prop61": { 
      "type": "long" 
     }, 
     "prop62": { 
      "type": "long" 
     }, 
     "prop63": { 
      "type": "long" 
     }, 
     "prop64": { 
      "type": "long" 
     }, 
     "prop65": { 
      "type": "long" 
     }, 
     "prop66": { 
      "type": "long" 
     }, 
     "prop67": { 
      "type": "long" 
     }, 
     "prop68": { 
      "type": "long" 
     }, 
     "prop69": { 
      "type": "long" 
     }, 
     "prop70": { 
      "type": "long" 
     }, 
     "prop71": { 
      "type": "long" 
     }, 
     "prop72": { 
      "type": "long" 
     }, 
     "prop73": { 
      "type": "long" 
     }, 
     "prop74": { 
      "type": "long" 
     }, 
     "prop75": { 
      "type": "long" 
     }, 
     "prop76": { 
      "type": "long" 
     }, 
     "prop77": { 
      "type": "long" 
     }, 
     "prop78": { 
      "type": "long" 
     }, 
     "prop79": { 
      "type": "long" 
     }, 
     "prop80": { 
      "type": "long" 
     } 
     } 
    } 
    }, 
    "settings": { 
"index": { 
     "number_of_shards": "1", 
     "number_of_replicas": "1" 
    } 
    } 
} 

Recherche: (supprimer/ajouter une agrégation pour voir le problème)

GET product_test/_search 
{ 
    "from": 0, 
    "size": 0, 
    "aggs": { 
    "prop1": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop1", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop2": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop2", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop3": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop3", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop4": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop4", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop5": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop5", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop6": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop6", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop7": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop7", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop8": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop8", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop9": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop9", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop10": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop10", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop11": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop11", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop12": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop12", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop13": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop13", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop14": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop14", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop15": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop15", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop16": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop16", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop17": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop17", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop18": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop18", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop19": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop19", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop20": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop20", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop21": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop21", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop22": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop22", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop23": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop23", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop24": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop24", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop25": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop25", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop26": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop26", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop27": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop27", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop28": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop28", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop29": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop29", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop30": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop30", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop31": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop31", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop32": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop32", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop33": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop33", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop34": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop34", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop35": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop35", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop36": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop36", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop37": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop37", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop38": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop38", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop39": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop39", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop40": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop40", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop41": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop41", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop42": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop42", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop43": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop43", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop44": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop44", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop45": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop45", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop46": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop46", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop47": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop47", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop48": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop48", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop49": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop49", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop50": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop50", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop51": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop51", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop52": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop52", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop53": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop53", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop54": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop54", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop55": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop55", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop56": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop56", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop57": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop57", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop58": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop58", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop59": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop59", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop60": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop60", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop61": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop61", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop62": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop62", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop63": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop63", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop64": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop64", 
      "size": 1000 
      } 
     } 
     } 
    }, 
    "prop65": { 
     "filter": {}, 
     "aggs": { 
     "terms": { 
      "terms": { 
      "field": "prop65", 
      "size": 1000 
      } 
     } 
     } 
    } 
    } 
} 

Answer 1

La raison du problème était en fait liée à la clause de filtre vide dans certains cas, car elle créait un avertissement pour chaque agrégation. Et Elastic Cloud avait une limite pour 64 avertissements - d'où l'erreur. (ils augmentent ce nombre maintenant)

Nous l'avons corrigé en ajoutant un match_all dans les cas où il était vide.

Answer 2

Quel est le but du conditionless « filtre » agrégation au-dessus de chaque agrégation « termes »? Si ce n'est pas juste pour simplifier l'exmaple que Il semble inutile. Peut-être que vous devriez essayer de supprimer ces aggs 'filter' et mettre tous les aggs 'terms' comme niveau supérieur à la place.