comment grouper wso2am-analytics-2.0.0

Question

Nous construisons wso2am en mode cluster. Existe-t-il un document sur la construction du cluster wso2am-analytics? J'ai essayé d'utiliser wso2das, référence comme ci-dessous. https://docs.wso2.com/display/DAS310/Working+with+Product+Specific+Analytics+Profiles

Mais obtenir l'erreur comme ci-dessous

TID: [-1234] [] [2016-12-09 15:00:00,101] ERROR {org.wso2.carbon.analytics.spark.core.AnalyticsTask} - Error while executing the scheduled task for the script: APIM_LATENCY_BREAKDOWN_STATS {org.wso2.carbon.analytics.spark.core.AnalyticsTask} 
org.wso2.carbon.analytics.spark.core.exception.AnalyticsExecutionException: Exception in executing query CREATE TEMPORARY TABLE APIMGT_PERHOUR_EXECUTION_TIME USING CarbonAnalytics OPTIONS(tableName "ORG_WSO2_APIMGT_STATISTICS_PERHOUREXECUTIONTIMES", schema " year INT -i, month INT -i, day INT -i, hour INT -i, context STRING, api_version STRING, api STRING, tenantDomain STRING, apiPublisher STRING, apiResponseTime DOUBLE, securityLatency DOUBLE, throttlingLatency DOUBLE, requestMediationLatency DOUBLE, responseMediationLatency DOUBLE, backendLatency DOUBLE, otherLatency DOUBLE, firstEventTime LONG, _timestamp LONG -i", primaryKeys "year, month, day, hour, context, api_version, tenantDomain, apiPublisher", incrementalProcessing "APIMGT_PERHOUR_EXECUTION_TIME, DAY", mergeSchema "false") 
     at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:764) 
     at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQuery(SparkAnalyticsExecutor.java:721) 
     at org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeQuery(CarbonAnalyticsProcessorService.java:201) 
     at org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeScript(CarbonAnalyticsProcessorService.java:151) 
     at org.wso2.carbon.analytics.spark.core.AnalyticsTask.execute(AnalyticsTask.java:60) 
     at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67) 
     at org.quartz.core.JobRunShell.run(JobRunShell.java:213) 
     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
     at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: Unknown options : incrementalprocessing 
     at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.checkParameters(AnalyticsRelationProvider.java:123) 
     at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.setParameters(AnalyticsRelationProvider.java:113) 
     at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.createRelation(AnalyticsRelationProvider.java:75) 
     at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.createRelation(AnalyticsRelationProvider.java:45) 
     at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158) 
     at org.apache.spark.sql.execution.datasources.CreateTempTableUsing.run(ddl.scala:92) 
     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58) 
     at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56) 
     at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70) 
     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132) 
     at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130) 
     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) 
     at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130) 
     at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55) 
     at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55) 
     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:145) 
     at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:130) 
     at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52) 
     at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817) 
     at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:760) 
     ... 11 more 

=============================== =======================

Toute suggestion sera appréciée !!

Answer 1

Juste pour être clair de ce que vous demandez. Utilisez-vous DAS ou les analyses wso2am fournies avec wso2am-2.0.0?

Si vous utilisez wso2am-2.0.0 avec le wso2das-3.1.0, il y a un problème avec les scripts d'analyse d'étincelles fournis avec le DAS.

Ceci est dû à l'utilisation de incremental processing. IncrementalProcessing devrait être changé en IncrementalParams.

Vous pouvez voir que cela a été corrigé par wso2 here, mais n'a pas encore été publié.

Vous pouvez mettre à jour les scripts de la console de carbone DAS sous principaux/batch Analytics/scripts

Answer 2

TKS ur réponse !! Nous avons besoin de conserver les données résumées dans statdb, donc quelle table doit être purgée? Je l'ai trouvé un lien ci-dessous http://www.rukspot.com/Publishing_APIM_1100_Runtime_Statistics_to_DAS.html Il a mentionné que juste purge tableau ci-dessous

ORG_WSO2_APIMGT_STATISTICS_DESTINATION

ORG_WSO2_APIMGT_STATISTICS_FAULT

ORG_WSO2_APIMGT_STATISTICS_REQUEST

ORG_WSO2_APIMGT_STATISTICS_RESPONSE

ORG_WSO2_APIMGT_STATISTICS_WORKFLOW

ORG_WSO2_APIMGT_STATISTICS_THROTTLE