Lire 10 fichiers de manière asynchrone dans Spring Boot

Question

Le code ci-dessous montre que je lis un fichier à la fois dans un dossier. Je veux lire 10 fichiers à la fois, parcourir un fichier METHOD_A et écrire dans un autre dossier. Il y a 5000 fichiers dans le dossier. La lecture du fichier 1 à la fois est extrêmement lente. Je veux l'accélérer. J'utilise Java Spring Boot. Des suggestions comment je peux le faire?

for (int i = 0; i < files.length; i++){ 
    Object obj = parser.parse(new FileReader(files[i])); 
    JSONObject obj1 = METHOD_A(obj); 

     try{ 
      PrintWriter writer = new PrintWriter(...); 
      writer.println(obj1); 
      writer.close(); 
     } catch (IOException e) { 
      // do something 
    } 
} 

Merci d'avance.

Answer 1

Oui vous pouvez! , Exemple;

ExecutorService executorService = Executors.newFixedThreadPool(/*Number of Threads*/); 


    for (int i = 0; i < /* Number of Requests */; i++) { 
     WorkerThread wt = context.getBean(WorkerThread.class, String.valueOf(i)); 
     tasks.add(wt); 
    } 

    List<Future<String>> futures = executorService.invokeAll(tasks); 

    for (Future<String> future : futures) { 
     String result = future.get(10000, TimeUnit.MILLISECONDS); 
    } 

    executorService.shutdown(); 

Answer 2

Selon vos besoins, j'ai ajouté les codes ci-dessous pour vous. Essayez avec cela et voyez si vous travaillez pour vous. Gardez à l'esprit que votre serveur doit disposer d'un nombre suffisant de processeurs pour les processus concurrents. Si vous confondez encore voir un exemple de programme à la fin: -

private int numberOfFileProcessed = 0; 
    private int numberOfThreadAlive = 0 ; 
    private int numberOfThreadAlive = 0; 
    private int numberOfThreadToBeAllowed = 10;//Change this value to control number of thread to be run concurrently 

    for (int i = 0; i < files.length; i++){ 
    Object obj = parser.parse(new FileReader(files[i])); 
    JSONObject obj1 = METHOD_A(obj); 
    try{ 
      Thread t = new Thread(new ReadFiles(obj1)); 
      t.start(); 
      numberOfThreadAlive++; 
     }catch (Exception e) { 
     //do something 
     } 

     while(numberOfThreadAlive > numberOfThreadToBeAllowed){//This while loop will control number of thread to be not more than 10 
     try{Thread.sleep(100);}catch(Exception e){}//Release the processor  
     } 
    } 

    private final synchronized void jobCompleted(){  
      numberOfFileProcessed++; 
      numberOfThreadAlive--;  
    } 

    while(numberOfFileProcessed < files.length){ 
    //wait till last thread complete it's task 
    //I am not using thread.join() for performance 
    try{Thread.sleep(100);}catch(Exception e){}//Release the processor 
    } 



private class ReadFiles implements Runnable { 
     JSONObject jobj; 
     public ReadFiles(JSONObject obj) { 
      jobj = obj; 
     } 
     @SuppressWarnings("unchecked") 
     public void run() { 
      try{  
      PrintWriter writer = new PrintWriter(...); 
      writer.println(jobj); 
      writer.close(); 
      jobCompleted(); 
      } catch (IOException e) { 
      // do something 
      } 
     } 

    } 

est Ci-dessous un fichier de test que vous pouvez utiliser pour comprendre

 package com.test.threadtest; 

    public final class ThreadTest { 
     private int numberOfFileProcessed = 0; 
     private int numberOfThreadAlive = 0 ; 
     int numberOfThreadToBeAllowed = 10; 

     public void processFiles(){ 

      for (int i = 0; i < 50; i++){ 
       try{ 
         Thread t = new Thread(new ReadFiles(i)); 
         t.start(); 
         numberOfThreadAlive++; 
       }catch (Exception e) { 
        //do something 
       } 

        while(numberOfThreadAlive > numberOfThreadToBeAllowed){//This while loop will control number of thread to be not more than 10 
        try{Thread.sleep(100);}catch(Exception e){}//Release the processor 
        System.out.println("Reached maximum"); 
        } 
      } 
       while(numberOfFileProcessed < 50){ 
       //wait till last thread complete it's task 
       //I am not using thread.join() for performance 
        System.out.println("Number of file processed :" + numberOfFileProcessed); 
       try{Thread.sleep(100);}catch(Exception e){} 
       } 
     } 

     private final synchronized void jobCompleted(){  
       numberOfFileProcessed++; 
       System.out.println("numberOfFileProcessed :" + numberOfFileProcessed); 
       numberOfThreadAlive--;  
     } 

     public static void main(String[] args) { 
      // TODO Auto-generated method stub 
      ThreadTest test = new ThreadTest(); 
      test.processFiles(); 
      System.out.println("Exit from the process"); 
      System.exit(0); 
     } 

     private class ReadFiles implements Runnable { 
      int i; 
      public ReadFiles(int val) { 
       i = val; 
      } 
      @SuppressWarnings("unchecked") 
      public void run() { 
       try{  
       System.out.println("I am Thread : " + i); 
       Thread.sleep(1000); 
       jobCompleted(); 
       } catch (Exception e) { 
       // do something 
       } 
      } 

     } 
    }