Introduction to Batch Cluster

Batch clustering is a powerful method used to efficiently manage and process large sets of tasks in a distributed environment. By leveraging batch clusters, developers can optimize resource utilization, reduce processing times, and enhance the scalability and reliability of their applications. In this guide, we will introduce the concept of batch clustering and showcase dozens of useful API examples. Additionally, we will provide a practical app example that demonstrates the use of these APIs.

API Examples

1. Initializing a Batch Cluster

  const batch = require('batch-cluster');
  const cluster = batch.createCluster({
    name: 'ExampleCluster',
    maxWorkers: 4
  });

2. Adding Tasks to the Cluster

  cluster.addTask((cb) => {
    // your task logic here
    cb(null, 'result of the task');
  });

3. Handling Task Results

  cluster.run((err, results) => {
    if (err) {
      console.error('Error processing tasks:', err);
    } else {
      console.log('All tasks completed successfully:', results);
    }
  });

4. Gracefully Shutting Down the Cluster

  cluster.shutdown(() => {
    console.log('Cluster has been shut down.');
  });

5. Setting Up Custom Task Handlers

  cluster.on('taskComplete', (task, result) => {
    console.log(`Task ${task.id} completed with result: ${result}`);
  });
  
  cluster.on('taskError', (task, error) => {
    console.error(`Task ${task.id} encountered an error:`, error);
  });

6. Monitoring Cluster Health

  cluster.on('workerStart', (worker) => {
    console.log(`Worker ${worker.id} has started.`);
  });

  cluster.on('workerStop', (worker) => {
    console.log(`Worker ${worker.id} has stopped.`);
  });

App Example

Below is a simple example of an app that uses the batch-cluster APIs introduced above to perform batch processing tasks.

  const batch = require('batch-cluster');
  
  const cluster = batch.createCluster({
    name: 'ExampleCluster',
    maxWorkers: 4
  });

  for (let i = 0; i < 10; i++) {
    cluster.addTask((cb) => {
      // Simulating a time-consuming task
      setTimeout(() => {
        cb(null, `result of task ${i}`);
      }, 1000);
    });
  }

  cluster.run((err, results) => {
    if (err) {
      console.error('Error processing tasks:', err);
    } else {
      console.log('All tasks completed successfully:', results);
    }

    cluster.shutdown(() => {
      console.log('Cluster has been shut down.');
    });
  });

With these APIs, developers can efficiently manage tasks and improve the performance of their applications using batch processing.

Hash: a64687ffb22fa3fcd58ce3c642735d5e3bb6d5ebd8964f21be4bf8dcf326d433