Introduction to Read-chunk

Read-chunk is a versatile module used for reading small chunks of a file asynchronously and synchronously. It is especially useful when dealing with large files to avoid loading the entire file into memory. In this comprehensive guide, we’ll explore the various APIs provided by read-chunk with code snippets and a practical app example.

API Examples

Basic Asynchronous Usage

Reading a chunk starting from a specific position using async/await.

const readChunk = require('read-chunk');

async function readFileChunk(filePath, start, length) {
    const buffer = await readChunk(filePath, start, length);
    console.log(buffer);
}

readFileChunk('example.txt', 0, 1024);

Basic Synchronous Usage

Similar to the asynchronous call, but using the synchronous API.

const readChunk = require('read-chunk');

function readFileChunkSync(filePath, start, length) {
    const buffer = readChunk.sync(filePath, start, length);
    console.log(buffer);
}

readFileChunkSync('example.txt', 0, 1024);

Reading Different Parts of a File

You can read different chunks of a file by specifying the start position and length of each chunk.

const readChunk = require('read-chunk');

async function readMultipleChunks(filePath) {
    const firstChunk = await readChunk(filePath, 0, 1024);
    const secondChunk = await readChunk(filePath, 1024, 1024);
    console.log(firstChunk);
    console.log(secondChunk);
}

readMultipleChunks('example.txt');

Practical Example: Read a Large Log File

Let’s build a practical example that reads a large log file in chunks to look for a specific keyword.

const fs = require('fs');
const readChunk = require('read-chunk');

async function searchKeywordInLog(filePath, keyword) {
    const stats = fs.statSync(filePath);
    const fileSize = stats.size;
    const chunkSize = 1024;
    let position = 0;

    while (position < fileSize) {
        const chunk = await readChunk(filePath, position, chunkSize);
        const chunkStr = chunk.toString('utf8');
        if (chunkStr.includes(keyword)) {
            console.log(`Keyword "${keyword}" found in chunk starting at position ${position}`);
        }
        position += chunkSize;
    }
}

searchKeywordInLog('large_log.txt', 'ERROR');

This example demonstrates how you can read a large log file chunk by chunk to search for a specific keyword without loading the entire file into memory, making it memory efficient and fast.

With these examples, you can start leveraging the power of read-chunk in your Node.js applications, enabling efficient file handling for large datasets.

Hash: 4fde24556426ab528b088d2e8a4239555f5329d121ca0223c0e4b1369d439427