Deep Dive into Node.js Streams

Streams are built-in objects in Node.js that let you read data from a source or write data to a destination in a continuous fashion.

Why Streams Matter

Reading a 2GB file into RAM with fs.readFile() will overwhelm your application’s memory pool. Streams process data chunk-by-chunk, consuming a fraction of memory regardless of file size.

Types of Streams

• Readable: Streams from which data can be read (e.g., fs.createReadStream()).
• Writable: Streams to which data can be written (e.g., fs.createWriteStream()).
• Duplex: Streams that are both Readable and Writable (e.g., net socket).
• Transform: Duplex streams that modify or transform data as it is written and read (e.g., zlib compression).

Pipe Example

import { createReadStream, createWriteStream } from 'fs';
import { createGzip } from 'zlib';

createReadStream('large-file.log')
  .pipe(createGzip())
  .pipe(createWriteStream('large-file.log.gz'));

Best Practices

1. Always handle error events (stream.on('error', ...)) or use pipeline from stream/promises.
2. Avoid loading entire payloads in memory when handling large HTTP uploads or downloads.
3. Monitor backpressure to prevent memory bloat during slow stream writing.