Streams in Node.js

Streams process data in chunks. Essential for handling large files without crashing your app.

The Problem

```javascript const fs = require('fs');

const data = fs.readFileSync('large-file.txt'); console.log(data); ```

If file is 1GB, this loads 1GB into memory. Your app crashes.

The Solution: Streams

```javascript const fs = require('fs');

const readStream = fs.createReadStream('large-file.txt');

readStream.on('data', (chunk) => { console.log(`Received ${chunk.length} bytes`); });

readStream.on('end', () => { console.log('Finished reading'); }); ```

Now it reads in small chunks (64KB by default). Memory usage stays low.

Real Example: Copy File

```javascript const fs = require('fs');

const readStream = fs.createReadStream('input.txt'); const writeStream = fs.createWriteStream('output.txt');

readStream.pipe(writeStream);

writeStream.on('finish', () => { console.log('File copied!'); }); ```

Real Example: Compress File

```javascript const fs = require('fs'); const zlib = require('zlib');

fs.createReadStream('input.txt') .pipe(zlib.createGzip()) .pipe(fs.createWriteStream('input.txt.gz')); ```

This compresses a file without loading it all into memory!

Real Example: HTTP Response

```javascript const express = require('express'); const fs = require('fs'); const app = express();

app.get('/download', (req, res) => { const fileStream = fs.createReadStream('large-video.mp4'); res.setHeader('Content-Type', 'video/mp4'); fileStream.pipe(res); });

app.listen(3000); ```

Video starts playing immediately. No need to wait for full download.

Transform Streams

```javascript const { Transform } = require('stream'); const fs = require('fs');

const uppercase = new Transform({ transform(chunk, encoding, callback) { this.push(chunk.toString().toUpperCase()); callback(); } });

fs.createReadStream('input.txt') .pipe(uppercase) .pipe(fs.createWriteStream('output.txt')); ```

Converts text to uppercase while streaming.

Error Handling

```javascript const { pipeline } = require('stream'); const fs = require('fs'); const zlib = require('zlib');

pipeline( fs.createReadStream('input.txt'), zlib.createGzip(), fs.createWriteStream('input.txt.gz'), (err) => { if (err) { console.error('Pipeline failed:', err); } else { console.log('Pipeline succeeded'); } } ); ```

`pipeline` handles errors automatically and cleans up streams.

Types of Streams

``` Readable → Read data (fs.createReadStream) Writable → Write data (fs.createWriteStream) Duplex → Both read and write (TCP socket) Transform → Modify data while streaming (zlib.createGzip) ```

Performance Comparison

``` Without Streams (1GB file): - Memory: 1GB - Time: 5 seconds - Risk: Out of memory crash

With Streams (1GB file): - Memory: 64KB - Time: 5 seconds - Risk: None ```

Key Takeaway

Streams handle large data efficiently by processing in chunks. Use `createReadStream` for reading, `pipe` to connect streams, `pipeline` for error handling. Perfect for large files, videos, and real-time data.