The String Decoder Module

When binary data streams in over the network in small pieces, converting each chunk to a string with .toString() seems reasonable — until a multi-byte character happens to get split across two chunks. string_decoder exists specifically to handle that edge case correctly.

The problem it solves

Some characters — many non-English letters, emoji — take more than one byte in UTF-8 encoding. If a network chunk boundary happens to fall in the middle of one of those multi-byte characters, calling .toString() on that incomplete chunk alone produces a broken, garbled character.

const incompleteChunk = Buffer.from([0xE0, 0xA6]); // first 2 of 3 bytes for a Bengali character
console.log(incompleteChunk.toString('utf8')); // prints replacement characters — broken

How StringDecoder fixes it

const { StringDecoder } = require('string_decoder');
const decoder = new StringDecoder('utf8');

const chunk1 = Buffer.from([0xE0, 0xA6]);       // incomplete
const chunk2 = Buffer.from([0xAC]);              // completes the character

let output = '';
output += decoder.write(chunk1); // holds back the incomplete bytes internally, returns ''
output += decoder.write(chunk2); // now has a complete character, returns it correctly

console.log(output); // the correctly assembled character

The decoder keeps a small internal buffer of any incomplete bytes from the previous .write() call, and combines them with the next chunk before decoding — silently handling the split for you.

Finishing a stream

console.log(decoder.end()); // flushes and returns any remaining buffered bytes as a string

Call .end() once you know no more chunks are coming, to make sure nothing gets left behind unprocessed.

A realistic use — decoding a readable stream correctly

const { StringDecoder } = require('string_decoder');
const fs = require('fs');

const decoder = new StringDecoder('utf8');
const readStream = fs.createReadStream('bengali-notes.txt');

readStream.on('data', (chunk) => {
  process.stdout.write(decoder.write(chunk));
});

readStream.on('end', () => {
  process.stdout.write(decoder.end());
});

When you actually need this

If you're reading an entire file at once with fs.readFile(path, 'utf8', callback), Node.js already handles encoding correctly for you — no string_decoder needed. It matters specifically when you're processing binary data incrementally, chunk by chunk, from a stream or socket, and converting each piece to text as it arrives rather than waiting for everything to finish first.