How do you design large file uploads as streams in an HTTP server without storing the file in memory?

Short answer

Pipe the HTTP request stream (req) directly through a pipeline to a disk file or cloud storage (S3) as a multipart stream as its chunks arrive.

Explanation

Loading the entire file into memory can crash the server as soon as a large file is uploaded. The right design treats the request body req as a Readable Stream, passes it to a multipart parser, and routes it directly to the final storage destination.

Code example

await pipeline(req, fs.createWriteStream("file.bin"));

Common mistakes

• Storing the file contents in in-memory variables before writing them to disk.

Follow-up questions

• How do you create and customize a Transform Stream to process data as it flows through Node.js?

Sources

• Node.js docs — http.IncomingMessage
• Node.js docs — Stream: stream.pipeline()