Webstreams in Node.JS

Introduction

Consider, You ordered a pizza. Now There are two ways to eat pizza.

• First: A whole pizza at once.
• Second: Divide it into small pieces.

You will choose the Second way because It is easy and comfortable to chew and eat small pieces of pizza. In the same way, We use streams to fetch, process, and serve the data in small pieces. These small pieces are called chunks.

Therefore, We use streams to handle large amounts of data in chunks. The Source of this data could be any type like text file, javascript file, or server code. Webstreams in Node js allow us to use The WHATWG Streams Standard that is available as Web Streams API. We can use Web Stream API to read the data, feed the data, and transform the data. Generally, The node module stream/web is used to create streams and utilize Web Streams API.

Types of Streams

• Generally, There are three types of Webstreams in Node Js:
  • Readable Stream
  • Writable Stream
  • Transform Stream
• Now, We will create a folder nodejs-web-streams, go to the current folder and initiate a node project.

• Initially, Our folder structure looks as shown below.

Readable Stream

A readable Stream acts like a source of data that is available to read. We use the class ReadableStream to create an instance of the readable stream. ReadableStream has two parameters: underlyingSource and queuingStrategy . Both of these parameters are optional.

• underlyingSource: An object that decides how streams act and process the chunks. It has the following methods and properties.

  • start(controller): When we create an instance of a readable stream, the start method immediately gets executed. The parameter controller controls the data added to the internal queue.
  • pull(controller): This method works same as start(controller) method. But it gets called each time the stream's internal queue becomes empty to get more data if available.
  • cancel(reason): This method is called when we cancel the stream. We can also provide the reason for canceling the stream.
  • type: It could be bytes or undefined. The default value of type is set to undefined
  • autoAllocateChunkSize: It defines the total size of chunks to determine the size of AarrayBuffer. We must set the type property to bytes while using autoAllocateChunkSize.

• queuingStrategy: An object helps to control the flow of chunks in the stream and notify whether the internal queue is full. It includes the following property and methods:

  • highWaterMark: It indicates the highest limit of total chunks a stream can process. The Default Value of highWaterMark is set to 1.
  • size(): It calculates the size of each chunk and must return a number. The controller also uses it to calculate the value of the property desiredSize.

Accessible Properties and Methods of Readable Stream

• locked: We use the locked properties to know whether a readable stream is locked or not.
• cancel(): It uses the cancel() method of the underlyingSource object. We use the cancel() method to cancel a stream. The cancel() method returns an error If a stream is locked.
• getReader(): It is used to obtain a reader and lock the stream for the current reader. Therefore, We can not use another reader until we unlock the stream using the method releaseLock().
• tee(): It creates a copy of a readable stream and returns an array of two readable streams.
• pipeTo(): This method is used to pass the chunks of a readable stream to another stream. -pipeThrough(): It is used to pass the chunks of a readable stream to other streams and transform the data.

Readable Stream Example

1. Create Readable Stream

2. Read Readable Stream

3. Async Iterators

Writable Stream:

A writable stream acts as a destination for data that is available to write. We use the class WritableStream to create an instance of the writable stream. WritableStream has two parameters: underlyingSink and queuingStrategy. Both parameters are optional. underlyingSink is An object that decides how writable streams act and process the chunks. It has the following methods and properties.

-start(): When we create an instance of a writable stream, the start method immediately gets executed. The parameter controller controls the data written to the internal queue.

• write(): This method is called when the writer is ready to write to the stream.
• close(): this method is called if there is nothing to write to the stream.
• abort(): This method is called to cancel the stream. Even if there is any data to be written in the queue.

Accessible Properties and Methods of Writable Stream

• locked: We use the locked properties to know whether a writable stream is locked or not.
• getWriter(): We use this method to obtain the writer and write the data to stream
• close(): We call the close method when we finish writing to the stream.
• abort(): We call the abort method to interrupt the writable stream that throws the error even if data exist in the queue to be written.

Writable Stream Example

1. Create Writable Stream

2. Write to Writable Stream

Transform Stream

It can read and write data depending on the received output. If data comes from a readable stream, the transform stream helps write data to the stream, and if data comes from a writable stream, the transfer stream helps to read the data from the stream. transformer is used to handle readable and writable chunks

• start(controller): It is called immediately after creating an instance of Transform Stream. We use start() to read the data in the queue.
• transform(chunk, controller): We use this method to write and transform the chunk. It could be something like the conversion of chunk to uppercase.
• flush(controller): It is called when the transform stream transforms all the writable chunks.

Accessible Properties and Methods of Writable Stream

• readable: It is used to read chunks and access the readable side of the transform stream.
• writable is used to write chunks to the writable side of the transform stream and access the writable side of the transform stream.

Transform Stream Example

1. Create Transform Stream

2. Read and Write Transform Stream

Pipe Chains

Piping is a process that helps to pass the information from one stream to another stream. We can connect multiple streams with piping. And piping more than two streams is called pipe chains. We use the following methods for piping two or more streams:

• pipeTo()
• pipeThrough()

Example

Backpressure

When We apply pipe chains, there is a chance that data may overflow at the receiver end. Therefore, the receivers send the signal backward using the pipe chains. And this process of signal flow is called backpressure. For Example: In the above section, We used the following pipe chains.

• In the above code block, If writableStream gets full, the pipeTo() method receives a signal from writableStream and stops reading data from the readable side of transformSream.
• Now, The writable side of the transformStream gets overloaded with data and sends a signal to the pipeThrough() method. Then, the pipeThrough() method stops reading data from readableStream.

Web Streams in Node.js

• NodeJs module stream/web is used to support web stream, as we already used in this article.
• NodeJs provides built-in web stream support for Fetch API. We can use the response of the html body as a readable stream.

ByteStream