But one fine day, you face a subtle problem:

“How do I make sure no two inputs have the same value?”

That’s where this story begins.

The Setup

Let’s imagine a form where users can add their skills dynamically.

Each skill sits inside a FormGroup, and all such groups live inside a FormArray called userHobby.

userForm = new FormGroup({
  userHobby: new FormArray([], [Validators.required]),
});

When the user clicks Add New Row, we push a new FormGroup into the array:

addNewRow(): void {
  this.userHobbyControl.push(
    new FormGroup({
      skill: new FormControl<string | null>(null, [
        Validators.required,
        this.uniqueSkillValidator(),
      ]),
    })
  );
}

Nothing fancy yet — a normal reactive form with required validation.
The fun starts now.

The Problem — Detecting Duplicates
Let’s say the user adds:

• “Angular”

• “React”

• “Angular”

We want to catch that third input — it shouldn’t be allowed.
You’d think this is simple: just look for duplicates in the array and mark errors. But the challenge here is context.
Each validator in Angular runs in isolation — it only knows about its control, not its siblings.
So how do we make one control’s validator aware of the others?

The Trick — Closing Over the Parent

The key is closure — using a function that captures the parent context (this.userHobbyControl) while validating each control.

uniqueSkillValidator(): ValidatorFn {
  return (control: AbstractControl): ValidationErrors | null => {
    let skills: string[] = [];

    for (
      let index = 0;
      index <= this.userHobbyControl.controls.length;
      index++
    ) {
      const group = this.userHobbyControl.controls[index];
      const skill = group?.get('skill')?.value;
      if (skill) skills.push(skill);
    }

    if (this.hasDuplicates(skills)) {
      return { duplicate: true };
    }

    // Reset errors if no duplicates
    for (let group of this.userHobbyControl.controls) {
      group.get('skill')?.setErrors(null);
    }

    return null;
  };
}

Here, uniqueSkillValidator() creates a validator function that closes over the userHobbyControl FormArray.
This gives it visibility into all sibling controls — and that’s how we make validation context-aware.

Detecting Duplicates — The Set Trick

The check itself is beautifully simple:

hasDuplicates(skills: string[]) {
  const noDups = new Set(skills);
  return skills.length !== noDups.size;
}

This is a common pattern in JavaScript — using a Set to strip duplicates and comparing lengths.

The Validator Lifecycle

Every time the user types in any skill, Angular re-runs its validator.

Here’s what happens under the hood:

1. Collect all the skill values.

2. Check if any duplicates exist.

3. If yes, mark all the duplicates with { duplicate: true }.

4. If not, clear all duplicate errors.

The form instantly reacts — that’s the reactive in Reactive Forms.

Why This Works Beautifully

This approach looks deceptively simple, but it captures three deep ideas:

• Validation by closure — validators can be dynamic and context-aware.

• State synchronization — every keystroke re-validates all controls.

• Minimal logic, maximum clarity — the heart of good Angular design.

TL;DR

✅ Use FormArray for dynamic forms.
✅ Write a custom validator using closure to access sibling controls.
✅ Use Set to detect duplicates.
✅ Keep your form reactive — let Angular do the rest.

🧩 Try it yourself

You can explore the full working example on StackBlitz here:

High-Level Design

Before we jump into code, let us understand what we are trying to build. A scheduler has three main parts:

1. Scheduler Service – This is the main manager. It keeps a list of tasks, starts them, stops them, and can reload them if needed.

2. Task – A task is one job that we want to run again and again. For example, calling an API every 5 minutes.

3. Runnable Function – This is the actual code that the task will execute. We keep this separate so that the scheduler is generic and can run any kind of job.

The design is simple: the Scheduler Service holds multiple Tasks, and each task has a Runnable Function attached to it. The task uses a ticker and a goroutine to run at the correct interval, while the service makes sure tasks can be managed safely with locks.

Defining the Task Model

Each scheduled job is represented by a Task. A task should know:

• its ID (unique name or identifier),

• its interval (how often to run),

• its status (new, started, or running),

• and some internal fields to manage execution (like ticker, cancel function, etc.).

A simple version of the Task struct can look like this:

type taskStatus int

const (
    statusNew taskStatus = iota
    statusStarted
    statusRunning
)

type Task struct {
    ID        string
    Interval  time.Duration
    Status    taskStatus
    cancel    context.CancelFunc
    ticker    *time.Ticker
    lastRun   *time.Time
    fn        func(ctx context.Context) error
}

Here,

• fn is the function that will run when the task executes.

• ticker is used to schedule execution at fixed intervals.

• cancel is used to stop the task gracefully.

• lastRun helps in calculating the next run time.

This structure keeps the task generic, so it can run any kind of function we provide.

Scheduler Service

The Scheduler Service is the main manager. It keeps track of all tasks, starts them, stops them, and reloads them if needed. It also ensures that multiple tasks can run safely at the same time without conflicts.

A simple Scheduler struct can look like this:

type Scheduler struct {
    tasks map[string]*Task
    mtx   sync.Mutex
}

Here, tasks is a map of active tasks, and mtx is a mutex to protect access to this map.

Some core methods of the service are:

1. AddTask – Adds a new task to the scheduler and starts it.

2. StartTask – Runs a single task manually.

3. Reload – Stops tasks that are no longer needed and starts new ones.

4. ListTasks – Returns a list of all current tasks and their status.

For example, adding a task could look like this:

func (s *Scheduler) AddTask(t *Task) {
    s.mtx.Lock()
    defer s.mtx.Unlock()
    s.tasks[t.ID] = t
    t.start() // start the task immediately
}

The scheduler makes sure that all operations on tasks are thread-safe, so adding, removing, or running tasks does not cause race conditions.

Starting and Stopping Tasks

Each task runs in its own goroutine and uses a ticker to trigger the job at the right interval. We also use a cancellable context to stop the task gracefully when needed.

A simple way to start a task could be:

func (t *Task) start() {
    if t.Status >= statusStarted {
        return
    }

    ctx, cancel := context.WithCancel(context.Background())
    t.cancel = cancel
    t.ticker = time.NewTicker(t.Interval)
    t.Status = statusStarted

    go func() {
        for {
            select {
            case <-ctx.Done():
                t.ticker.Stop()
                t.Status = statusNew
                return
            case <-t.ticker.C:
                t.Status = statusRunning
                t.fn(ctx) // run the task function
                t.Status = statusStarted
            }
        }
    }()
}

To stop a task safely:

func (t *Task) stop() {
    if t.Status == statusRunning && t.cancel != nil {
        t.cancel()
    }
    if t.ticker != nil {
        t.ticker.Stop()
    }
}

We can also trigger a task manually, without waiting for the next ticker tick:

func (t *Task) runNow() {
    go t.fn(context.Background())
}

Key points:

• Each task has its own goroutine and ticker.

• The cancellable context allows the task to stop gracefully.

• Using mutexes (or status flags) ensures tasks do not run multiple times concurrently.

Scheduling Logic

A task is usually scheduled to run repeatedly at a fixed interval. For example, every 5 minutes, every hour, or every day. The scheduler needs to calculate when the next run should happen, based on the last run or a defined start time.

Here’s a simplified example:

func (t *Task) nextRun() time.Duration {
    if t.lastRun == nil {
        return t.Interval // first run uses the defined interval
    }

    next := t.lastRun.Add(t.Interval)
    now := time.Now()
    if next.Before(now) {
        // if we missed the scheduled time (system was down), fast-forward
        missedIntervals := (now.Sub(next) / t.Interval) + 1
        next = next.Add(missedIntervals * t.Interval)
    }

    return next.Sub(now)
}

Explanation:

• t.lastRun tracks when the task ran successfully last time.

• If lastRun is nil, it’s the first run, so we use the interval directly.

• If the next scheduled time is in the past (maybe the system was down), we fast-forward to the next valid time.

The scheduler uses this duration to reset the ticker, ensuring that tasks run at the correct intervals even after delays or downtime.

Key points:

• Recurrence calculation is critical to ensure tasks are reliable.

• The scheduler can handle missed runs automatically.

• By tracking lastRun, we can make the system resilient and predictable.

Concurrency and Safety

Tasks run in separate goroutines, and the scheduler manages multiple tasks at once. To avoid race conditions, we use mutexes (sync.Mutex or sync.RWMutex).

• The scheduler locks the task map when adding, removing, or listing tasks:

s.mtx.Lock()
defer s.mtx.Unlock()

• Each task uses its own mutex when updating status or last run time:

t.mtx.Lock()
t.Status = statusRunning
t.mtx.Unlock()

This ensures tasks do not run twice at the same time and that adding or removing tasks is safe, keeping the scheduler reliable.

Error Handling and Recovery

Tasks can fail or even panic while running. A good scheduler should catch errors, log them, and keep running other tasks without crashing.

• Wrap task execution in a recover block to handle panics:

go func() {
    defer func() {
        if r := recover(); r != nil {
            log.Println("Task panicked:", r)
        }
    }()
    err := t.fn(ctx)
    if err != nil {
        log.Println("Task error:", err)
    }
}()

• Log the start time, end time, and duration for monitoring and debugging.

• Keep track of last successful run to calculate the next run correctly.

By handling errors and panics, the scheduler becomes resilient and tasks continue running even if one fails.

Extending with Business Logic

The scheduler itself is generic — it does not care what the task actually does. To run real jobs, we attach a Runnable function to each task.

A Runnable is just a Go function with a simple signature:

type Runnable func(ctx context.Context) error

When adding a task, we pass this function:

task := &Task{
    ID:       "heartbeat",
    Interval: 10 * time.Second,
    fn: func(ctx context.Context) error {
        fmt.Println("Heartbeat at", time.Now())
        return nil
    },
}
scheduler.AddTask(task)

Now the scheduler can run this function repeatedly at the defined interval.

Key points:

• Decoupling task logic from the scheduler makes it reusable.

• Any job (API call, cleanup, email, etc.) can be scheduled the same way.

• Manual triggers and automatic scheduling work for all task types.

Practical Usage Examples

Once we have the scheduler and task structs ready, using them is simple.

1. Create the Scheduler

scheduler := &Scheduler{
    tasks: make(map[string]*Task),
}

2. Add a Task

task1 := &Task{
    ID:       "heartbeat",
    Interval: 5 * time.Second,
    fn: func(ctx context.Context) error {
        fmt.Println("Heartbeat at", time.Now())
        return nil
    },
}
scheduler.AddTask(task1)

3. Add Another Task

task2 := &Task{
    ID:       "cleanup",
    Interval: 10 * time.Second,
    fn: func(ctx context.Context) error {
        fmt.Println("Cleaning up old data at", time.Now())
        return nil
    },
}
scheduler.AddTask(task2)

4. Manual Trigger

You can run a task immediately without waiting for the next scheduled interval:

task1.runNow()

5. List All Tasks

for _, t := range scheduler.ListTasks() {
    fmt.Println(t.ID, "Status:", t.Status)
}

Key Points:

• Adding tasks is simple and flexible.

• Any function can be scheduled — API calls, cleanup jobs, logs, or emails.

• Manual triggers allow testing or one-off execution.

• The scheduler takes care of concurrency, next run calculation, and error handling.

Full Working Scheduler Code

package main

import (
    "context"
    "fmt"
    "sync"
    "time"
)

// ----------------------- Task -----------------------

type taskStatus int

const (
    statusNew taskStatus = iota
    statusStarted
    statusRunning
)

type Task struct {
    ID        string
    Interval  time.Duration
    Status    taskStatus
    cancel    context.CancelFunc
    ticker    *time.Ticker
    startTime time.Time
    lastRun   *time.Time
    fn        func(ctx context.Context) error
    mtx       sync.Mutex
}

// Start the task
func (t *Task) start() {
    t.mtx.Lock()
    if t.Status >= statusStarted {
        t.mtx.Unlock()
        return
    }

    ctx, cancel := context.WithCancel(context.Background())
    t.cancel = cancel
    t.ticker = time.NewTicker(t.Interval)
    t.Status = statusStarted
    t.mtx.Unlock()

    go func() {
        for {
            select {
            case <-ctx.Done():
                t.ticker.Stop()
                t.mtx.Lock()
                t.Status = statusNew
                t.mtx.Unlock()
                return
            case <-t.ticker.C:
                t.mtx.Lock()
                t.Status = statusRunning
                t.mtx.Unlock()

                defer func() {
                    if r := recover(); r != nil {
                        fmt.Println("Task panicked:", r)
                    }
                }()

                err := t.fn(ctx)
                if err != nil {
                    fmt.Println("Task error:", err)
                }

                now := time.Now()
                t.mtx.Lock()
                t.lastRun = &now
                t.Status = statusStarted
                t.mtx.Unlock()
            }
        }
    }()
}

// Stop the task
func (t *Task) stop() {
    t.mtx.Lock()
    defer t.mtx.Unlock()
    if t.cancel != nil {
        t.cancel()
    }
    if t.ticker != nil {
        t.ticker.Stop()
    }
    t.Status = statusNew
}

// Run the task immediately
func (t *Task) runNow() {
    go t.fn(context.Background())
}

// ----------------------- Scheduler -----------------------

type Scheduler struct {
    tasks map[string]*Task
    mtx   sync.Mutex
}

// AddTask adds and starts a task
func (s *Scheduler) AddTask(t *Task) {
    s.mtx.Lock()
    defer s.mtx.Unlock()
    s.tasks[t.ID] = t
    t.start()
}

// StopTask stops a task by ID
func (s *Scheduler) StopTask(id string) {
    s.mtx.Lock()
    defer s.mtx.Unlock()
    if t, ok := s.tasks[id]; ok {
        t.stop()
        delete(s.tasks, id)
    }
}

// ListTasks returns all tasks
func (s *Scheduler) ListTasks() []*Task {
    s.mtx.Lock()
    defer s.mtx.Unlock()
    tasks := []*Task{}
    for _, t := range s.tasks {
        tasks = append(tasks, t)
    }
    return tasks
}

// ----------------------- Example Usage -----------------------

func main() {
    scheduler := &Scheduler{
        tasks: make(map[string]*Task),
    }

    // Heartbeat task
    task1 := &Task{
        ID:       "heartbeat",
        Interval: 5 * time.Second,
        fn: func(ctx context.Context) error {
            fmt.Println("Heartbeat at", time.Now())
            return nil
        },
    }
    scheduler.AddTask(task1)

    // Cleanup task
    task2 := &Task{
        ID:       "cleanup",
        Interval: 10 * time.Second,
        fn: func(ctx context.Context) error {
            fmt.Println("Cleaning up old data at", time.Now())
            return nil
        },
    }
    scheduler.AddTask(task2)

    // Manual run example
    task1.runNow()

    // List all tasks
    for _, t := range scheduler.ListTasks() {
        fmt.Println("Task:", t.ID, "Status:", t.Status)
    }

    // Keep the program running
    select {}
}

Conclusion and Takeaways

Key lessons from this implementation:

• Keep the scheduler generic so it can run any job.

• Use goroutines and tickers to run tasks at intervals.

• Protect shared data with mutexes to avoid race conditions.

• Handle errors and panics to keep tasks running reliably.

• Track last run times to calculate next runs and recover from downtime.

• Decouple task logic from scheduler for flexibility and easier testing.

Note: This blog was created with assistance from ChatGPT

What is Chart.js and Why Use It?

Chart.js is an open-source JavaScript library that simplifies the creation of various types of charts, including bar, line, pie, radar, and scatter charts. Its flexibility, ease of use, and extensive customization options make it a preferred choice for developers aiming to add elegant visualizations to their projects.

Key features of Chart.js:

• Supports eight chart types out of the box.

• Offers animation and interaction capabilities.

• Highly customizable with plugins and configuration options.

• Responsive by default, ensuring compatibility across devices.

Why This Blog Series?

As a developer, I’ve often found it hard to get clear and detailed resources for Chart.js. While the documentation is useful, it sometimes lacks real-world examples or in-depth explanations. This inspired me to create this blog series—to share what I’ve learned and offer a practical guide for developers with similar challenges.

If you have tips, ideas, or improvements to share, I’d love for you to contribute. Together, we can build a helpful resource for the community and make life easier for developers.

You can find the full code and contributions on the repository: angular-chartjs.

Setting Up Chart.js in Angular

This series will guide you step-by-step on integrating Chart.js into your Angular application using ng2-charts, an Angular wrapper for Chart.js. For this series, we will use the following versions:

• Angular v16: The latest stable release with modern features and optimizations.

• Chart.js v4.4.0: A recent version of Chart.js with enhanced capabilities.

• ng2-charts v5.0.3: A powerful library that bridges Chart.js with Angular seamlessly.

Installation Steps Overview

The first post in this series will cover:

1. Installing Angular v16 (if not already set up).

2. Adding Chart.js and ng2-charts to your project:

    npm install chart.js@4.4.0 ng2-charts@5.0.3
   

Stay tuned as we explore how to build, customize, and enhance charts in Angular using Chart.js. Let’s make data visualization easier, one step at a time!
This blog series was created with the help of ChatGPT to ensure clear and concise explanations.

Streams provide a flexible and efficient way to handle data in Node.js, especially when dealing with large files or processing data in real-time. Throughout this blog, we will cover essential techniques, along with real-world examples, to help you fully grasp the potential of ReadStream and WriteStream. By the end of this journey, you will have a solid understanding of how to leverage streams for optimized file operations and data manipulation in your Node.js applications. Get ready to unlock the power of streams and take your Node.js skills to the next level!

What are Streams?

Streams in Node.js are like pipelines that enable us to read and write data from various sources. They make it easy to read data from files, network connections, or other sources, and write it to files, network connections, or other destinations. Streams are especially handy when working with large amounts of data since they process it in smaller, manageable chunks instead of all at once. They also offer a convenient way to transfer data between different sources and destinations without manual handling.

In simpler words, the stream is the flow of data from one source to another.

Types of stream

There are four fundamental stream types within Node.js:

• Writable: streams to which data can be written (for example, fs.createWriteStream()).

• Readable: streams from which data can be read (for example, fs.createReadStream()).

• Duplex: streams that are both Readable and Writable (for example, net.Socket).

• Transform: Duplex streams that can modify or transform the data as it is written and read (for example, zlib.createDeflate()).

This blog will explore the details of ReadStream and WriteStream, specifically highlighting their usage for file operations.

WriteStream in NodeJS

Writestream is a powerful stream type that enables developers to effortlessly write data to various destinations like files. With Writestream, you can seamlessly process substantial data volumes without concerns about memory consumption or performance bottlenecks. It offers a convenient solution for creating custom streams tailored to your application's needs.

Creating WriteStream

In this blog, we will explore the various events, properties, and methods offered by the WriteStream object.

To create a WriteStream in Node.js, we can use the createWriteStream function provided by the fs module. This function allows you to specify the filename you want to write to, and optionally, you can pass additional options as well.

const fs = require("fs");

// immediately invoked function expression (IIFE)
(() => {
    // creating writestream object
    const writeStream = fs.createWriteStream("destination.txt")
})()

If the destination.txt file does not exist then write stream will create the file for you.

To write data into a file using the WriteStream object, you can utilize the write method.

const fs = require("fs");

// immediately invoked function expression (IIFE)
(() => {
    // creating writestream object
    const writeStream = fs.createWriteStream("destination.txt")

    for (let index = 0; index < 100000; index++) {
        writeStream.write(`${index} `)
    }
})()

To properly handle the completion of writing data, it is important to close the WriteStream. We can achieve this by utilizing the close method on the WriteStream object. By invoking writeStream.close(), you ensure that any pending operations are finalized, resources are released, and the WriteStream is gracefully closed.

const writeStream = fs.createWriteStream("destination.txt")
writeStream.close()

When you call writeStream.write(data) in Node.js, the data is not immediately written directly into the file instead, it is initially written to a buffer. The buffer acts as a temporary storage area for the data. As more data is written, it accumulates in the buffer until it reaches its capacity.

Once the buffer is full, the entire contents of the buffer are written to the file. This process ensures that the data is efficiently written in chunks, reducing the number of disk operations and improving overall performance. It also helps in optimizing resource utilization, especially when dealing with large amounts of data.

By default, the buffer size for a WriteStream in Node.js is set to 16kB (16384 bytes), which is known as the highWaterMark value. However, you have the flexibility to modify this value according to your specific requirements.

To change the highWaterMark value when creating a WriteStream, you can use the fs.createWriteStream method and provide the desired value as an option. For example:

fs.createWriteStream("destination.txt", { highWaterMark: 400 });

In this example, the highWaterMark value is set to 400 bytes. Adjusting the highWaterMark allows you to control the buffer size and optimize the balance between memory consumption and the frequency of disk writes.

To avoid memory issues, it's important to empty the buffer before writing more data to a file. When the buffer becomes full, any additional data will be stored in memory until the buffer is cleared. Accumulating data in the buffer without emptying it can consume excessive memory, especially with large datasets, leading to performance and stability problems.

To avoid memory issues, the WriteStream object's write methods return a boolean indicating if the buffer is full. By pausing or stopping the data source while writing, we can prevent excessive data accumulation. The WriteStream also emits a "drain" event when the buffer is emptied and the data is successfully written to the file. By listening to this event, we can know when it's safe to resume reading from the source.

Here is an example of the same

const fs = require("fs");

// immediately invoked function expression (IIFE)
(() => {
    // creating writestream object
    const writeStream = fs.createWriteStream("destination.txt")

  let i = 0
  let MAX_WRITE = 10000000

  const writeToFile = () => {

    while (i < MAX_WRITE) {

      if (i === MAX_WRITE - 1) {
        return writeStream.end(`${i} `)
      }

      const isWrite = writeStream.write(`${i} `)

      if (!isWrite) break
      i++
    }
  }

  writeToFile()

  writeStream.on("drain", () => {
    console.log("Draining");
    writeToFile()
  })

  writeStream.on("finish", () => {
    console.log("finish event emitted")
    writeStream.close()
  })

})()

In the given example, a WriteStream object is created to write data to a file called "destination.txt". The purpose is to write a sequence of numbers from 0 to 10,000,000 in that file. To accomplish this, a function named writeToFile is defined, which handles the writing process.

Inside the writeToFile function, two methods provided by the WriteStream object are utilized. The write method is used to write each number to the stream, ensuring that the data is stored in the file. This method also returns a boolean value that indicates whether the buffer used for writing is full or not.

When all the numbers have been written, the end method is called. This signifies that it is the final write operation, allowing the WriteStream to complete any remaining writes and finalize the writing process. Upon finishing, the WriteStream emits a special event called finish, which can be utilized to perform any necessary tasks, such as gracefully closing the WriteStream.

ReadStream in NodeJS

Understanding the ReadStream object is indeed simpler once you are familiar with the WriteStream. Creating a ReadStream object in Node.js follows a similar approach to creating a WriteStream object.

Here is an example for creating readstream object.

const fs = require("fs")
const readStream = fs.createReadStream("src.txt");

Similar to the WriteStream object, the ReadStream object provides a range of events, properties, and methods. One important property of the ReadStream object is the highWaterMark, which specifies the size of the internal buffer used for reading data. In the case of the ReadStream, the default value of the highWaterMark property is 64kB (65536 bytes).

Readstream reads data from a source and stores it in a buffer. To access and work with this data, we can utilize the data event provided by the ReadStream.

const readStream = fs.createReadStream("../src.txt")

readStream.on("data", (chunk) => {
    console.log(chunk.toString());
})

By registering a listener for the data event using the on method, we can specify a function to be executed whenever new data becomes available. In the given example, the listener function takes a parameter called chunk, which represents a portion of the data read from the source. It's important to note that the data received through the data event is in bytes. To work with this data as a string, we can use the toString() function to convert it into a readable format.

In the ReadStream object, there is an end event that is triggered when all the data has been read from the source. This event serves as an indicator that the reading process is complete.

To gracefully close the ReadStream after reading is finished, we can use the close() method provided by the ReadStream object. Invoking readstream.close() allows us to properly terminate the ReadStream and release any associated resources.

By closing the ReadStream, we ensure that all operations related to reading data are finalized, preventing any potential memory leaks or unnecessary resource usage.

const readStream = fs.createReadStream("../src.txt")

readStream.on("data", (chunk) => {
    console.log(chunk); // output -> <Buffer 48 65 6c 6c 6f 20 57 6f 72 6c 64>
    console.log(chunk.toString());
})

readStream.on("end", () => {
    readStream.close()
})

In the ReadStream object, data is not read from the source until the data event is triggered on the ReadStream. This means that the ReadStream waits for the data event to occur before actively fetching and processing data from the source.

The ReadStream in Node.js offers two essential methods: pause and resume. These methods play a vital role when we need to control the reading process by pausing and resuming it.

The pause method does exactly what its name suggests - it pauses the ReadStream. When we invoke this method, it temporarily halts the reading process, giving us the ability to stop consuming data from the source. This can be useful in various situations, such as when we want to prioritize other tasks or when we need to handle data at a slower pace.

On the other hand, the resume method allows us to restart the ReadStream after it has been paused. Once we call this method, the ReadStream continues from where it left off, ensuring that we don't miss any data from the source. This is particularly valuable when we're ready to consume the data again or when we've finished performing a task that required the ReadStream to be paused.

Here is a basic example of using all the concepts that I have covered in this blog.

const basicReadWrite = async () => {
  const readStream = fs.createReadStream("../src.txt")
  const writeStream = fs.createWriteStream("../dest2.txt")

  writeStream.on("drain", () => {
    console.log("drain called");
    readStream.resume()
  })

  readStream.on("data", (chunk) => {
    if (!writeStream.write(chunk)) {
      readStream.pause()
    }
  })

  readStream.on("end", () => {
    readStream.close()
  })
}

basicReadWrite()

Conclusion

In conclusion, understanding the concepts of ReadStream and WriteStream in Node.js opens up a world of possibilities for handling large amounts of data efficiently. These powerful tools provide us with the means to read data from a file or stream it in real-time, as well as write data to a file or another stream. With their simple yet effective APIs, ReadStream and WriteStream make it easier than ever to handle data operations seamlessly.

By leveraging the capabilities of ReadStream, we can read data in chunks, enabling efficient memory usage and faster processing. This is particularly useful when dealing with large files or streaming data from external sources. The ability to handle data incrementally not only improves performance but also allows us to apply transformations or perform actions on the fly, making our applications more responsive and versatile. On the other hand, WriteStream empowers us to efficiently write data to a file or a stream, piece by piece. This is crucial when dealing with large data sets or when the data is being generated dynamically. By chunking the data and writing it in smaller portions, we can minimize memory consumption and optimize the overall process. Additionally, WriteStream provides convenient methods to handle events and ensure data integrity. When used together, ReadStream and WriteStream form a powerful duo, enabling seamless data processing pipelines. We can effortlessly read data from a ReadStream, perform any necessary operations or transformations, and then write the processed data to a WriteStream. This allows us to build efficient data pipelines, which are especially beneficial in scenarios where data needs to be transformed or transmitted in real-time.

Node.js has truly revolutionized the way we handle I/O operations, and ReadStream and WriteStream are prime examples of its capabilities. Whether you're working with large files, streaming data, or processing data in real-time, these features provide the flexibility and performance you need. So, embrace the power of ReadStream and WriteStream in Node.js, and unlock a whole new level of data processing capabilities. Start exploring their APIs, experimenting with different scenarios, and witness firsthand how they can enhance the efficiency, speed, and overall performance of your applications. Happy coding!