What are iterators and generators in JavaScript and what are they used for?

TL;DR

In JavaScript, iterators and generators are tools for managing sequences of data and controlling the flow of execution in a more flexible way.

Iterators are objects that define a sequence and potentially a return value upon its termination. They adhere to a specific interface:

An iterator object must implement a next() method.
The next() method returns an object with two properties:
- value: The next value in the sequence.
- done: A boolean that is true if the iterator has finished its sequence, otherwise false.

Here's an example of an object implementing the iterator interface.

const iterator = {
  current: 0,
  last: 5,
  next() {
    if (this.current <= this.last) {
      return { value: this.current++, done: false };
    } else {
      return { value: undefined, done: true };
    }
  },
};

let result = iterator.next();
while (!result.done) {
  console.log(result.value); // Logs 0, 1, 2, 3, 4, 5
  result = iterator.next();
}

Generators are special functions that can pause execution and resume at a later point. They use the function* syntax and the yield keyword to control the flow of execution. When you call a generator function, it doesn't execute completely like a regular function. Instead, it returns an iterator object. Calling the next() method on the returned iterator advances the generator to the next yield statement, and the value after yield becomes the return value of next().

function* numberGenerator() {
  let num = 0;
  while (num <= 5) {
    yield num++;
  }
}

const gen = numberGenerator();
console.log(gen.next()); // { value: 0, done: false }
console.log(gen.next()); // { value: 1, done: false }
console.log(gen.next()); // { value: 2, done: false }
console.log(gen.next()); // { value: 3, done: false }
console.log(gen.next()); // { value: 4, done: false }
console.log(gen.next()); // { value: 5, done: false }
console.log(gen.next()); // { value: undefined, done: true }

Generators are useful for creating iterators on demand, especially for infinite sequences or complex iteration logic. They can be used for:

Lazy evaluation – processing elements only when needed, improving memory efficiency for large datasets.
Implementing iterators for custom data structures.
Creating asynchronous iterators for handling data streams.

Generator-backed iteration lifecycle

A generator implements the iterator protocol by pausing at each yield and resuming only when its consumer asks for another result.

Generator and iterator lifecycle

Every next() call returns an iterator result with value and done; once completed, the iterator cannot resume normal execution.

Iterators

Iterators are objects that define a sequence and provide a next() method to access the next value in the sequence. They are used to iterate over data structures like arrays, strings, and custom objects. The key use cases of iterators include:

Implementing the iterator protocol to make custom objects iterable, allowing them to be used with for...of loops and other language constructs that expect iterables.
Providing a standard way to iterate over different data structures, making code more reusable and maintainable.

Creating a custom iterator for a range of numbers

In JavaScript, we can provide a default implementation for an iterator by implementing [Symbol.iterator]() in any custom object.

// Define a class named Range
class Range {
  // The constructor takes two parameters: start and end
  constructor(start, end) {
    // Assign the start and end values to the instance
    this.start = start;
    this.end = end;
  }

  // Define the default iterator for the object
  [Symbol.iterator]() {
    // Initialize the current value to the start value
    let current = this.start;
    const end = this.end;

    // Return an object with a next method
    return {
      // The next method returns the next value in the iteration
      next() {
        // If the current value is less than or equal to the end value...
        if (current <= end) {
          // ...return an object with the current value and done set to false
          return { value: current++, done: false };
        }

        // ...otherwise, return an object with value set to undefined and done set to true
        return { value: undefined, done: true };
      },
    };
  }
}

// Create a new Range object with start = 1 and end = 3
const range = new Range(1, 3);
// Iterate over the range object
for (const number of range) {
  // Log each number to the console
  console.log(number); // 1, 2, 3
}

Built-in objects using the iterator protocol

In JavaScript, several built-in objects implement the iterator protocol, meaning they have a default @@iterator method. This allows them to be used in constructs like for...of loops and with the spread operator. Here are some of the key built-in objects that implement iterators:

Arrays: Arrays have a built-in iterator that allows you to iterate over their elements.

const array = [1, 2, 3];
const iterator = array[Symbol.iterator]();

console.log(iterator.next()); // { value: 1, done: false }
console.log(iterator.next()); // { value: 2, done: false }
console.log(iterator.next()); // { value: 3, done: false }
console.log(iterator.next()); // { value: undefined, done: true }

for (const value of array) {
  console.log(value); // Logs 1, 2, 3
}

Strings: Strings have a built-in iterator that allows you to iterate over their characters.

const string = 'hello';
const iterator = string[Symbol.iterator]();

console.log(iterator.next()); // { value: "h", done: false }
console.log(iterator.next()); // { value: "e", done: false }
console.log(iterator.next()); // { value: "l", done: false }
console.log(iterator.next()); // { value: "l", done: false }
console.log(iterator.next()); // { value: "o", done: false }
console.log(iterator.next()); // { value: undefined, done: true }

for (const char of string) {
  console.log(char); // Logs h, e, l, l, o
}

DOM NodeLists

// Create a new div and append it to the DOM
const newDiv = document.createElement('div');
newDiv.id = 'div1';
document.body.appendChild(newDiv);

const nodeList = document.querySelectorAll('div');
const iterator = nodeList[Symbol.iterator]();

console.log(iterator.next()); // { value: HTMLDivElement, done: false }
console.log(iterator.next()); // { value: undefined, done: true }

for (const node of nodeList) {
  console.log(node); // Logs each <div> element, in this case only div1
}

Maps and Sets also have built-in iterators.

Generators

Generators are a special kind of function that can pause and resume their execution, allowing them to generate a sequence of values on-the-fly. They are commonly used to create iterators but have other applications as well. The key use cases of generators include:

Creating iterators in a more concise and readable way compared to manually implementing the iterator protocol.
Implementing lazy evaluation, where values are generated only when needed, saving memory and computation time.
Simplifying asynchronous programming by allowing code to be written in a synchronous-looking style using yield and await.

Generators provide several benefits:

Lazy evaluation: They generate values on the fly and only when required, which is memory efficient.
Pause and resume: Generators can pause execution (via yield) and can also receive new data upon resuming.
Asynchronous iteration: With the advent of async/await, generators can be used to manage asynchronous data flows.

Creating an iterator using a generator function

We can rewrite our Range example to use a generator function:

// Define a class named Range
class Range {
  // The constructor takes two parameters: start and end
  constructor(start, end) {
    // Assign the start and end values to the instance
    this.start = start;
    this.end = end;
  }

  // Define the default iterator for the object using a generator
  *[Symbol.iterator]() {
    // Initialize the current value to the start value
    let current = this.start;

    // While the current value is less than or equal to the end value...
    while (current <= this.end) {
      // ...yield the current value
      yield current++;
    }
  }
}

// Create a new Range object with start = 1 and end = 3
const range = new Range(1, 3);
// Iterate over the range object
for (const number of range) {
  // Log each number to the console
  console.log(number); // 1, 2, 3
}

Iterating over data streams

Generators are well-suited for iterating over data streams, such as fetching data from an API or reading files. This example demonstrates using a generator to fetch data from an API in batches:

async function* fetchDataInBatches(
  url,
  { numBatches = 5, batchSize = 10, signal } = {},
) {
  let startIndex = 0;
  let currBatch = 0;

  while (currBatch < numBatches) {
    const endpoint = new URL(url, location.origin);
    endpoint.searchParams.set('start', startIndex);
    endpoint.searchParams.set('limit', batchSize);

    const response = await fetch(endpoint, { signal });
    if (!response.ok) throw new Error(`Request failed (${response.status})`);

    const data = await response.json();
    if (data.length === 0) break;
    yield data;
    startIndex += batchSize;
    currBatch += 1;
  }
}

async function fetchAndLogData() {
  const controller = new AbortController();
  const dataGenerator = fetchDataInBatches('/api/todos', {
    signal: controller.signal,
  });

  try {
    for await (const batch of dataGenerator) {
      console.log(batch);
    }
  } finally {
    controller.abort();
  }
}
fetchAndLogData();

This generator function fetchDataInBatches fetches data from an API in batches of a specified size. It yields each batch of data, allowing you to process it before fetching the next batch. This approach can be more memory-efficient than fetching all data at once.

Implementing asynchronous iterators

Generators can be used to implement asynchronous iterators, which are useful for working with asynchronous data sources. This example demonstrates an asynchronous iterator for fetching data from an API:

async function* fetchDataAsyncIterator(url, pagesToFetch = 3, { signal } = {}) {
  let currPage = 1;

  while (currPage <= pagesToFetch) {
    const endpoint = new URL(url, location.origin);
    endpoint.searchParams.set('page', currPage);
    const response = await fetch(endpoint, { signal });
    if (!response.ok) throw new Error(`Request failed (${response.status})`);
    const data = await response.json();
    if (data.length === 0) break;
    yield data;
    currPage++;
  }
}

async function fetchAndLogData() {
  const asyncIterator = fetchDataAsyncIterator('/api/todos');

  for await (const chunk of asyncIterator) {
    console.log(chunk);
  }
}
fetchAndLogData();

The generator function fetchDataAsyncIterator is an asynchronous iterator that fetches data from an API in pages. It yields each page of data, allowing you to process it before fetching the next page. This approach can be useful for handling large datasets or long-running operations.

Generators are also used extensively in JavaScript libraries and frameworks, such as Redux-Saga and RxJS, for handling asynchronous operations and reactive programming.

Summary

Iterators define a standard protocol for traversing data sequences. Generators implement iterators by suspending and resuming at yield; they can also be used to coordinate asynchronous operations and compose data pipelines.

Exercises

Beta