Explain the concept of code coverage and how it can be used to assess test quality

TL;DR

Code coverage reports which statements, branches, functions, or lines executed during a test run. Use it as a map for finding important behavior the suite never exercised, not as a score that proves test quality. A test can execute a line without asserting the result, while a low-value file can raise the percentage without reducing product risk. Vitest, Jest, and Istanbul-based tooling can collect JavaScript coverage.

What is code coverage?

Code coverage is a software testing metric that determines the amount of code that is executed during automated tests. It provides insights into which parts of the codebase are being tested and which are not.

Types of code coverage

Statement coverage: Measures the number of statements in the code that have been executed.
Branch coverage: Measures whether each branch (e.g., if and else blocks) has been executed.
Function coverage: Measures whether each function in the code has been called.
Line coverage: Measures the number of lines of code that have been executed.
Condition coverage: Measures whether each boolean sub-expression has been evaluated to both true and false.

Example

Consider the following JavaScript function:

function isEven(num) {
  if (num % 2 === 0) {
    return true;
  } else {
    return false;
  }
}

A test suite for this function might look like this:

test('isEven returns true for even numbers', () => {
  expect(isEven(2)).toBe(true);
});

test('isEven returns false for odd numbers', () => {
  expect(isEven(3)).toBe(false);
});

Running code coverage tools on this test suite would show 100% statement, branch, function, and line coverage because all parts of the code are executed.

How to measure code coverage

Tools

Istanbul: JavaScript coverage instrumentation and reporting used by many tools.
Vitest: Supports V8- or Istanbul-based coverage providers.
Jest: Includes coverage collection and reporting.

Example with Jest

To measure code coverage with Jest, you can add the --coverage flag when running your tests:

jest --coverage

This will generate a coverage report that shows the percentage of code covered by your tests.

Assessing test quality with code coverage

Practical use

Inspect uncovered branches in risky code such as authorization, payment, validation, and retries.
Add a regression test that first fails for a reported bug, then confirm that the fix passes.
Use a threshold to prevent accidental large drops when it suits the project, but allow reviewed exceptions for generated or environment-specific code.
Review whether assertions would fail for a meaningful defect. Mutation testing can provide another signal by changing code and checking whether tests detect the change.

Limitations

False sense of security: High coverage does not guarantee correct requirements, useful assertions, realistic test doubles, or integration behavior.
Optimization for the metric: A mandated percentage can encourage shallow tests and avoidance of difficult but important scenarios.
Instrumentation differences: Source maps, transpilation, ignored files, and the chosen provider can change reported numbers. Compare results produced by the same configuration.

Exercises

Beta