The MongoDB Aggregation Pipeline Explained

The aggregation pipeline is how MongoDB transforms and summarizes data on the server, without pulling raw documents into your application and processing them there. If you come from SQL, it is the closest equivalent to GROUP BY, JOIN, and the analytical parts of SELECT combined, but the model is different: instead of one declarative statement, you build an ordered list of stages, and documents flow through them one stage at a time.

This guide covers the pipeline model, the stages you will use most, and the behaviors that trip people up.

How the pipeline works

You call aggregate() with an array of stages. Each stage receives the documents emitted by the previous stage, does one thing to them, and passes its output to the next stage.

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } }
])

Read it top to bottom: filter to shipped orders, group them by customer and sum the amounts, then sort by the total descending. The same document can pass through a stage more than once conceptually, but the mental model that always holds is: output of stage N is the input of stage N+1.

A stage is a single-key document where the key is the stage name (always prefixed with $) and the value is its specification. Some stages can appear multiple times in one pipeline ($match, $project, $group can all repeat).

The stages you will use most

$match: filter documents

$match uses the same query syntax as find(). It drops documents that do not match and passes the rest through unchanged.

{ $match: { status: "active", age: { $gte: 21 } } }

The single most important performance rule in the entire pipeline: place $match as early as possible. When $match is the first stage, MongoDB can use indexes to satisfy it, exactly as it would for a find(). Once a document has passed through a stage that reshapes it (like $group or $project), the index no longer applies, and a later $match becomes a full in-memory scan of the intermediate results.

$group: aggregate documents

$group collapses multiple documents into one per distinct value of the grouping key. The _id field defines that key, and it is mandatory.

{
  $group: {
    _id: "$category",
    count: { $sum: 1 },
    avgPrice: { $avg: "$price" },
    maxPrice: { $max: "$price" }
  }
}

{ $sum: 1 } counts documents (add one per document). { $sum: "$amount" } sums a field. To group by multiple fields, make _id a document:

{ $group: { _id: { region: "$region", year: "$year" }, total: { $sum: "$sales" } } }

To count every document into a single group, set _id to null:

{ $group: { _id: null, totalRevenue: { $sum: "$amount" } } }

The accumulator operators you will reach for most are $sum, $avg, $min, $max, $first, $last, $push (collect values into an array), and $addToSet (collect distinct values).

$project: reshape documents

$project controls which fields appear in the output and can compute new ones. Use 1 to include a field, 0 to exclude it.

{ $project: { _id: 0, name: 1, fullName: { $concat: ["$first", " ", "$last"] } } }

_id is the one field included by default; exclude it explicitly with _id: 0 if you do not want it. In modern MongoDB, $set (alias $addFields) is often clearer when you only want to add a computed field while keeping everything else, and $unset when you only want to drop a field.

$sort, $limit, $skip: order and paginate

{ $sort: { createdAt: -1 } },
{ $skip: 20 },
{ $limit: 10 }

1 is ascending, -1 is descending. A $sort immediately followed by a $limit is a special case the optimizer recognizes: it keeps only the top N in memory instead of sorting the whole set, which matters for large collections.

$lookup: join another collection

$lookup performs a left outer join. It matches documents from the current pipeline against a second collection and embeds the matches as an array.

{
  $lookup: {
    from: "customers",
    localField: "customerId",
    foreignField: "_id",
    as: "customer"
  }
}

The matched documents land in a new array field (customer here), even if there is exactly one match, so you commonly follow $lookup with $unwind: "$customer" to flatten it to a single embedded document. For join conditions beyond simple field equality, $lookup also supports a let/pipeline form.

$unwind: flatten arrays

$unwind turns one document containing an array into one document per array element.

{ $unwind: "$tags" }

A document with tags: ["a", "b", "c"] becomes three documents. By default, documents where the field is missing, null, or an empty array are dropped. To keep them, set preserveNullAndEmptyArrays: true:

{ $unwind: { path: "$tags", preserveNullAndEmptyArrays: true } }

This default is a frequent source of "rows disappeared" surprises after a $lookup.

A worked example

Suppose you want, for each shipped order, the customer name and the total spent per customer, top 5 customers only:

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 5 },
  { $lookup: {
      from: "customers",
      localField: "_id",
      foreignField: "_id",
      as: "customer"
  }},
  { $unwind: "$customer" },
  { $project: { _id: 0, customerName: "$customer.name", total: 1 } }
])

Note the order: filter and aggregate first to shrink the data, then join and reshape. Doing the $lookup before the $group would join far more documents than necessary.

Behaviors that catch people out

Stage order changes results, not just speed. $match before $group filters raw documents; $match after $group filters aggregated results. Both are valid and mean different things.

The 100 MB memory limit per stage. Pipeline stages that buffer data (notably $group and $sort) are capped at 100 MB of RAM. When a stage would exceed it, the query errors unless you set allowDiskUse: true, which lets it spill to temporary files on disk at the cost of speed.

db.orders.aggregate([ /* stages */ ], { allowDiskUse: true })

Field references need the $ prefix. Inside expressions, "$price" means "the value of the price field"; "price" is the literal string. Mixing these up produces silently wrong results rather than errors.

Only the first $match (and certain $sort and $geoNear stages) can use an index. Everything downstream operates on in-memory intermediate documents.

Inspecting and debugging a pipeline

Run explain to see how the optimizer rewrites and executes your pipeline, including whether the leading $match used an index:

db.orders.aggregate([ /* stages */ ], { explain: true })

When a pipeline returns unexpected results, the fastest way to debug is to truncate it: run only the first stage, check the output, then add stages back one at a time. Because each stage's output is just documents, you can inspect the intermediate result at any point.

Trying pipelines interactively

Building a multi-stage pipeline is iterative, and seeing intermediate output after each stage speeds it up considerably. Mako connects to MongoDB and offers AI-assisted query writing, which helps when you are translating a question into the right sequence of stages or recalling the exact accumulator syntax. Mako is a read and query tool, so it is suited to exploring and analyzing data rather than administering the deployment.

For loading data to experiment with, see our guides on importing CSV, JSON, and Excel into MongoDB.

Mako connects to MongoDB, PostgreSQL, MySQL, and more with AI-powered autocomplete. Try it free at mako.ai.