Understanding the Cost of a Shuffle

In F03.1, we learned how a distributed aggregation calculates partial results in map tasks and combines them in a reduce stage. Now we will examine the performance cost of moving those partial results between tasks. Our nightly processing job needs to calculate the number of events generated by each customer. The dataset contains 60 million events, distributed across 12 map tasks running on a cluster with:

• 12 workers
• 4 nodes
• 12 map tasks
• 12 reduce tasks

For every event, the map stage produces a record containing the customer ID and a count of 1. For example:

The reduce stage must bring together records belonging to the same customer and add their counts. However, there’s a problem.

Every Input Record Becomes an Intermediate Record

Without local aggregation, each of the 60 million events produces one intermediate record These intermediate records must be written by the map tasks and made available to the appropriate reduce tasks. Since records belonging to the same customer are spread across the cluster, many intermediate records must travel between nodes.

In our reference model:

• 60 million intermediate records are generated.
• 1.08 GB of intermediate data is written.
• 0.81 GB is transferred between nodes.

And the data movement isn’t the only problem. One reduce task receives more than 10 million intermediate records, significantly more than the average reduce task. Even after most reduce tasks finish, the job must wait for this overloaded task.

Why Does This Matter?

A shuffle can introduce two important performance costs:

1. 01Data movement: Intermediate records must be written, transferred, and read by reduce tasks.
2. 02Uneven processing: Some reduce tasks may receive substantially more records than others, creating stragglers.

These costs can increase execution time, memory usage, disk I/O, and network traffic. But do we really need to transfer one intermediate record for every input event? If a map task already sees several events belonging to the same customer, perhaps it could combine those events before sending anything across the cluster. That’s the idea behind a combiner, which we’ll explore shortly.

Let’s understand how much data our shuffle produces, how that data moves between nodes, and why one reduce task becomes a bottleneck.

Step 1: Calculate the Intermediate Data Size

Our job counts events per customer. Without a combiner, every input event produces one intermediate record containing:

• Customer ID: 10 bytes in our model.
• Count: 8 bytes.

Therefore, each intermediate record occupies: 10 + 8 = 18 bytes
Since our nightly job processes 60 million events, the total intermediate data written is:
60,000,000 × 18 = 1,080,000,000 bytes

That’s approximately 1.08 GB. This data is written by the 12 map tasks to their local disks before being read by the reduce tasks.

Step 2: How Much Data Moves Across the Network?

Our cluster has four nodes, with map and reduce tasks distributed across them. Each reduce task is responsible for a subset of customers, determined by the partitioning rule. Since records belonging to a customer may originate from map tasks on different nodes, the reduce task must retrieve its intermediate data from those locations. Some data is already stored on the reduce task’s node and can be read locally. Other data is stored on different nodes and must be transferred over the network. Our simulator assumes that approximately 75% of the shuffle data is read remotely.

Therefore: Network transfer = 1.08 GB × 0.75 = 0.81 GB
The remaining 0.27 GB is read locally.

Using the transfer price from F02.3: Transfer cost = 0.81 × 0.25 ≈ 0.20 units
This is the network cost of the shuffle alone.

Our distributed job:

• Processes 60 million events.
• Produces one 18-byte intermediate record per event.
• Writes 1.08 GB of intermediate data.
• Runs on a four-node cluster where approximately 75% of shuffle data must be fetched from other nodes.

Step 3: Why Does One Reduce Task Take Longer?

Now let’s examine the reduce stage. We have 12 reduce tasks processing 60 million intermediate records. If the records were distributed perfectly evenly, each reduce task would receive: 60,000,000 ÷ 12 = 5,000,000 records

Our simulator assumes that reading and merging one intermediate record takes approximately 3 µs. So processing 5 million records requires roughly: 5,000,000 × 3 µs = 15 seconds
Including task overhead, the average reduce-task duration is approximately 16.1 seconds. However, the data is not evenly distributed. One reduce task receives 10,746,904 intermediate records, more than twice the average. That task takes approximately 34 seconds to complete.

Understanding Reduce Skew

As we learned in F02, skew measures how unevenly work is distributed. For the reduce stage:
Reduce skew = Largest reduce-task input ÷ Average reduce-task input

Substituting our values: 10,746,904 ÷ 5,000,000 ≈ 2.15

The skew factor is 2.15, meaning the largest reduce task receives approximately 2.15 times the average amount of data.

Why Is This Task So Large?

The overloaded reduce task owns several high-volume customers. One of them is cust-0001.
In the larger simulated workload, this customer contributes 3,576,629 events.

Because all records belonging to the same customer must reach the same reduce task, those events cannot simply be distributed among different reduce tasks under the current design. Other customers also happen to hash to the same reduce partition. As a result, the task receives more than 10.7 million intermediate records. This is the same data-skew problem we studied in F02, but it now occurs after the shuffle, in the reduce stage.

Step 4: How Does This Affect the Job?

Since the reduce stage must wait for its slowest task, the overloaded task determines when the stage finishes. Our simulator reports:

Even though most reduce tasks receive fewer records, the job cannot finish until the overloaded task completes. This raises an important question: Can we reduce the number of intermediate records before they reach the shuffle?

We’ll explore that next.

So far, our map tasks have produced one intermediate record for every input event. But consider what happens when a map task processes several events belonging to the same customer. For example, suppose a map task reads:

Without a combiner, the task produces four intermediate records. But we don’t need to send three separate records for cust-001. We can combine them locally:

Now the map task produces only two intermediate records instead of four. This is called map-side aggregation, commonly implemented using a combiner. The combiner maintains one running count per customer within each map task. Instead of emitting one record for every event, it emits one partial count for every customer encountered by that task.

What Happens in Our Dataset?

Let’s use our smaller fixture of 21,154 events from 797 customers, distributed across 12 map tasks.
Without a combiner: 21,154 events → 21,154 intermediate records
With a combiner: 21,154 events → 6,370 intermediate records

Why 6,370 rather than 797? Because the same customer can appear in multiple map tasks. Each map task produces one partial count for a customer it encounters, but it cannot directly combine counts held by other map tasks. For example, cust-0001 appears in all 12 tasks. Without a combiner, it generates 1,261 intermediate records in the fixture. With a combiner, it generates only 12 partial counts, one from each map task. The reduce stage then merges those twelve counts into the customer’s final event count.

Measuring Combiner Gain

We can measure how effectively a combiner reduces intermediate data using:
Combiner gain = Input records ÷ Intermediate records after combining

For our dataset: Combiner gain = 21,154 ÷ 6,370 ≈ 3.32

This means the combiner reduces the number of intermediate records by a factor of approximately 3.32. The gain depends on how frequently the same key appears within a map task. If each map task encounters many events from the same customer, local aggregation can greatly reduce the output. If nearly every record has a unique key, the combiner may provide little or no reduction.

How Does the Combiner Improve Reduce Performance?

Previously, our largest reduce task received more than 10.7 million intermediate records. With the combiner enabled, that input falls to approximately 1.8 million partial records. Why? Because repeated events belonging to the same customer are combined within their map tasks before the shuffle. This substantially reduces the amount of data the reduce tasks must read and merge. The reduce skew factor also decreases:

• Without combiner: 2.15
• With combiner: 1.17

As a result:

• The reduce stage decreases from 34.0 seconds to 6.0 seconds.
• Total job runtime decreases from 237.4 to 206.2 seconds.
• Total run cost decreases from 2.58 to 2.12 units.

The combiner improves performance by reducing both shuffle data volume and reduce-task workload.

Does a Combiner Always Help?

Not necessarily. The benefit depends on how many records share the same key within each map task. Let’s compare different aggregation keys using our smaller fixture.

Notice the difference.
Aggregating by customer_id: Many events belong to the same customer, so map tasks can combine several records into one partial result. This produces a substantial reduction.

Aggregating by order_id: Fewer records share the same order ID within each task, so the combiner provides a smaller benefit.

Aggregating by event_id: Each event has its own unique ID, so there is nothing to combine locally. The combiner produces the same number of intermediate records as the input.

In this case, maintaining an additional aggregation structure may introduce overhead without reducing shuffle data.

What Should We Learn from This?

A combiner can significantly improve distributed aggregation performance when multiple records share the same key within a map task. However, its effectiveness depends on the actual data distribution and aggregation operation.

Before enabling or designing local aggregation, consider:

• How often does the same key appear within a map task?
• Can the intermediate state be combined safely?
• How much intermediate data will local aggregation eliminate?
• Does the reduction justify the additional local processing and memory usage?

As discussed in F03.1, the intermediate state must support correct merging. For example, sums and counts can be combined directly, while averages require maintaining both sum and count.

One technical distinction: our simulator explicitly performs local aggregation when the combiner is enabled. In some distributed frameworks, a combiner is an optional optimisation and may not execute on every run. Correctness must not depend on it being invoked.

In the previous section, we learned how a combiner reduces shuffle data and improves reduce-stage performance. But suppose the team decides not to use a combiner. Instead, they propose increasing the number of reduce tasks to distribute the workload more evenly.
Currently, our nightly job has:

• 60 million intermediate records
• 12 reduce tasks
• 10,746,904 records assigned to the largest reduce task
• 34 seconds for the reduce stage
• 237.4 seconds of total runtime

The largest reduce task processes more than twice the average number of records. The team believes that increasing the number of reduce tasks from 12 to 48 will divide the workload into smaller pieces and eliminate the bottleneck. But will it?

Why Didn’t More Reduce Tasks Eliminate the Bottleneck?

The answer lies in how hash partitioning works. Our reduce tasks use the following rule:
Reduce partition = stable_hash(customer_id) % Number of reduce tasks

Every record belonging to the same customer must reach the same reduce task. Consider cust-0001. This customer generates 3,576,629 records in our scaled workload. Whether we have 12, 48, or 128 reduce tasks, all those records still belong to one reduce partition because they share the same customer_id. Increasing the number of reduce tasks can move other customers into different partitions, but it does not split the records belonging to cust-0001. This creates a limit on how much the largest task can shrink.

Why Did the Skew Factor Increase?

Notice something surprising in the results:

• 12 reduce tasks: Skew factor of 2.15.
• 48 reduce tasks: Skew factor of 3.70.

The skew factor increases even though the largest reduce task processes fewer records.

Remember: Skew factor = Largest task input ÷ Average task input
With 12 reduce tasks: Average = 60,000,000 ÷ 12 = 5,000,000 records
With 48 reduce tasks: Average = 60,000,000 ÷ 48 = 1,250,000 records

The average task becomes much smaller as we add reduce tasks. However, the largest task still contains all 3.58 million records from cust-0001, along with records from other customers. As a result, the largest task becomes more uneven relative to the average.

Understanding the Hot-Key Limit

In our current design, all records belonging to one customer must be processed by the same reduce task. This means the largest customer’s records establish a minimum amount of work that one reduce task must handle. For cust-0001, that minimum is approximately 3.58 million records.
Our smaller fixture demonstrates the same behaviour:

• With 48 reduce tasks, the largest task contains 1,632 records.
• With 128 reduce tasks, the largest task contains 1,342 records.
• Of those records, 1,261 always belong to cust-0001.

Adding reduce tasks can separate other customers from the hot customer, but it cannot divide that customer’s own records under the existing partitioning rule.

We’ve identified two related problems:

1. 01Too much intermediate data: Without a combiner, every input event produces a separate record for the shuffle.
2. 02Uneven reduce-task workloads: Some customers generate significantly more records than others, creating overloaded reduce tasks.

There are several ways to address these problems. The first question we should ask is:
Can we combine records locally before sending them across the cluster?

Below are a couple of solutions for this:

Solution 1: Use a Combiner to Reduce Intermediate Data

When the aggregation supports merging partial states, a combiner is usually the first optimisation worth considering. For our event-counting job, each map task can maintain a running count for every customer. Instead of sending one intermediate record per event, it sends one partial count per customer it encounters.
As we saw in Section 03, this reduces:

• Intermediate records: From 60 million to approximately 18 million.
• Network transfer: From 0.81 GB to 0.24 GB.
• Largest reduce-task input: From 10.7 million to approximately 1.8 million records.
• Total runtime: From 237.4 to 206.2 seconds.

The combiner improves both data movement and workload distribution without changing the final aggregation result. However, this approach requires an intermediate state that can be merged correctly, as we learned in F03.1.

Solution 2: Isolate High-Volume Keys

What if local aggregation provides little benefit, or the computation requires retaining individual records? In that case, we can try reducing contention between high-volume keys. Recall that cust-0001 produces 3,576,629 records. Under our original hash partitioning, this customer shares a reduce task with many other customers, creating a task containing more than 10.7 million records. We can improve the distribution by giving cust-0001 a dedicated reduce task. This technique is called hot-key isolation.

Here’s how it works:

1. 01Identify customers that generate unusually large numbers of records.
2. 02Assign cust-0001 to a dedicated reduce partition.
3. 03Hash the remaining customers across the 12 regular reduce partitions.
4. 04Process the isolated customer’s records separately from the others.

This prevents the largest customer from sharing a reduce task with other customers. But does that completely eliminate the bottleneck?

We keep the combiner disabled. The job now uses:

• 12 regular reduce tasks for the remaining customers.
• 1 additional reduce task dedicated to cust-0001.

Why Does Another Task Become the Bottleneck?

When we isolate cust-0001, we remove its records from the original overloaded partition. However, the remaining customers still need to be distributed across the regular reduce tasks.
In our reference dataset, cust-0002 and several other customers now form the largest reduce partition. That task receives approximately 7.17 million records, becoming the new bottleneck. The total runtime improves from 237.4 to 226.3 seconds, but the reduce stage still contains a straggler. This teaches an important lesson:
Isolating one hot key may reduce skew, but it can also expose another overloaded partition.

Even isolating the three largest customers doesn’t guarantee the shortest runtime. In our simulator, it reduces the largest task’s input further, but the overall reduce stage takes longer than with only one customer isolated. The number of tasks, scheduling, and task overhead also influence the result. We need to measure the new distribution after every change rather than assuming that isolating more keys will always improve performance.

Why Is the Combiner More Effective Here?

Compare the approaches. Hot-key isolation changes where intermediate records are processed. A combiner changes how many intermediate records are produced in the first place. This distinction matters. With hot-key isolation, nearly the same amount of intermediate data still needs to move across the cluster.
In our simulator:

• Isolating cust-0001 moves approximately 0.82 GB.
• Isolating the top three customers moves approximately 0.81 GB.
• Using a combiner reduces transfer to 0.24 GB.

The combiner is more effective because it reduces the number of records generated by every high-volume customer before the shuffle. For this particular event-counting workload, the combiner provides the best result among the tested configurations.

Solution 3: Split a Hot Key’s Work When the Computation Allows It

Sometimes, even after local aggregation, one key may still require too much processing. Or the intermediate state may be too large to reduce effectively. In these cases, we may need to divide the work belonging to one hot key across multiple tasks. For example, instead of routing every record from cust-0001 to a single reduce task, we could distribute its records using additional sub keys:

cust-0001#0
cust-0001#1
cust-0001#2
cust-0001#3

Each sub key can be processed independently. Afterward, another processing stage combines the partial results into the final result for cust-0001. This introduces an additional aggregation step:
Split the key → Process partial groups → Merge the results

This approach can improve parallelism for operations that support safe partial aggregation. However, it requires additional data movement and coordination. It may also be inappropriate for operations that require all events for a key to be processed together in strict order.

We’ll explore the implementation and tradeoffs of this technique in S04: Skew, Partition Sizing & Distribution.

Choosing the Right Optimisation

The appropriate strategy depends on the computation and its data distribution.

Our experiments demonstrate four important lessons about shuffle performance.

1. 01Shuffles can generate significant data movement: Without a combiner, our event-counting job produces 60 million intermediate records, writing 1.08 GB and transferring approximately 0.81 GB between nodes.
2. 02Combiners reduce shuffle costs: By aggregating records locally, the combiner reduces intermediate data and the workload assigned to reduce tasks. Its effectiveness depends on how many records share the same key within each map task.
3. 03More reduce tasks don’t eliminate hot keys: Increasing the number of reduce tasks can separate other customers from a hot key, but the hot key’s own records still reach one reduce task under ordinary hash partitioning.
4. 04Hot-key isolation has limits: Assigning a high-volume key to its own reduce task may improve performance, but it can expose another overloaded partition without substantially reducing the total shuffle volume.

Limitations of the Model

Our simulator uses several simplifying assumptions:

• Simplified intermediate records: Each partial contains only a key and its aggregation state. Real shuffle data may include serialization overhead, metadata, and other information.
• Proportional workload scaling: The simulator uses the smaller fixture’s key distribution to model a 60-million-event workload. For example, cust-0001 contributes 1,261 events in the fixture and approximately 3.58 million events in the scaled model. Actual customer activity may change over time.
• No sorting, spilling, or compression: The model doesn’t include additional sorting work, disk spilling, or shuffle compression. These can significantly affect performance in real processing engines.
• No hot-key splitting: We keep every customer’s records together in one reduce task. Splitting a hot key into independently processed groups requires an additional merge step, which we’ll explore in the advanced Spark curriculum.
• No failures or retries: Every task finishes successfully. We’ll study how distributed computations recover from failures in F04.

Why Measuring Shuffle Behaviour Matters

The same aggregation can behave differently depending on its input distribution and intermediate state. For example, our reference fixture shows different combiner gains:

A combiner that significantly reduces intermediate records for one aggregation may provide almost no benefit for another. Likewise, increasing reduce tasks or isolating hot keys may help some workloads but worsen others because of task overhead and scheduling behavior.

We’ve seen how combiners, reduce-task counts, and hot-key isolation affect shuffle performance. Let’s explore two additional scenarios where an optimization that seems reasonable may not produce the expected improvement.

Scenario 1: Can a Combiner Reduce Data When We Need Every Value?

So far, our aggregation has counted events per customer. This works well with a combiner because multiple records can be replaced with a single partial count. But suppose the reporting team changes the requirement. Instead of counting events, they now want a complete list of order IDs for each customer. For example, the input contains:

A combiner can group these records locally:

The number of intermediate records decreases from four to two. However, notice something important: all four order IDs must still be preserved. Unlike counting events, we cannot replace three order IDs with a single number because the final report requires every individual ID.

What Happens to Shuffle Data?

Our simulator produces the following results for this list-based aggregation:

The combiner reduces the amount of data written, but the improvement is relatively small.

Why?

The combiner eliminates repeated customer IDs and some intermediate-record overhead, but it must still retain every order ID in the output.

This illustrates a limitation of local aggregation.

A combiner can reduce the number of intermediate records without significantly reducing the total data size.

Its effectiveness depends on how much information the intermediate state must preserve.

For event counts, thousands of records can be represented by a single number. For complete order lists, all order IDs must remain available.

Scenario 2: What If the Input Is Already Partitioned by Customer?

In F03.1, we learned that a shuffle brings partial results belonging to the same key together. But what if those records are already together? Suppose our input is partitioned by customer_id instead of event_id.
This means:

• Every customer’s events are assigned to the same map task.
• The combiner calculates the customer’s complete event count within that task.
• No additional cross-node redistribution is needed in our simulator because the reduce work can use the existing customer-key ownership and local placement.

This sounds more efficient because we eliminate network transfers. However, remember what we learned about data skew in F02.2. Some customers generate far more events than others. When we partition by customer_id, these high-volume customers create uneven workloads. Could avoiding the shuffle make the overall job slower?

Our job counts events per customer using a combiner and 12 workers.
We compare two input partitioning strategies:

• Partition by event_id: Customer events are distributed across multiple tasks, requiring a shuffle.
• Partition by customer_id: Each customer’s events remain together, eliminating cross-node shuffle transfers in our simulator.

Why Does Avoiding the Shuffle Make the Job Slower?

The problem is data skew. When partitioning by customer_id, some tasks receive significantly more records than others. In our reference dataset, customer-based partitioning produces a skew factor of 2.15 with 12 partitions. The slowest map task must process a much larger share of the data, while other workers finish earlier and wait. Even though we eliminate network transfers, the additional processing delay exceeds the shuffle cost we saved.

Compare the overall results:

• Partition by event_id with a combiner and shuffle: 206.2 seconds.
• Partition by customer_id with local aggregation: 314.1 seconds.

The event-partitioned approach is approximately 108 seconds faster, despite requiring data movement.
This demonstrates an important engineering tradeoff: Avoiding a shuffle doesn’t always make a job faster. A shuffle can be worthwhile when it allows work to be distributed more evenly. These results depend on the simulator’s key distribution, task scheduling, and data placement. Real systems may behave differently.

How can we recognise when shuffle operations are slowing down a distributed job? In our simulator, look for four warning signs:

1. 01Too many intermediate records: The map stage produces nearly as many intermediate records as the original input, suggesting that little or no local aggregation is occurring.
2. 02High shuffle volume: Map tasks write large amounts of intermediate data, and reduce tasks retrieve much of it over the network.
3. 03Uneven reduce-task workloads: One or a few reduce tasks receive significantly more records than the others.
4. 04Limited improvement from adding reduce tasks: Increasing the reduce-task count doesn’t eliminate the bottleneck because a high-volume key still belongs to one task.

What Does This Look Like in Production?

Similar symptoms may appear as:

• Shuffle data written is nearly as large as the input dataset.
• A few reduce tasks continue running long after the others have finished.
• Network traffic increases significantly during aggregation stages.
• Increasing shuffle partitions produces little improvement.
• One high-volume key dominates a reduce task’s input.
• Task execution time increases because intermediate data must be sorted, split, or repeatedly read from disk.

For example, our original event-counting job produced 60 million intermediate records and transferred 0.81 GB between nodes. One reduce task received 10.7 million records, creating a significant bottleneck. Enabling the combiner reduced intermediate records to approximately 18 million and lowered network transfer to 0.24 GB.

What Should You Investigate?

When a shuffle-heavy job performs poorly, examine:

• Intermediate record count: How many records are produced before the shuffle?
• Shuffle data volume: How many bytes are written and transferred between nodes?
• Key distribution: Are a few high-volume keys responsible for unusually large reduce partitions?
• Task durations: Which reduce tasks take the longest, and how much data do they process?
• Local aggregation: Can map tasks combine records before redistribution without affecting correctness?

These measurements help determine whether the real problem is excessive data movement, uneven key distribution, or both.

Before changing the number of reduce tasks or modifying aggregation logic, consider five important questions:

1. 01How many intermediate records are produced? Determine whether each input record becomes a separate partial or whether map-side aggregation already reduces the output.
2. 02Can we use a combiner? Check whether the aggregation supports a safely mergeable intermediate state and measure how much data local aggregation can eliminate.
3. 03Are some keys much larger than others? Identify high-volume keys and measure how much data they contribute to the largest reduce tasks.
4. 04How much data is written and transferred? Compare shuffle write volume with actual cross-node transfer to understand disk and network costs.
5. 05Would changing the input partitioning help? Consider whether keeping related records together would eliminate data movement or introduce additional skew.

For example, our event-counting job benefited significantly from a combiner because multiple events for the same customer could be represented by one partial count within each map task. However, keeping all customer records together from the beginning eliminated network transfer while making the map stage much slower.

Suppose our nightly job uses:

• 12 map tasks
• 12 reduce tasks
• A combiner that calculates partial event counts per customer

A colleague says:
“Each reduce task should read data from only one node because it owns a particular group of customers.” Is this correct?

Try It Yourself

When the F03 workbench is available, run the event-counting job without a combiner:

The reference result should show:

• Intermediate records: 60,000,000
• Data transferred: 0.81 GB
• Largest reduce-task input: 10,746,904 records
• Total runtime: 237.4 seconds

Now run the same command without --no-combine:

The expected results are:

• Intermediate records: 18,067,505
• Data transferred: 0.24 GB
• Total runtime: 206.2 seconds

Compare both executions.

Pay attention to how the combiner changes intermediate-record counts, data movement, and reduce-task performance.

Note: The F03 workbench is planned. These commands are intended for use once it becomes available.

A shuffle redistributes intermediate data so records or partial results belonging to the same key can be processed together. Although this is necessary for many distributed computations, it can introduce significant network, disk, and processing costs.

In this lesson, we learned that:

1. 01Shuffle volume depends on intermediate data: Without local aggregation, every input record may produce a separate partial result.
2. 02Combiners can reduce shuffle costs: Combining records locally reduces the amount of intermediate data written and transferred.
3. 03Hot keys create reduce-stage bottlenecks: A high-volume key can dominate one reduce task, even when the number of reduce tasks increases.
4. 04More reduce tasks don’t always solve skew: They can separate different keys but cannot divide one key’s records under ordinary hash partitioning.
5. 05Avoiding a shuffle isn’t always faster: Keeping all records for a key together may eliminate data movement while introducing severe skew in an earlier stage.

Our Nightly Event-Counting Example

Compare the results with and without a combiner:

The combiner substantially reduces the amount of intermediate data and the work performed by the busiest reduce task. However, its benefit depends on how many records share the same key within each map task.

Sources. Capture: shuffle simulator 0.1, Python 3.13, 2026-10-14; fixture orders_s0_small.jsonl sha256 5ac4be13…; profiles nightly_revenue, lab_cluster (learner pack). The combiner is MapReduce’s (Dean and Ghemawat, OSDI 2004, §4.3); the reduce-side schedule is the same greedy list schedule as F02’s (Graham, 1969); the page’s numbers are the module’s own.