What determines a job’s runtime?
Sixteen processes, one disk: why the hourly job stopped speeding up at 63 seconds
Basic Python is enough. You do not need to install anything to read this lesson. The monetary costs later in the lesson use illustrative units, not a real currency. The ratios are what matter.
Why understanding processing costs matters
Let’s consider an Order platform. Any order on such a platform goes through different states. These can be placed, paid, shipped, delivered, or cancelled. Now let’s say, every night, a Python job reads the day’s events, checks their fields, amounts, and timestamps, then counts accepted and rejected records. On average, we get 60 million events (16.2 GB). On a one-process workload, the job takes 20.7 minutes. Peak volumes are much larger, and order volume keeps on increasing, so someone suggests running it on more machines. Will that make the job finish sooner? And is it worth the cost?
To answer these questions, we need to know how much work one event creates, which machine resources do that work, and which resources the processes share.
Breaking down the cost of one event
Here is one event from our input:
{
"event_id": "ev-000001",
"delivery_id": "dl-27e4102e32",
"supersedes_event_id": null,
"order_id": "ord-00001",
"customer_id": "cust-0134",
"event_type": "order_placed",
"amount_cents": 699,
"event_time_utc": "2026-03-02T00:08:54.332Z",
"ingest_time_utc": "2026-03-02T00:09:02.929Z"
}Processing an event involves up to three types of work:
- Disk I/O: Read the event’s data from storage – approximately 270 bytes in our example above.
- CPU: Decode the JSON and validate the fields.
- Network I/O: Transfer the event if the worker reads it from another machine or storage service.
Let’s estimate the disk-read time. An SSD, which usually reads at 500 MB/s, transfers 500,000,000 bytes per second:
A microsecond (µs) is one millionth of a second. In our reference benchmark, we tested a Python program called a validator. For each order event, the validator parses the JSON, checks that all nine required fields are present, and verifies values such as the order amount and timestamps. The validator processed 21,154 events in approximately 0.352 seconds. Dividing the total processing time by the number of events gives us an average of 16.63 µs per event. This means the validator can process approximately 60,000 events per second on a single processor core under the benchmark conditions. Importantly, this measures only the time spent parsing and validating events. It does not include reading data from disk or transferring it over a network. We’ll use 20 µs of CPU time per event throughout this lesson for simpler calculations.
For network transfer, a 1 Gbps connection can theoretically transfer about 125 MB per second. At that speed, transferring a 270-byte event takes approximately 2.16 µs.
Below is a Resource cost table for a given 270-byte event.
| Resource | Time per record | Assumed rate or cause |
|---|---|---|
| CPU: parse and validate | 20 µs | Simplified validator cost |
| Local SSD read | 0.54 µs | 500 MB/s |
| Hard disk (HDD) read | 1.80 µs | 150 MB/s |
| NVMe disk read | 0.09 µs | 3 GB/s |
| Network transfer | 2.16 µs | 1 Gbps, approximately 125 MB/s |
CPU validation costs 20 µs per record. Reading the same record from the local SSD costs 0.54 µs.
How do they compare?
For one worker reading from the SSD, CPU work is about 37 times the SSD-read cost. CPU is therefore the dominant part of this workload. That does not mean CPU will always be the bottleneck. If many processes share one disk or network link, the shared resource may limit the whole job.
Now the worker gets its records from a remote source. For this example, keep the source’s 0.54 µs SSD read and add a 2.16 µs network transfer.
How much does the modelled per-record cost increase?
Latency and throughput
Latency tells us how long it takes to process a single record or an event.
Throughput tells us how many records can be processed per second.
In our reference benchmark, the validator took an average of 16.63 µs to parse and validate one record. For a dataset containing 21,154 records, the total processing time is approximately:
This means the validator can process approximately 60,000 records per second under the benchmark conditions. Remember two important distinctions:
- Measured vs. estimated cost: The benchmark measured 16.63 µs per record which is a measured cost, but we use a rounded estimate of 20 µs for simpler and more conservative calculations throughout this lesson — an estimated cost.
- Processing time vs. total job time: The 16.63 µs measures only the time spent parsing and validating one record. These are processing times of a Job. The Job takes additional time to read data from disk, transfer data over the network when needed, start workers, and coordinate their work.
Add a second identical machine. Each machine processes its own 21,154-record file.
What happens to the time spent validating an individual record?
This example scales total throughput, not the speed of one record. To shorten the wall time of a single job, we must split that job’s input. The next example shows why splitting work after a certain point stops helping.
Multiple processes, one disk
An hourly validation job processes 24 million events (6.48 GB) on a machine with 32 CPU cores and one HDD. This HDD reads at 150 MB/s. Each process also uses one CPU core. With 8 processes, the job takes 81.4 seconds to complete. To speed things up, the operations team doubles the number of processes to 16.
But will doubling the processes cut the runtime in half?
How long does the hourly job take with 16 processes?
At 8 processes, each process has 65.4 seconds of work, longer than the disk floor. The CPU work is the limiting factor in this model. At 16 processes, each process has only 32.7 seconds of work, but the shared disk still needs 43.2 seconds. The disk now sets the pace. More CPU cores cannot make it read the file faster. At 32 processes, wall time actually rises to 71.2 seconds. The disk floor stays the same, while process overhead increases. In our benchmark, the processors are idle for about 62% of the processing phase.
The bottleneck moved from CPU to disk I/O. Adding more CPU processes no longer helps because all of them depend on the same HDD.
Fixing the bottleneck
Instead of adding more processes, we should improve the resource limiting the job’s performance: the disk. Let’s replace the HDD with a faster NVMe disk that can read 3 GB/s. This reduces the minimum time needed to read the entire dataset from 43.2 seconds to approximately 2.2 seconds.
With 16 processes, the job now completes in 50.1 seconds instead of 63.2 seconds, a 21% reduction in runtime. The disk is no longer the bottleneck. In this model, the job becomes CPU-bound again, meaning CPU processing now takes longer than reading the data.
| Processes | Work per process | Disk floor | Wall time | Cost per run |
|---|---|---|---|---|
| 8 | 60.3 s | 2.2 s | 76.3 s | 0.66 units |
| 16 | 30.1 s | 2.2 s | 50.1 s | 0.87 units |
| 32 | 15.1 s | 2.2 s | 43.1 s | 1.49 units |
But faster hardware costs more. The HDD drive mentioned above is billed at 3.0 units per core-hour; the NVMe machine at 3.9 units per core-hour.
For 16 processes, the NVMe run costs: 16 × 50.1 ÷ 3,600 × 3.9 ≈ 0.87 units.
The corresponding HDD run costs about 0.84 units. The NVMe option is roughly 4% more expensive using these rounded prices, but completes about 21% sooner. Is this additional cost justifiable? It depends on the business needs and what a faster result is worth.
Another option is reducing the bytes read through compression or a more efficient file layout. How to design that layout belongs to Data Modelling & Storage Design. Here, the important question is whether reading those bytes is actually the bottleneck.
What this benchmark shows and its limitations
This benchmark demonstrates three important lessons:
- Understand where processing time goes: Measuring the CPU, disk, and network costs of processing each record helps identify potential bottlenecks.
- Shared resources limit performance: When multiple processes share the same disk, adding more CPU cores cannot reduce the minimum time required to read the data.
- More processes don’t always mean faster execution: Once a resource becomes the bottleneck, adding processes may increase overhead and make the job slower.
Limitations of the benchmark
The simulation run makes a few simplifying assumptions:
- Equal workload: Every process handles the same amount of work. We’ll explore uneven data distribution in F02.
- Simplified resource costs: The model uses fixed processing costs, while real systems may read data and perform computations simultaneously.
- No failures: The model doesn’t account for task failures or retries. We’ll cover these in F04.
Note: These results illustrate how bottlenecks affect performance. Always measure your actual CPU, disk, network, and workload behaviour before making optimisation decisions.
How to spot a shared-resource bottleneck
When multiple processes share a resource, such as a disk or network connection, adding more processes may stop improving performance. Always look for these three warning signs:
- Less work per process, but no faster execution: Each process handles less work, yet the processing phase stops getting shorter.
- The shared-resource limit is reached: The processing phase cannot drop below the minimum time required by the shared resource.
- Total runtime stops improving: Adding more processes produces little improvement or even increases the job’s runtime.
In real systems, similar signs include:
- Throughput stops increasing as more workers are added.
- Disk or network utilisation remains high, indicating a saturated resource.
- CPU utilisation stays low because processes are waiting for data rather than performing computations.
What to check before adding workers
Before increasing the number of workers, ask yourself four questions:
Estimate the CPU time, disk reads, and network transfers required.
Identify shared disks, network connections, or services that could become bottlenecks.
Calculate how long the shared resource needs to handle the entire workload, regardless of the number of workers.
Consider startup, coordination, and communication costs. We’ll explore these in F01.2.
Check your understanding, then run the model
The nightly job has 60 million events (16.2 GB). Compare two ways to provide 16 CPU cores:
- Option 1: One 16-core machine, with the file on its local SSD.
- Option 2: Sixteen one-core machines, each with its own SSD and one-sixteenth of the file.
The CPU work per record is the same.
Which option finishes sooner in the model?
Try it in the workbench. The L-F01 simulator will let you reproduce the hourly result:
python -m sim run --job hourly_validate --cluster one_node_hdd --workers 16
Check that the disk floor and processing phase are both 43.2 seconds. Then change --workers 16 to --workers 32. Watch what happens to work per process and wall time.
The article itself requires no installation. Record your answers to P01–P05 before viewing their reveals.
Note: The workbench is a small Python repository you clone and run on your laptop (Python 3.11, nothing else); it opens at /learn/distributed/parallelism.
Summary
Every record requires processing resources, including CPU time, disk reads, and sometimes network transfers. Understanding these costs helps identify bottlenecks and determine whether adding more workers will actually improve performance.
In our hourly validation example:
- A single HDD needs at least 43.2 seconds to read the entire dataset.
- With 16 processes, the shared disk becomes the bottleneck, and the job takes 63.2 seconds
- Upgrading to faster storage reduces runtime by approximately 21%, while increasing cost by only 4%.
More workers don’t always mean faster execution. Identify the bottleneck first, then optimise the resource that actually limits performance.
Interview form: “We doubled the cores and the job did not get faster. Why?” Start with “which part of the machine set the pace?” Production symptom: throughput flat across process counts, a disk or link at 100%, processors idle at a steady fraction. In Spark the same floor appears as executors on one node sharing its disk (S04).
Sources and reproducibility. Figures reproduced from reference model sim 0.1, Python 3.13.16, captured 2026-10-13. Fixture: orders_s0_small.jsonl, SHA-256 prefix 5ac4be13…, mean size 269.8 bytes/line. Validator benchmark: 16.63 µs/record, parse + contract validation, 2.1 GHz Xeon, best of five. Profiles: hourly_validate, nightly_validate, one_node_hdd, one_node_nvme, one_node_ssd, and lab_cluster. The price-list approach is inspired by Peter Norvig’s Teach Yourself Programming in Ten Years; the figures here come from the module’s model or measurements, not Norvig’s table.