Balancing Job Performance and Cost

Our nightly validation job has a strict deadline. The fulfilment system needs the validated order data within four minutes (240 seconds) after the day ends. In F01.2, we explored how increasing the number of workers affects job performance. We discovered that 20 workers completed the job in 171.6 seconds, making it the fastest configuration in our model. But do we really need the fastest configuration? What if fewer workers could complete the job within the required four minutes at a much lower cost? This brings us to an important engineering question:

How do we choose the least expensive configuration that still meets our processing deadline?

Before answering, let’s first understand why Worker count affects cost. In simplified billing, every worker is charged for the entire time the job runs, including when it is:

• Actively processing records.
• Waiting for the coordinator to assign or collect work.
• Waiting for slower workers to finish.

Adding workers can reduce processing time, but it also increases the number of computing resources we pay for. For example, using 20 workers may finish a job faster than using 8, but the additional speed might not justify the higher cost if both configurations meet the deadline.

The fastest configuration isn’t always the most cost-effective. When workers are billed according to allocated cores and runtime, we need to balance execution time, resource usage, and cost. The goal is to find the least expensive configuration that meets the job’s requirements, not simply the one with the most workers.

When running a distributed job, we pay for the computing resources allocated to it, even when those resources are waiting rather than processing data. Let’s compare our nightly validation job running on a single processor versus a cluster of 16 workers.

Running the Job on a Single Processor

On a single processor, the nightly validation job takes 1,242.4 seconds to complete. Since we’re using one CPU core, the total resource consumption is:

A core-second represents one CPU core allocated for one second. If computing resources cost 3.0 units per core-hour, the total cost is: Cost = (1,242.4 × 3.0) ÷ 3,600 ≈ 1.04 units

In our simplified model, the single processor spends its time performing useful work, so we treat its efficiency as 100%.

Running the Same Job with 16 Workers

Now let’s distribute the same job across 16 single-core workers. From F01.2, we know that the job finishes in 175 seconds. However, all 16 workers are allocated for the entire job duration, including time spent waiting for startup, coordination, and other workers.

Therefore: Billed core-seconds = 16 × 175 = 2,800 core-seconds

The amount of useful work remains approximately 1,242 core-seconds, but we’re now paying for 2,800 core-seconds of allocated resources. The difference comes from the overhead and waiting time introduced by distributed execution.

Measuring Processing Efficiency

We can measure how effectively our allocated resources are used:

Here, busy core-seconds represent the useful-work estimate from our reference model.

Our nightly job uses 16 workers and completes in 175 seconds. The job requires approximately 1,242 core-seconds of useful work, but the cluster bills us for 2,800 core-seconds.

We’ve seen that adding workers can reduce runtime, but it also increases the amount of computing resources we pay for. Now let’s compare execution time, processing efficiency, and cost for different worker configurations. Our model assumes a computing price of 3.0 units per core-hour.

For each configuration: Cost per run = (Workers × Runtime × Price per core-hour) ÷ 3,600

How Worker Count Affects Runtime and Cost

Costs are based on the simulator’s illustrative billing model.

What Can We Learn from These Results?

1. Faster execution can cost significantly more.
Increasing workers from 8 to 20:

• Reduces runtime from 228.1 to 171.6 seconds, saving 56.5 seconds.
• Increases cost from 1.52 to 2.86 units, approximately 88% more.

The job finishes sooner, but that additional speed comes at a higher cost.

2. More workers can make a job both slower and more expensive.
Increasing workers from 20 to 64:

• Increases runtime by 89.7 seconds.
• Raises the cost from 2.86 to 13.93 units, nearly five times as much.

Here, adding workers makes performance and cost worse.

We shouldn’t optimise only for the lowest cost or the shortest runtime. We need the least expensive configuration that meets the business requirements.

Our nightly validation job has a strict deadline. The fulfilment system needs validated order data within 4 minutes (240 seconds) after the day ends.

The operations team is considering three configurations:

• 8 workers
• 20 workers
• 64 workers

The goal is to choose the least expensive configuration that meets the deadline.

Choosing a Safer Configuration?

Our simulator assumes consistent execution times, but real-world jobs can take longer due to larger datasets, resource contention, or unexpected delays. In our example, 8 workers finish with only 12 seconds to spare. While this meets the deadline in the simulator, it may not be reliable enough in production. To choose a safe configuration, we should measure how much runtime varies across runs and allow enough extra time to handle unexpected delays.

The first step in choosing computing resources is to define the requirement before selecting the worker count. For our nightly job, the requirement is straightforward:

Complete validation within 240 seconds at the lowest practical cost.

Under our simulator’s assumptions, 8 workers satisfy that requirement at the lowest cost. If the deadline becomes stricter, we can compare other configurations. For example, increasing from 8 to 16 workers:

• Reduces runtime from 228.1 to 175.0 seconds, saving approximately 53 seconds.
• Increases cost from 1.52 to 2.33 units, an additional 0.81 units.

This gives us a measurable tradeoff between faster execution and higher cost. However, computing resources are not always the only expense.

Don’t Forget the Cost of Moving Data

In F01.1, we estimated how long it takes to transfer a record over a network. But in cloud and distributed environments, moving data may also introduce monetary charges, often based on the amount transferred. Let’s extend our nightly validation example. Suppose:

• The input dataset contains 16.2 GB of order events.
• The data is stored remotely in object storage.
• 16 workers read the data over the network.
• Network transfer costs 0.25 units per GB.
• Computing resources cost 3.0 units per core-hour.

We want to determine whether processing or transferring the data costs more.

Why Does This Matter?

Let’s compare the same 16-worker job with local and remote data.

In our model, reading data remotely adds only about 8 seconds to the runtime. However, it increases the total cost from 2.33 to 6.49 units, almost three times as much.

This illustrates an important distinction:

• Performance cost: How much additional time does transferring data require?
• Monetary cost: How much are we charged for transferring that data?

A network transfer can have a relatively small impact on execution time while contributing significantly to the overall bill. The transfer price used here is illustrative. Actual charges depend on the storage provider, network location, and pricing arrangement. This example also considers only the input transfer, not output writes or communication between workers.

We’ll explore where data should be placed and how it should be distributed in F02 and Data Modelling & Storage Design.

Our simulation demonstrates three important lessons:

1. 01More workers can increase costs: Adding workers may reduce execution time, but it also increases the computing resources we pay for. Faster execution doesn’t always mean lower cost.
2. 02Choose resources based on requirements: The goal is to find the least expensive configuration that meets the job’s deadline, rather than simply choosing the fastest configuration.
3. 03Data movement can be expensive: Even when network transfers add little processing time, their monetary cost can exceed the cost of computation.

Limitations of the Model

Our simulation makes a few simplifying assumptions:

• Illustrative pricing: We use fixed prices per core-hour and per GB transferred. Real cloud providers have different billing models, discounts, and pricing for reserved or spot capacity.
• Simplified resource billing: Every worker is billed for the entire job runtime. The coordinator’s cost is excluded.
• Limited network costs: We calculate the cost of reading input data remotely. Writing results and transferring data between workers can introduce additional costs, which we’ll explore in F03.
• No failures or retries: Every job completes successfully. In production, failures and retries can increase costs. We’ll explore retry behaviour and correctness in F04.
• Equal workload distribution: Every worker receives an equal share of the workload. Uneven distribution can cause some workers to finish later, increasing execution time and cost. We’ll cover this in F02.

Distributed processing doesn’t always require multiple machines. Sometimes, a single powerful machine can complete a job faster and at a lower cost. Let’s consider our hourly validation job, which takes 503 seconds when running on a single CPU core.

The team is considering two alternatives:

1. 01Distributed cluster: Use 8 single-core workers running on separate machines.
2. 02Single machine: Use one machine with 16 CPU cores and a local SSD, running 16 processes in parallel.

Both approaches provide parallel processing, but they have different startup and coordination costs.

Why Is the Single Machine Faster?

1. The cluster has higher startup and coordination overhead.
The distributed cluster spends:

• 40 seconds on cluster startup and preparation.
• 24 seconds managing its 8 workers.
• 10 seconds performing serial work.

Together, these activities require 74 seconds, before accounting for parallel processing time. The cluster’s fixed 40-second startup alone represents approximately 30% of its total runtime.

2. The single machine has much lower overhead.
The 16-core machine runs multiple processes locally without the same cluster startup costs. Its total serial work and overhead amount to only 20 seconds. With 16 processes sharing the workload, the parallel processing phase takes approximately 30.8 seconds. Its SSD also reads data fast enough that disk I/O does not become the bottleneck in this example.

3. The single machine is both faster and cheaper.
Compare the results:

• Distributed cluster: 135.6 seconds at 0.90 units.
• Single 16-core machine: 50.8 seconds at 0.68 units.

The single machine completes the job approximately 2.7 times faster, while also costing less.

What Should We Learn from This?

Distributing work across machines introduces costs such as cluster startup, scheduling, communication, and coordination. For relatively small jobs, these costs may outweigh the benefits of distributing computation. A useful strategy is to consider scaling up before scaling out:

• Scale up: Add CPU, memory, or faster storage to a single machine.
• Scale out: Distribute processing across multiple machines.

Scaling up can be simpler and more cost-effective when the workload fits comfortably on one machine. However, larger workloads may eventually exceed a single machine’s CPU, memory, storage, or throughput limits. At that point, distributing the work may become necessary.

How can you tell when a distributed job is using more resources than necessary? In our simulator, look for three warning signs:

1. 01Low processing efficiency: Less than 50% of the billed computing time is spent on useful work. This suggests that a significant portion of resources is spent waiting or handling overhead.
2. 02Costs increase without meaningful performance gains: Adding workers increases the bill, but the job finishes only slightly faster or even becomes slower.
3. 03Data transfer costs exceed computing costs: Moving data across the network contributes more to the total bill than processing it.

In real production systems, similar problems may appear as:

• Higher costs for small performance improvements: Computing costs double, but the job finishes only 10% faster.
• Overprovisioned clusters: A cluster sized for peak traffic runs at the same capacity even when workloads are much smaller.
• Unexpected network charges: Data transfer costs were not included in the original estimate.
• High startup overhead: A short job spends a significant portion of its runtime starting and coordinating workers.

These are warning signs rather than definitive problems. For example, low resource efficiency may be acceptable when meeting a strict deadline is more important than minimising cost.

Before deciding how many workers a job needs, ask yourself five questions:

In our simplified model, the smallest worker count that meets the deadline is also the least expensive qualifying configuration. Real systems may have different pricing and performance characteristics.

Let’s revisit the hourly validation job from F01.1. The job runs with 16 processes, but we have two machine configurations to choose from:

• HDD machine: Costs 3.0 units per core-hour and completes the job in 63.2 seconds.
• NVMe machine: Costs 3.9 units per core-hour and completes the job in 50.1 seconds.

The NVMe machine is faster, but its computing resources cost 30% more per core-hour.

Try It Yourself

When the F01 workbench is available, run:

Compare the execution time and cost for each worker count.

Pay particular attention to the 8-worker configuration, which completes the nightly job in 228.1 seconds at 1.52 units.

Ask yourself: Does adding more workers save enough time to justify the additional cost?

Running a distributed job involves more than choosing the number of workers. We must consider execution time, computing costs, resource efficiency, and data movement.

In this lesson, we learned that:

1. 01More workers can increase costs: Every allocated worker contributes to the bill, even when it is waiting instead of processing data.
2. 02The fastest configuration isn’t always the best: Choose the least expensive configuration that reliably meets the required deadline.
3. 03Network transfers can be expensive: Moving data may add little processing time while significantly increasing the total cost.
4. 04A single machine can sometimes outperform a cluster: For smaller workloads, avoiding distributed coordination overhead can improve both performance and cost.

Where This Matters

In production systems such as Apache Spark, similar tradeoffs arise when choosing executor resources, cluster sizes, and storage locations. For example, a cluster might finish a job 10% faster but cost twice as much. Alternatively, reading data from remote storage may introduce transfer charges that exceed the cost of computation.

We’ll explore these considerations further in S05 (Spark Memory, Spill & Cluster Resources) and I03 (Production Architecture & Failure-Driven Design).

Sources. Capture: reference model sim 0.1, Python 3.13.16, 2026-10-13; profiles nightly_validate, hourly_validate, lab_cluster, remote_cluster, one_node_ssd, one_node_hdd, one_node_nvme, prices.json (learner pack). All prices are the module’s illustrative units.