Deduplication & latest-record
Concept  ·  SQL & Querying

Deduping after a join can't undo fan-out already created — dedup at the source grain (P06 crossover).

Fan-out isn't only a data-model property — it's often just dirt: a double-loaded file, a retried insert, a source never unique on the key you assumed. The join faithfully multiplies by the dirt. Deduping the result is the losing move (which of the multiplied fact rows do you keep?); dedup the side that should have been unique, at its own grain, before the join — DISTINCT, or row_number = 1 when a winner is needed.

See it break
stock
itemqtydonor_id
rice31
beans22
donors
donor_idname
1Ada
1Ada
2Ben
The trap
SELECT
  SUM(s.qty) AS total
FROM
  stock s
  JOIN donors
  d ON d.donor_id = s.donor_id;
The fix
SELECT
  SUM(s.qty) AS total
FROM
  stock s
  JOIN (
    SELECT DISTINCT
      donor_id,
      name
    FROM
      donors
  ) d ON d.donor_id = s.donor_id;
The trap
8
The fix
5

A double-entered donor row fanned the stock — the joined SUM read 8, not 5, because rice matched the duplicated donor twice; deduping the donor list before the join fixes it.

The rule

DISTINCT or rn = 1 on the dimension pre-join; the two-count probe as a habit.

Shows up elsewhere
Fan-out before aggregate'Latest' without a tiebreaker
Try one →Prove it in practice →