Deduplication & latest-record
Concept  ·  SQL & Querying

Each dedups differently and costs differently — DISTINCT ON and QUALIFY keep a chosen row, DISTINCT/GROUP BY collapse.

DISTINCT's unit is the projected tuple — it answers 'unique combinations of what I selected', nothing more. Entity dedup ('one row per customer') requires a choice of which row represents the entity — exactly what DISTINCT can't make, and what ROW_NUMBER = 1 exists to make explicitly. The gateway misuse: a fan-out produces duplicates, DISTINCT makes the output look right, and the multiplication lives on.

See it break
members
custcity
7Rome
7Oslo
The trap
SELECT DISTINCT
  *
FROM
  members
ORDER BY
  cust,
      city;
The fix
SELECT
  cust,
  city
FROM
  (
    SELECT
      cust,
      city,
      ROW_NUMBER() OVER (
        PARTITION BY
          cust
        ORDER BY
          city DESC
      ) AS rn
    FROM
      members
  )
WHERE
  rn = 1;
The trap
7Oslo
7Rome
The fix
7Rome

SELECT DISTINCT * kept both of customer 7's rows — DISTINCT dedups whole output rows, so a differing city leaves both; entity dedup needs a chosen winner (ROW_NUMBER = 1), which DISTINCT can't make.

The rule

ROW_NUMBER = 1 with a stated winner for entity dedup; fix the producing join; DISTINCT only for genuine 'unique combinations'.

Shows up elsewhere
'Latest' without a tiebreakerDedup before vs after join
Try one →Prove it in practice →