Fixing N+1 queries on a Django list endpoint without changing the contract.
A list endpoint that serializes orders with their customers and lines issues two extra queries
per row in its naive form. This case keeps the JSON payload byte-identical, moves the work into
the queryset, and pins the result with query-count tests.
The endpoint works; the query plan does not scale.
A list endpoint that returns orders with their customer and lines reads fine in code review:
fetch the orders, serialize each one. The ORM turns that loop into one query for the list and
two more for every row. At fifty rows the endpoint issues 101 queries to build one response.
The response is correct at every size, so the failure mode is cost, not errors.
Each added row adds two round trips: one for the customer, one for the lines.
Nothing in the payload hints at the problem; only the query count names it.
The fix must not change the contract, because callers already depend on the shape.
02 — Constraints
Public-safe, deterministic, and honest about what the numbers are.
This case stays public-safe. The domain is a synthetic order catalog, the database is
in-memory SQLite, and the seed data is fixed. The query counts on this page are measured by
Django's own test tooling against this repository, not observed on any production system.
Seeded data keeps every query count reproducible.
No employer schema, customer data, or production identifiers appear.
The counts are exact query totals, not latency estimates.
The demo keeps the naive endpoint only to prove the contract stays identical.
03 — Architecture
One serializer, two query plans, and tests that pin both.
The project is deliberately small: three models, one serializer, and two list views that differ
only in the queryset. That shape is what makes the release-safety argument executable — the
contract test compares the two endpoints directly.
Endpoint query plans
Endpoint
Query plan
GET /api/orders/naive/
One query for the order list, then a customer query and a lines query for every row. 1 + 2N queries.
GET /api/orders/
select_related joins the customer and prefetch_related loads all lines up front. 2 queries at any size.
Both endpoints walk the same serializer, so the only honest variable is the queryset. The
naive plan grows by two queries per row; the fixed plan is a constant two queries, and the
guardrail tests fail if either plan drifts.
04 — Contract and tests
Release safety is a test, not a hope.
The deploy is safe because the proof runs in CI: the contract test compares both endpoints
byte for byte, and the query-count tests pin the plans at two sizes.
Tests
The repo has 6 tests across 2 files. Run them with python manage.py test, and print the measured table with python scripts/report_queries.py.
Test files
File
Tests
Coverage
catalog/tests/test_contract.py
2
Naive and fixed endpoints return identical payloads; the response shape stays stable.
catalog/tests/test_query_counts.py
4
assertNumQueries pins the naive growth at two sizes and the fixed plan at a constant two queries.
05 — Measured queries
The naive plan grows; the fixed plan stays at two.
The report script seeds the same data at three sizes and measures both endpoints with Django's
query capture. The naive count is exactly 1 + 2N: one list
query, then a customer query and a lines query per order.
5 orders: the naive endpoint issues 11 queries; the fixed endpoint issues 2.
25 orders: the naive endpoint issues 51 queries; the fixed endpoint issues 2.
50 orders: the naive endpoint issues 101 queries; the fixed endpoint issues 2.
The fixed plan is constant because select_related and prefetch_related move per-row lookups into two set-based queries.
docs/query-report.txt
Measured query counts at three sizes
Generated by python scripts/report_queries.py in the companion repository.
The schema, the shared serializer, the two querysets, the guardrails.
The companion repository keeps each concern in its own file. The snippets below are the parts
that define the relationships, keep the contract in one place, separate the two query plans,
and pin the counts.
catalog/models.py
Three models and the relationships the endpoint walks.
Orders belong to customers and have lines; the serializer traverses both relationships per row.
"""Public-safe demo schema: customers place orders, orders have lines.
The shape is deliberately small. The case is not the schema; it is the query
plan of the list endpoint that serializes all three tables.
"""
from django.db import models
class Customer(models.Model):
account_ref = models.CharField(max_length=32, unique=True)
display_name = models.CharField(max_length=120)
def __str__(self) -> str:
return self.account_ref
class Order(models.Model):
class Status(models.TextChoices):
DRAFT = "draft", "Draft"
SUBMITTED = "submitted", "Submitted"
FULFILLED = "fulfilled", "Fulfilled"
customer = models.ForeignKey(Customer, related_name="orders", on_delete=models.CASCADE)
reference = models.CharField(max_length=32, unique=True)
status = models.CharField(max_length=16, choices=Status.choices, default=Status.DRAFT)
placed_on = models.DateField()
class Meta:
ordering = ["reference"]
def __str__(self) -> str:
return self.reference
class OrderLine(models.Model):
order = models.ForeignKey(Order, related_name="lines", on_delete=models.CASCADE)
sku = models.CharField(max_length=32)
quantity = models.PositiveIntegerField()
unit_price_cents = models.PositiveIntegerField()
class Meta:
ordering = ["sku"]
@property
def line_total_cents(self) -> int:
return self.quantity * self.unit_price_cents
catalog/serializers.py
One serializer shared by both endpoints.
The contract lives in exactly one place, so the naive and fixed paths cannot drift apart.
"""Serialization shared by both endpoints so the contract stays identical.
Keeping one serializer is the point of the case: the fixed view changes the
query plan, not the response shape, so the deploy is release-safe.
"""
def serialize_order(order) -> dict:
lines = list(order.lines.all())
return {
"reference": order.reference,
"status": order.status,
"placed_on": order.placed_on.isoformat(),
"customer": {
"account_ref": order.customer.account_ref,
"display_name": order.customer.display_name,
},
"lines": [
{
"sku": line.sku,
"quantity": line.quantity,
"unit_price_cents": line.unit_price_cents,
"line_total_cents": line.line_total_cents,
}
for line in lines
],
"total_cents": sum(line.line_total_cents for line in lines),
}
catalog/views.py
Two views that differ only in the queryset.
The fix is one line of queryset: select_related for the customer join, prefetch_related for the lines.
"""Two list endpoints with the same contract and different query plans.
`order_list_naive` walks orders and touches customer and lines per row: one
query for the orders, then two queries per order. `order_list` fetches the
same graph with select_related + prefetch_related: a constant two queries.
"""
from django.http import HttpRequest, JsonResponse
from catalog.models import Order
from catalog.serializers import serialize_order
def order_list_naive(request: HttpRequest) -> JsonResponse:
orders = Order.objects.all()
return JsonResponse({"orders": [serialize_order(order) for order in orders]})
def order_list(request: HttpRequest) -> JsonResponse:
orders = Order.objects.select_related("customer").prefetch_related("lines")
return JsonResponse({"orders": [serialize_order(order) for order in orders]})
catalog/tests/test_query_counts.py
assertNumQueries pins both query plans.
A regression that reintroduces per-row queries fails the suite before it ships.
"""Query-count guardrails.
The naive endpoint issues 1 + 2 queries per order: one for the order list,
then a customer lookup and a lines lookup per row. The fixed endpoint is a
constant two queries (select_related join + lines prefetch) at any size.
These tests pin both facts so a regression fails CI.
"""
from django.test import TestCase
from catalog.seed import seed_orders
class QueryCountTest(TestCase):
def test_naive_queries_grow_with_row_count(self):
seed_orders(order_count=5)
with self.assertNumQueries(11): # 1 + 2 * 5
self.client.get("/api/orders/naive/")
def test_fixed_queries_constant_at_small_size(self):
seed_orders(order_count=5)
with self.assertNumQueries(2):
self.client.get("/api/orders/")
def test_fixed_queries_constant_at_large_size(self):
seed_orders(order_count=50)
with self.assertNumQueries(2):
self.client.get("/api/orders/")
def test_naive_at_large_size_shows_the_growth(self):
seed_orders(order_count=50)
with self.assertNumQueries(101): # 1 + 2 * 50
self.client.get("/api/orders/naive/")
What this case shows, and what it deliberately does not claim.
I would rather pin the query count in a test than argue about the ORM in a review.
01
Contract stability first
Both endpoints share one serializer, so the fix is a queryset change with a byte-identical payload. The tradeoff is carrying the naive endpoint in the demo purely to prove that equivalence.
02
Query counts over latency claims
The page quotes query counts because they are exact and reproducible. Latency on a production system depends on data shape, indexes, and cache state, and none of that is claimed here.
03
Guardrail tests as the deliverable
assertNumQueries turns the fix into a regression gate: any future change that reintroduces per-row queries fails the test suite. The cost is that the counts are brittle by design and must be updated deliberately.
Intentionally absent
No production database, employer schema, or real customer data. The domain is synthetic and the database is in-memory SQLite.
No latency or throughput claims. Query counts are the only numbers, and they come from this repository's own tests.
No DRF, caching, or pagination layers. The case isolates the query plan so the before and after stay readable.
Production path
Query counting in CI. Keep assertNumQueries on the endpoints whose plans matter, so ORM changes that add per-row queries fail before release.
Slow-query and trace visibility. Pair the tests with database logging or APM spans in production so an N+1 that slips through is visible in telemetry, not just in tests.
Prefetch discipline in review. Treat serializer field access as the query-plan contract: any new nested field needs a matching select_related or prefetch_related decision.
Pagination before unbounded lists. A constant query plan still returns every row. Production list endpoints need page bounds so the constant stays cheap.