How to Prefetch Across GenericForeignKeys When You Can't Change the Schema

You don't always control the schema you work with. Sometimes you inherit a codebase where GenericForeignKey is threaded through the models, and a migration isn't on the table. Loading a page of 20 audit entries runs ~60 queries because Django's ORM can't prefetch across a GFK. Here's how to fix it without changing a single model.

---

The Problem

GenericForeignKey lets one table point at rows in multiple other tables without knowing which one at query time. The classic use case is an audit log, a notification feed, or a tagging system where the set of target types is open-ended.

But select_related and prefetch_related can't traverse a GFK. The ORM needs to know the target table ahead of time, and a content_type_id/object_id pair doesn't give it that until runtime. The result: every time you touch the GFK on an instance, you pay a query.

Take an activity log. One table, every row references some target: an order, a user, a product:

# models.py
from django.contrib.contenttypes.fields import GenericForeignKey, GenericRelation
from django.contrib.contenttypes.models import ContentType
from django.db import models

class ActivityLog(models.Model):
    actor = models.ForeignKey("User", on_delete=models.CASCADE)
    action = models.CharField(max_length=50)           # "created", "updated", "deleted"
    content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
    object_id = models.CharField(max_length=255)
    target = GenericForeignKey("content_type", "object_id")
    created_at = models.DateTimeField(auto_now_add=True)

class Order(models.Model):
    number = models.CharField(max_length=20)
    status = models.CharField(max_length=20)
    logs = GenericRelation(ActivityLog)

class Product(models.Model):
    name = models.CharField(max_length=255)
    sku = models.CharField(max_length=50)
    logs = GenericRelation(ActivityLog)

class User(models.Model):
    email = models.EmailField()
    logs = GenericRelation(ActivityLog)

This is a legitimate use of GFK. Audit logs should be polymorphic. But rendering a page of 20 activity feed entries, each showing the target's name and a link, hits the database once per entry to resolve log.target: 20 individual single-row lookups.

If the audit trail also needs actor details, or the target's related data, the count multiplies fast. Twenty entries can easily become 60-80 queries before pagination.

---

The Solution

Three steps. First, batch-resolve every GFK lookup in the page at once using compound queries. Second, attach the results to each instance as a cached private attribute. Third, let serializers read the cache with a live-query fallback, so the optimization is optional and the code never breaks if you forget to run it.

Step 1: Resolve the targets in bulk, not one by one

Collect the log entries you want to display, then group their object_id values by content type. Fire one query per target model, map the results back by ID, and attach the resolved instances:

from collections import defaultdict
from django.contrib.contenttypes.models import ContentType

def hydrate_activity_targets(entries):
    """Resolve all ActivityLog.target references in bulk."""
    if not entries:
        return

    # Collect all (content_type_id, object_id) pairs from the page
    refs_by_ct = defaultdict(set)
    for entry in entries:
        if entry.content_type_id and entry.object_id:
            refs_by_ct[entry.content_type_id].add(entry.object_id)

    # Resolve content types dynamically from the entries. One query,
    # no hardcoded model list — inherited codebases gain new types
    # and this adapts automatically.
    ct_by_id = {
        ct.id: ct
        for ct in ContentType.objects.filter(id__in=refs_by_ct.keys())
    }

    # One query per target model, map results back by (ct_id, str(id))
    target_map = {}
    for ct_id, obj_ids in refs_by_ct.items():
        ct = ct_by_id.get(ct_id)
        if ct is None:
            continue
        model = ct.model_class()
        if model is None:
            continue
        for obj in model.objects.filter(id__in=obj_ids):
            target_map[(ct_id, str(obj.id))] = obj

    # Attach to each entry
    for entry in entries:
        key = (entry.content_type_id, str(entry.object_id))
        entry._resolved_target = target_map.get(key)

This resolves every target reference on the page, regardless of type, with one query per content type. For a feed mixing 4 types, that's 4 queries instead of 20. The ContentType map is built from the entries themselves (no hardcoded model list), so the function automatically handles content types added after the hydration code was written.

Step 2: Handle the related data with compound queries

Resolving the target object is only the first layer. Audit entries also need the actor's name, or a display string that lives on a related model. You can extend the hydration to fetch those in bulk too. The pattern is the same: bucket by content type, compound Q filter across all buckets, map back by (content_type_id, object_id):

from django.db.models import Q

def hydrate_activity_details(entries):
    """Bulk-fetch display details for all resolved targets on the page."""
    if not entries:
        return

    # Build a model → ct_id map up front. Unlike Step 1, display formatting
    # is inherently type-specific — a product shows name and SKU, a user
    # shows their full name — so hardcoding known types here is correct.
    ct_map = ContentType.objects.get_for_models(Product, User, Order)
    ct_id_by_model = {model: ct.id for model, ct in ct_map.items()}

    # Group entries by content type
    obj_ids_by_ct = defaultdict(list)
    entry_by_target_id = defaultdict(list)  # target.id → [entries with that target]
    for entry in entries:
        target = getattr(entry, "_resolved_target", None)
        if target is None:
            continue
        ct_id = ct_id_by_model.get(type(target))
        if ct_id is not None:
            obj_ids_by_ct[ct_id].append(target.id)
            entry_by_target_id[(ct_id, target.id)].append(entry)

    # Fetch products in one query
    product_ct = ct_id_by_model.get(Product)
    if product_ct in obj_ids_by_ct:
        product_display = {
            p.id: f"{p.name} ({p.sku})"
            for p in Product.objects.filter(
                id__in=obj_ids_by_ct[product_ct]
            )
        }
        for product_id, display in product_display.items():
            for entry in entry_by_target_id[(product_ct, product_id)]:
                entry._target_display = display

    # Fetch users with their profile in one query
    user_ct = ct_id_by_model.get(User)
    if user_ct in obj_ids_by_ct:
        user_display = {
            u.id: u.get_full_name()
            for u in User.objects.filter(
                id__in=obj_ids_by_ct[user_ct]
            ).select_related("profile")
        }
        for user_id, display in user_display.items():
            for entry in entry_by_target_id[(user_ct, user_id)]:
                entry._target_display = display

    # Repeat the same pattern for Order or any other content type.
    # Targets that don't match a handled type fall back gracefully through
    # the serializer's str(target) path.

Building the display map in a dict comprehension first keeps each pass over entries to O(n). One iteration per content type, not a nested loop over every object × entry combination. For 200 entries and 50 products, that's 50 + 200 checks instead of 50 × 200.

Step 3: Serializers that read the cache with a fallback

Serializers read the cached attributes when they're present, and fall back to a live query when they aren't, so the serializer works whether or not the hydration ran:

class ActivityLogSerializer(serializers.Serializer):
    target_name = serializers.SerializerMethodField()
    target_type = serializers.SerializerMethodField()

    def get_target_name(self, entry):
        display = getattr(entry, "_target_display", None)
        if display is not None:
            return display
        # Cache wasn't populated — fall back to a live GFK lookup.
        target = entry.target
        return str(target) if target else None

    def get_target_type(self, entry):
        target = getattr(entry, "_resolved_target", entry.target)
        if target is None:
            return None
        return type(target).__name__

The getattr(entry, "_target_display", None) pattern is the decoupling mechanism. If hydration ran, the attribute is present and entry.target never fires. If it didn't run, the serializer falls back to a live GFK query and everything still works. The hydration logic is an optional optimization — the serializer doesn't care whether it was called.

---

Why This Works

One query per content type, not per row. The id__in filter resolves all targets of a given type in a single query regardless of how many entries reference that type. A feed with 20 entries across 4 types goes from 20+ queries to 4.

The private-attribute convention is lazy and safe. _resolved_target and _target_display are set when hydration runs and silently absent when it doesn't. The getattr(..., fallback) pattern means no code path breaks. The serializer degrades to live queries without any conditional branching.

No schema change required. This pattern doesn't touch the models, doesn't need a migration, and doesn't alter how GFK works. It's a presentation-layer optimization. You can introduce it incrementally to the slowest pages and leave the rest untouched.

It composes with existing select_related/prefetch_related. If your queryset already uses ORM-level prefetch for non-GFK relationships (like select_related("actor")), the hydration runs after that, filling in only the gaps the ORM can't reach.

---

Design Decision: The Fallback Is Silent, and That's Both a Feature and a Risk

Because getattr(entry, "_resolved_target", entry.target) swallows a missing attribute silently, a typo in the attribute name (_resolvedTarget) will never raise an error. It'll just quietly bypass the cache and fire a live query on every call. The page still renders correctly, but your query count silently regresses.

Mitigations: keep the _ prefix convention consistent, test the query count in integration tests (not just correctness), and consider a debug-mode assertion that logs a warning when hydration is expected but the cache attribute is absent.

One more operational note: if the set of target types is large, the id__in per-content-type approach fires many small queries rather than one big one. At dozens of content types, a single query that fetches all targets of any type via UNION may be faster — but for the typical audit log with half a dozen entity types, the per-type approach is simpler and easier to debug.

---

The Result

An activity feed that used to cost ~60 queries for 20 entries — the initial queryset (1), a per-entry GFK target lookup (20), an actor display-name lookup per entry (20), and a related-field fetch per target (20+, e.g. user profiles, product details) — now costs about 6: one per target content type plus the initial fetch.

The two hydration functions compose between the queryset and the serializer:

entries = ActivityLog.objects.select_related("actor").order_by("-created_at")[:20]
hydrate_activity_targets(entries)
hydrate_activity_details(entries)
return ActivityLogSerializer(entries, many=True).data

No schema change, no migration, no ORM tricks. The hydration functions are drop-ins that run between the queryset and the serializer, and if you forget to call them, nothing breaks. The serializers fall back to live queries.

How are you handling N+1 across GenericForeignKeys in your apps? Are you hand-rolling prefetches, or have you found a different pattern? Drop your approach in the comments.

¡Hasta luego!