Let suppose, when Read heavy system (i.e. Netflix, YT, Amazon etc.) serves a homepage to a user, that single page render triggers hundreds of asset requests of thumbnails, images, banners, logos etc. Multiply that by 200+ million users worldwide, and we are looking at billions of asset metadata queries and image serves per day. A read-heavy DAM at this scale requires a fundamentally different architecture than storing files in a folder.

 

Part 1: Understanding the Read-Heavy Problem

Why Read Heavy System is Different

• Billions of asset reads per day — Read heavy system alone serves 200M+ users, each requesting assets constantly

• Concurrent spike traffic — i.e. when a new show launches or any new asset add, asset traffic can spike 10x in minutes

• Global distribution — Users in Tokyo, Germany, India all requesting the same assets simultaneously

• Cache misses are catastrophic — If a transformation or metadata lookup takes 500ms and you have a cache miss rate of just 5%, you've instantly created 50ms of extra latency across millions of users

• Read-to-write ratio of 1000:1 — Assets are uploaded once but read thousands of times

What We're Actually Optimizing For

1. Can I get the asset metadata fast? (Where is the asset? Who can see it? What's its resolution?)

2. Can I get the transformed variant I need? (This user is on mobile, I need 320px WebP. This user is on laptop, I need 1920px HEIC)

3. Can I deliver it from a location close to the user? (User in India shouldn't wait for asset to travel from US data center)

 

Part 2: The Caching Architecture (Multiple Layers)

Read heavy we can at least 5 caching layers, each solving a different problem.

Layer 1: Browser Cache (User's Device)

The cheapest cache is the one on the user's device. When the frontend loads an asset, it doesn't need to fetch it again for 30 days (or until the page refreshes) or we can set it accordingly.

Why it matters: A single homepage might reference the same logo asset 50 times (in different UI regions). The browser sees the same URL and uses the cached version instead of making 50 requests.

How Directus fits: Directus serves assets with HTTP headers that tell the browser "cache this for 30 days." Immutable assets (logos, brand images) get very long cache times. Mutable assets (user-uploaded content) get shorter times.

Layer 2: CDN Cache (Geographic Distribution)

Read heavy platform serve assets from a single data center in origin. It uses a Content Delivery Network (CDN) like Cloudflare or AWS CloudFront to distribute assets globally.

Here's the mental model:

• User in Tokyo requests asset

• Tokyo CDN edge doesn't have it (cache miss)

• Tokyo edge fetches from Singapore regional cache

• Singapore doesn't have it, fetches from central S3 bucket in us-east-1

• Asset travels back through the chain and gets cached at each level

• Next Tokyo user requesting same asset hits the Tokyo cache (milliseconds)

Why it matters: The difference between fetching from a CDN edge 100km away (10-50ms) vs. a central server 10,000km away (200-500ms) is the difference between a snappy app and a sluggish one. Multiply that latency across 1,000 assets per page and you've just killed the user experience.

How Directus fits: we don't serve assets from api.weframe.com/assets/. We serve from cdn.assets.weframe.com/assets/. The CDN sits in front of your Directus instance. Directus returns the asset once; the CDN replicates it to 200+ edge locations globally.

Cache header strategy: Different asset types get different cache times:

• Static logos: 1 year (they never change)

• User avatars: 1 month (infrequent updates)

• Live trending content: 5 minutes (changes during the day)

Layer 3: Application-Level Cache (Redis)

Even before hitting the database, first we need a fast in-memory cache for asset metadata.

I.e. if a user homepage needs 200 assets. If we query the database 200 times for metadata, that is 200 round trips to PostgreSQL (each ~5-20ms). Total: 1-4 seconds just for metadata queries. That's before you even grab the assets.

Instead, after the first user queries asset metadata, we store it in Redis. The next user gets the same metadata in 1ms, not 10ms.

Why it matters: At Read heavy scale, the difference between 1ms and 10ms per asset × 200 assets = 1.8 second difference in page load time.

Cache invalidation:

• Immediate invalidation: Delete from Redis, next user gets a slight delay (cache miss). Best for accuracy.

• Event-driven: Asset update event triggers cache invalidation across all nodes. More complex but clean.

• TTL-based: Cache expires after 5 minutes regardless. Simple but potentially stale data.

Layer 4: Database Query Cache (PostgreSQL)

PostgreSQL has a query cache, but it's not the main optimization here. Instead, the strategy is query optimization—making sure your queries are so fast they don't need caching.

we can achieve this through:

Indexing

Denormalization

Layer 5: Read Replicas (Horizontal Scaling)

Even with perfect indexing, you hit a ceiling: the single PostgreSQL server can only handle so many queries per second (typically 1,000-10,000 depending on hardware).

Read heavy solution: read replicas. You have a primary database that handles writes (uploads, metadata updates) and multiple replica databases that handle only reads.

• User A uploads an asset (write goes to primary)

• Replication lag: 1-10ms

• User B queries for assets (read goes to replica)

Why it matters: Instead of one database handling all read traffic, you can have 5, 10, or 100 read replicas. Each handles a fraction of the load. Your throughput scales horizontally.

The tradeoff: Now you have consistency issues. If User A uploads an asset and User B immediately queries, will User B see it? Depends on which replica they hit. This is called eventual consistency

 

Part 3: The Transformation Caching Problem

Why Transformation Caching is Critical

A user on an iPhone needs a 320px WebP image. A user on an Android tablet needs a 512px JPEG. All requesting the same master asset.

The naive approach: download the 50MB master image, transform it to 320px WebP on-the-fly, send it to the user. Do this for every request.

At read heavy, this is a disaster:

• Your CPU is pegged just doing image transformations

• Your network is saturated uploading transformed images to users

• Your storage fills up because you're storing variations temporarily while processing

The solution: transformation caching.

How Transformation Caching Works

Instead of transforming on-demand, you:

1. Store the master image in S3 (once)

2. When a user requests 320px WebP, check if this transformation already exists in cache

3. If yes: serve the cached version instantly

4. If no: transform the master → save result to cache → serve to user

The next user requesting the same 320px WebP hits the cache (fast). The request after that also hits the cache. Only the first request pays the transformation cost.

Why it matters: If 10,000 users request the same image in different sizes, you do 10 transformations total (one per unique size), not 10,000

Why Directus specifically: Directus lets you define these presets in configuration. Every asset transformation request goes through this whitelist. If a malicious request tries to use custom dimensions, Directus rejects it

Cache Expiration Strategy

You can't cache transformations forever. Master images change, new variants are added, old presets are removed.

Options:

• Version-based: Include a version number in the cache key. When you update the master, increment the version. New requests get the new transformation.

• TTL-based: Transformations expire after 30 days. For evergreen content (logos, brand images), extend to 1 year.

• Event-based: When a master image is updated, invalidate all cached transformations of that image.

 

Part 4: The Database Scaling Problem

Read Traffic to the Database

CDN handles image delivery. Your application cache handles metadata. But sometimes you still need the database.

Scenarios:

• A user filters assets by tag, date, and license type (requires database query)

• The cache TTL expired, you need fresh metadata

• A new user browses assets for the first time (no cache)

This is still thousands of queries per second hitting PostgreSQL. One slow query means thousands of users experience slowness.

Query Performance Optimization

Indexing Strategy

Materialized Views

Denormalization

Read Replicas

I.e. When a photographer uploads an asset (write), it goes to the primary. Milliseconds later, the replica databases have the same data (replication).

User queries now distribute across replicas:

• 20% of read traffic hits primary (for up-to-date data)

• 80% hits read replicas (for eventual consistency)

Why it matters: Your primary isn't overloaded by read traffic. It can focus on handling writes consistently.

Replication Lag: The delay between write and replica update is typically 1-100ms. For asset metadata, users almost never notice 100ms staleness.

Sharding

If you have billions of assets, even read replicas aren't enough. You need sharding—splitting data across multiple database clusters.

Example: Instead of one PostgreSQL cluster handling all assets, you have:

• Cluster A: Assets tagged with A-H

• Cluster B: Assets tagged with I-P

• Cluster C: Assets tagged with Q-Z

Each cluster is smaller, faster, and handles less traffic.

Why it matters: You can scale to any size, read heavy system doesn't have one database. It has thousands.

The cost: Application logic becomes complex. When searching for assets by tag, you might need to query multiple clusters and merge results.

Directus handles basic sharding scenarios, but at ready heavy system scale, you'd likely build custom sharding logic.

 

Part 5: Storage and CDN Architecture

S3 as the Source of Truth

Master assets live in S3 (Amazon's object storage). Why?

• Durability: S3 replicates your files across multiple data centers. If one burns down, your data survives.

• Scalability: S3 can handle unlimited files and unlimited concurrent requests.

• Cost: Cheap at scale ($0.023 per GB/month).

• Integration: Everything in the cloud ecosystem integrates with S3.

When Directus stores a file, it goes to S3. The database only stores metadata (filename, size, permissions).

Why it matters: If master files were in the database, queries would timeout trying to fetch large images. By keeping them separate, metadata is fast and images are reliable.

S3 Partitioning Strategy

Read heavy system doesn't store all assets in one S3 bucket. They use multiple buckets or prefixes strategically:

• primary-assets/: High-traffic, frequently accessed assets.

• archive-assets/: Rarely accessed assets.

• transformation-cache/: Temporary transformed images (can be deleted if needed)

• cdn-origin/: Optimized for CDN serving

CDN in Front of S3

Don't have clients fetch directly from S3. You place a CDN (like Cloudflare or CloudFront) in front.

Architecture:

User → CDN Edge (Tokyo) → CDN Regional Hub (Singapore) → CDN Origin (us-east-1) → S3 Bucket

When the Tokyo edge receives a request for asset X:

1. Check local cache (hit: serve instantly, 10ms)

2. Miss: fetch from Singapore regional hub (20ms)

3. Hub miss: fetch from origin (100ms)

4. Origin miss: fetch from S3 (150-300ms)

Cache headers: Directus tells the CDN "cache this for 30 days" or "cache for 5 minutes". The CDN respects these.

Why it matters: A user in India gets assets from a server 200km away, not 10,000km away. Latency drops from 500ms to 50ms.

 

Part 6: Consistency vs. Performance Tradeoffs

Strong Consistency

When we upload an asset, every subsequent read sees the new asset immediately.

Pros: No stale data, simple mental model.

Cons: Requires writing to primary database, waiting for replication to all replicas, invalidating all caches. Slower.

Example: Photographer uploads image. Instantly, all users see it.

Eventual Consistency

When you upload an asset, replicas might not have it for 100ms. Caches might show old data for 5 minutes.

Pros: Fast writes, simple architecture, scales infinitely.

Cons: Brief periods of inconsistency.

Example: Photographer uploads image. 99% of users see it in <100ms. 1% of users see old version for 5 minutes.

Read heavy system approach: Eventual consistency for asset metadata, strong consistency for permissions (you don't want to accidentally grant access to an asset).

Multi-Region Consistency

Problem: If you update metadata in the US region, does the Asian region immediately see it? If not, users in Asia might see stale data.

Solution: eventual consistency across regions. Metadata updates replicate asynchronously. Replication lag is typically <1 second globally, but during network issues, can be minutes.

Users usually don't notice because:

1. They see the update on their local region immediately

2. Other users in other regions see eventual updates

3. By the time they travel or switch regions, data is consistent

 

Part 7: The Role-Based Access Control Problem at Scale

Simple RBAC Doesn't Scale

Basic approach: Every asset request checks "does this user have permission?"

User → Directus → Check if user_id 12345 can view asset_id 67890

→ Query database (5ms)

→ Return yes/no

At a heavy scale, with millions of assets and millions of users, these permission checks become a bottleneck.

RBAC Optimization Layers

Layer 1: Permission Caching

After checking "can User A view Asset B?", cache the result in Redis for 5 minutes. The next 100 requests for the same combination hit Redis (1ms) instead of database (5ms).

Layer 2: Role-Based Aggregation

Instead of checking per-asset permissions, check role-level policies.

User has role "Marketing Manager"

Marketing Manager role can see all assets tagged with "marketing"

So User can see all assets tagged with "marketing"

This is pre-computed and cached. One permission check covers thousands of assets.

Layer 3: Token-Based Access

When a user logs in, issue a token that contains their permissions (as a bitmap or hash).

Token contains: [can_view_marketing, can_view_archived, can_view_user_uploads]

Asset request includes token

Directus validates permissions directly from token (no database lookup)

This is how read heavy handles billions of permission checks. The token is cryptographically signed, so it can't be forged.

Public vs. Private Assets

Not all assets need permission checks. Logos are public (everyone sees them). User profile photos are private (only that user sees theirs).

Optimization: If an asset is tagged "public," skip permission checks entirely. Serve from cache instantly.

In a ready heavy system, 99% of assets are public (show thumbnails, logos, backgrounds). Only 1% are private (user data). This dramatically reduces permission checks.

 

Part 8: Scaling Directus Itself (The Application Layer)

Horizontal Scaling

Directus is a Node.js application. A single Node.js process can handle maybe 1,000-5,000 concurrent requests.

Architecture:

Load Balancer

↓

Directus Server 1

Directus Server 2

Directus Server 3

... (100 more)

↓

Redis Cluster (shared cache)

↓

PostgreSQL Replicas

Load balancer distributes requests across servers. Each server is stateless (doesn't store state locally), so any server can handle any request.

Why it matters: If one server dies, 99 others handle the load. You never have an outage.

Cache Coherency

When Server 1 invalidates a Redis cache entry, all servers need to know. Otherwise, Server 2 still serves stale data.

Solution: cache coherency events. When an asset is updated on Server 1, it publishes an event to a message queue. All other servers listen and invalidate their caches.

This is automatic in Redis Cluster—all servers see the invalidation.

Database Connection Pooling

Each Directus server can't open unlimited connections to the database. Instead, it maintains a pool of 10-20 connections, reused across requests.

When requesting traffic spikes, all servers compete for connections. If the pool is exhausted, requests queue and slow down.

Optimization: increase pool size during peak hours, or add more database connections.

 

Part 9: Directus-Specific Optimizations

Why Directus for reading heavy systems scale?

Directus separates files (stored in S3) from metadata (stored in database). This is the foundational design that makes read heavy possible.

A file-in-database CMS would explode under load—the database would be querying and serving multi-gigabyte images constantly.

Configuration Tuning

Asset transformation presets: Define specific sizes (thumbnail, mobile, desktop, 4K) instead of allowing arbitrary sizes. This prevents malicious cache-filling DoS attacks.

Cache settings: Configure Redis as the cache store instead of local disk. Distributed, shared cache that scales horizontally.

Database indexing: Directus doesn't automatically index custom fields. For Netflix scale, manually add indexes on frequently-queried fields (tags, approval_status, uploaded_by).

Replication lag tolerance: Configure how stale data can be before queries retry. Directus defaults are reasonable (re-query after 100ms staleness).

CDN headers: Configure HTTP cache headers per asset type. Logos: 1 year. User uploads: 1 month.

Workflow for Updates

When new assets updates:

1. Upload new master image to S3 (5 seconds)

2. Update metadata in Directus database (1 second)

3. Invalidate cache (1 second)

• Delete transformation cache entries

• Publish invalidation event to CDN

• Clear Redis cache

4. Verification (5 seconds)

• Ping all CDN edges to confirm invalidation

• Spot-check asset requests to confirm new image serves

Total: ~15 seconds from click to all users seeing the new image. Directus orchestrates most of this automatically.

 

Conclusion: From Directus to Read heavy Scale

A stock Directus installation serves a few thousand daily users. To serve read heavy-scale (billions of requests daily)

1. Browser caching (users keep assets locally)

2. CDN caching (global edge distribution)

3. Application caching (Redis for hot metadata)

4. Database optimization (indexes, materialized views, replicas)

5. Storage strategy (S3 for files, database for metadata)

6. Transformation caching (pre-computed images for different devices)

7. Permission caching (quick access control decisions)

8. Horizontal scaling (100s of Directus instances)

9. Observability (metrics, alerts, debugging)

10. Graceful failure (fallbacks, circuit breakers, graceful degradation)

Directus, with its file-metadata separation, is already ahead of monolithic CMSs for this use case. But turning it into a Read-Heavy-grade DAM requires discipline, planning, and deep operational knowledge.