Design Twitter / X

Design Twitter's core functionality — users post tweets, follow other users, and see a timeline feed showing tweets from people they follow. The system must handle celebrities with millions of followers.

Capacity Estimation

• Write QPS: 5,000 tweets/sec
• Read QPS: 500,000 timeline reads/sec → 100:1 read-write ratio
• Tweet storage: avg tweet = 300 bytes. 5K × 86400 × 365 × 5 = ~2.4 TB/year
• Media: most tweets have media. Stored separately on a CDN (e.g., S3 + CloudFront)
• Timeline cache: store ~800 tweets per user in Redis. 300M users × 800 × 8 bytes ≈ 1.9 TB Redis

Architecture

[Write Path]
User → API Gateway → Tweet Service
                          │
                    Kafka (tweet events)
                          │
              ┌───────────┴────────────┐
              │                        │
       Fan-out Service            Search Indexer
              │                  (Elasticsearch)
     injects tweet into
     followers' timeline caches

[Read Path]
User → API Gateway → Timeline Service → Redis Cache → DB fallback

Fan-out on write (push model): when a tweet is posted, a background worker writes it to all followers' timeline caches. Timeline reads are O(1) from Redis.

API Design

POST /api/v1/tweets
  Body: { text, mediaIds? }
  Response: { tweetId, createdAt }

GET /api/v1/timeline/home?cursor=&limit=20
  Response: { tweets: [...], nextCursor }

GET /api/v1/tweets/{tweetId}

POST /api/v1/users/{userId}/follow
DELETE /api/v1/users/{userId}/follow

GET /api/v1/search?q=keyword&cursor=

Database Design

Tweets (Cassandra — append-only, time-series)

tweets
  tweet_id    UUID    PK
  user_id     UUID
  content     TEXT
  media_urls  LIST<TEXT>
  created_at  TIMESTAMP
  like_count  COUNTER

User social graph (dedicated graph store or Cassandra)

followers
  user_id     UUID    PK
  follower_id UUID

following
  user_id     UUID    PK
  followee_id UUID

Timeline cache (Redis Sorted Set, score = tweet timestamp)

timeline:{userId}  →  ZSet of {tweetId: timestamp}

Scaling Strategy

The Celebrity Problem (hot write path)

Fan-out on write breaks for celebrities (e.g., Obama with 130M followers). Writing to 130M timeline caches per tweet takes ~10 seconds.

Solution — Hybrid fan-out:

• Regular users (< 1M followers): fan-out on write (push to timelines)
• Celebrities (≥ 1M followers): fan-out on read — inject celebrity tweets at read time

At read time, the timeline service merges:

1. Pre-computed timeline cache (regular users' tweets)
2. Recent celebrity tweets fetched from their profile at read time

Timeline cache eviction

Retain only the 800 most recent tweet IDs per user. Older tweets loaded from DB on scroll.

• Fan-out on write (push): O(followers) on write, O(1) on read. Great for read-heavy systems, bad for celebrities.
• Fan-out on read (pull): O(1) write, O(following_count) read. Simple but slow timelines.
• Hybrid: best of both worlds but most complex to implement.

• Hot celebrity accounts — handled by hybrid fan-out
• Redis memory — LRU eviction; cold users' caches are rebuilt from DB on next login
• Kafka consumer lag — add consumers to the fan-out service; scale horizontally

• Kafka down: tweet still saved to DB. Fan-out delayed. Timeline slightly stale — acceptable.
• Redis down: timeline service falls back to DB. Slower but correct.
• Fan-out service overload: shed load for non-priority users, prioritise active users.