Design LeetCode

An online judge is a straightforward CRUD application wrapped around one genuinely hard problem: executing arbitrary code written by strangers on your hardware. Spend your answer there.

Requirements

Functional

• Browse problems; submit a solution in one of many languages.
• Run against test cases, report accepted/wrong answer/TLE/MLE/runtime error.
• Timed contests with a live leaderboard.
• Submission history and per-user progress.

Non-functional

• Verdict within a few seconds.
• Untrusted code must not escape, persist, or affect other submissions.
• Contest starts are enormous, synchronised traffic spikes.

5M users, 500K submissions/day ≈ 6/second average
Contest start: 50,000 submissions in the first few minutes → ~500/second peak
Each submission runs against ~100 test cases within a few seconds

The load profile is the thing: average load is trivial, peak load is 100× that, and it is perfectly predictable because contests are scheduled. Pre-scale for the contest.

Architecture

Client → API → submission service
                    │  persist submission (status = PENDING)
                    ↓
              queue (Kafka / SQS)
                    ↓
         judge workers (autoscaled, isolated)
                    │  run against test cases
                    ↓
            result service → DB → WebSocket push to the client

Judging is asynchronous, always. The API returns a submission id immediately and the verdict arrives over a WebSocket or by polling. Holding an HTTP request open for a ten-second judge run wastes a connection and fails badly under contest load.

The queue absorbs the contest spike — submissions pile up, workers drain them, and the system degrades to "your verdict takes longer" instead of failing.

The sandbox

The core of the question. Untrusted code must not read other users' data, exhaust the host, call out to the network, or persist anything.

Layer 1  Container per submission (namespaces + cgroups)
           • read-only root filesystem, tmpfs scratch space
           • no network namespace at all
           • dropped capabilities, non-root user
           • seccomp-bpf allowlist of syscalls
Layer 2  Hard resource limits
           • CPU time (and wall-clock, to catch sleeps)
           • memory (cgroup limit, not just ulimit)
           • process/thread count (fork bomb defence)
           • output size (a print loop can fill a disk)
           • file descriptors
Layer 3  Disposable environment
           • fresh container per submission, destroyed after
           • the host is immutable; workers are cattle

Containers alone are not a security boundary against a determined attacker — a kernel vulnerability escapes them. For strong isolation, name gVisor (a user-space kernel intercepting syscalls) or Firecracker microVMs (hardware virtualisation with millisecond boot). That distinction is what separates a good answer from a generic one.

Container startup is the latency bottleneck, so keep a warm pool of pre-created sandboxes rather than cold-starting one per submission.

Test execution

1. Compile (own time limit — an infinite template expansion must not hang a worker)
2. For each test case: feed stdin, capture stdout, enforce limits
3. Compare output (exact, trimmed, or a special checker for float tolerance
   or problems with multiple valid answers)
4. Stop early on the first failure for a normal submission
5. Report the verdict plus the failing case, time and memory

Time limits must be scaled per language — the same algorithm in Python is several times slower than in C++, so a single global limit either fails correct Python or lets slow C++ through. State that; it is a favourite follow-up.

Test cases live in blob storage and are cached on the worker; large ones are streamed rather than loaded into memory.

Contest leaderboard

The interesting scaling piece. Fifty thousand participants, scores changing constantly, and everyone refreshing.

Redis sorted set per contest:  ZADD contest:123 score user_id
  • ZREVRANGE for the top N          → O(log N + M)
  • ZREVRANK for one user's rank     → O(log N)

A sorted set is exactly the right structure and gives both queries cheaply. Two refinements worth mentioning: rank by (score, -penalty_time) so ties break on who solved faster, encoded into a single sortable score; and freeze the leaderboard for the final period of a contest, which is both a competition tradition and a convenient load reduction.

Serve the leaderboard from cache with a short TTL rather than pushing every change to every viewer — 50,000 clients watching a live-updating list is a broadcast problem you do not need to take on.

Storage

problems     problem_id, title, description, difficulty, tags, limits
test_cases   problem_id, index, input_url, expected_output_url, is_sample
submissions  submission_id, user_id, problem_id, language, code, verdict,
             runtime_ms, memory_kb, created_at
user_stats   user_id, solved_count, streak, rating

Submissions are the fastest-growing table and are append-only. Partition by time, keep recent ones hot, and archive the rest.

Follow-ups to expect

• Anti-cheating — plagiarism detection across submissions (token-level similarity such

as MOSS), and detecting solutions submitted suspiciously fast.

• Custom judges — problems with multiple valid answers need a checker program, itself

sandboxed.

• Interactive problems — the solution talks to a judge process over stdin/stdout.
• Fairness under contest load — a per-user queue quota so one person cannot monopolise

workers.

• Deterministic timing — noisy neighbours make runtimes vary; pin CPUs and run the

timing measurement more than once.

• Cost — judge workers are expensive and idle most of the day; autoscale to near zero

between contests.

Track this problem on the System Design sheet.