76AI systems challenges·no GPU required

Build modern AI from scratch.

Implement the internals of modern AI — from attention to RLHF, diffusion models, and distributed training. Real PyTorch tests, instant feedback.

Total

76 problems

Coverage

13 categories

Runtime

CPU · local

Judge

torch_judge

attention / causal_self_attention.pyMEDIUM · 38%

1# Implement causal self-attention.

2import torch

3import torch.nn.functional as F

5def causal_attention(q, k, v):

6 d = q.size(-1)

7 scores = q @ k.transpose(-2,-1) / d**0.5

8 T = q.size(-2)

9 mask = torch.triu(torch.ones(T,T), 1).bool()

10 scores = scores.masked_fill(mask, float('-inf'))

11 return F.softmax(scores, -1) @ v

Tests · 4

✓shape_check3.2ms

✓masked_entries4.9ms

✓softmax_rows4.1ms

·grad_flowrunning

MultiHeadAttentionFlash Attention (tiled)Rotary Position EmbeddingDPO LossGRPO LossSpeculative DecodingPaged AttentionLoRA / QLoRAMamba SSMMixture of ExpertsFSDP Training StepRing AttentionFlow MatchingadaLN-ZeroMulti-Token PredictionMultiHeadAttentionFlash Attention (tiled)Rotary Position EmbeddingDPO LossGRPO LossSpeculative DecodingPaged AttentionLoRA / QLoRAMamba SSMMixture of ExpertsFSDP Training StepRing AttentionFlow MatchingadaLN-ZeroMulti-Token Prediction

§ 03 — How it works

Read the paper, then write the code.

Instant Feedback

Pass/fail per test case, execution time, and error messages — right after you submit.

Real Tests

Tests run against actual PyTorch — no mocks, no shortcuts. If it passes here, it works.

Progress Tracking

Track solved problems, best times, and attempt counts across sessions.