MethodologyAnnotator qualityUpdatesHugging Face436.7k downloadsSuggest a model or benchmarkRequest a private benchmark
Benchmarks
Dimensions
License
Models
Categories
category
source
LLM to Animation v1updated 2026-11-19

Language Models: Animation Benchmark

Rapidatabenchmark@rapidata.ai

Abstract

When a new language model comes out, one of the most common ways to show it off is a three.js animation: a single clip of a scene the model wrote in code, chosen by the person posting it. This benchmark ranks language models on that task under fixed conditions. Every model receives the same 200 prompts, drawn from requests people post on X and type into chatbots and widened with scene descriptions from video benchmarks, and writes one HTML page per prompt that plays a three.js animation. Each page is recorded as a 7-second clip, and human annotators compare the clips of two models for the same prompt, without seeing the prompt or the code, and say which one they would continue watching and which one they like more. The 925,025 responses collected this way are aggregated into ratings with a Bradley–Terry model; the details describe how the prompts were chosen and the clips made, and the methodology covers matchmaking and score estimation.

LeaderboardCompare OutputsMatchupsAnalysisPromptsModel settingsDetails
#
Model
Organization
Score
95% CI
Interval
Responses
Matches
Win rate
#
Model
Score

Table 1. Overall standings, LLM to Animation benchmark. Scores from a Bradley–Terry fit rescaled to Elo; intervals are 95% Wald intervals. Click any row to view sample outputs in Figure 1.

Methodology notesCompare Outputs
Figure 1. Sample outputs from gemini-4-neon on representative prompts from the fixed set. Outputs shown are those used in matches.
Score vs. cost
80010001200$0.005$0.02$0.1$0.5$2cost, USD per sample (log scale)score

Figure 2. Overall Elo of each model against its mean cost per sample, on a logarithmic axis. A sample’s cost is the sum of the charges the provider reported for the calls that produced it, and a model’s cost is the mean over its samples. Error bars are 95% confidence intervals of the score. Models ringed in teal and joined by the step line form the Pareto frontier: no other model has both a lower cost and a higher score. The remaining models are faded.

Evaluation: Models are compared on the same prompts through pairwise human judgements.
Score: Scores show relative performance within this benchmark. A higher score means a model was more likely to be preferred over the other evaluated models.
Uncertainty: The 95% confidence interval shows the uncertainty around each model’s estimated score.
Views: Results can be explored by continue watching, preference, and by prompt category.
Coverage: This benchmark includes a fixed prompt set across multiple categories, evaluated on all ranked models.
Data: Prompts, model outputs and standings are readable on this record; no downloadable dataset is released for this round yet.
Version: Results correspond to LLM to Animation v1, last updated on 2026-11-19.

We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.

Benchmarks
Text-to-ImageOutput Diversity (Mode Collapse)Multi-Step Image Editing (Identity Drift)Camera MovementRelative Camera MovementPhysicsPhysicsRealismImage-to-3DSVGLLM-to-3DLLM to AnimationSuggest a model or benchmark
Method
MethodologyData on Hugging Face
Rapidata
rapidata.ai Private benchmarksContactX LinkedIn
Rapidata, benchmark.ai © 2026