Methodology
The only comparison that shows the real cost of a video that actually works, and warns you before your model gets shut down.
How we test
- Blind, same-prompt testing. Every model gets the identical prompt set, identical duration and identical resolution target. Raters never see which model produced which clip.
- Weekly re-runs. Scores and Elo are recomputed every week, so vendor updates and silent regressions show up in the 7-day delta.
- Elo from pairwise votes. Raters pick a winner per prompt pair; Elo is derived from those matchups, not from a single quality opinion.
- Prices are real prices. We track per-provider list pricing and observed latency, including retries, because a cheap render that fails is not cheap.
- Deprecation warnings. When a vendor announces a sunset date, the model is flagged everywhere on the site.
Independence
We take no vendor sponsorships and no paid placements. We do earn affiliate commission when you sign up through a provider link. Ranking order is produced entirely by the blind test pipeline and is never adjusted for commercial reasons.
Team
Benchmark lead
Placeholder bio
Raters panel
Placeholder bio
Data engineering
Placeholder bio
Contact
Corrections, missing models or pricing changes: hello@vidbenchmark.com
Deployment and hosting
- Current preview
- https://id-preview--722617b0-aba9-475d-846c-7209f2332ad4.lovable.app
- Published URL
- https://ai-vid-score.lovable.app
- Custom domain target
- vidbenchmark.com (DNS on Cloudflare)
- DNS setup
- CNAME vidbenchmark.com to ai-vid-score.lovable.app. Keep the Cloudflare proxy (orange cloud) OFF initially to avoid a redirect loop; it can be enabled once the certificate is issued.
- SSL
- Provisioned automatically by Lovable hosting.
- CI
- Merges to main build with Bun and deploy to Cloudflare Pages (project vidbenchmark) via wrangler-action.
- Sample video storage
- Cloudflare R2, naming convention documented in /README-samples.txt.