ClipForge - AI Agent that Turns Long-Form Video into Multi-Platform Clips and Captions
Upload a 30-minute video, the AI agent auto-generates 8-12 short-form clips (optimized for TikTok, Reels, Shorts), extracts captions, adds branded watermarks, and queues them for publishing. One upload, 10 pieces of content.
Difficulty
intermediate
Category
Creator Tool
Market Demand
Very High
Revenue Score
8/10
Platform
Web App
Vibe Code Friendly
No
Hackathon Score
🏆 9/10
What is it?
Content creators spend 4-6 hours per week repurposing a single long-form video (podcast, course, webinar) into short clips for TikTok, Instagram Reels, and YouTube Shorts. They manually trim, caption, add graphics, and manage uploads. ClipForge automates this with an AI agent that ingests a video file, uses Claude Vision to identify the most engaging 30-90 second segments (based on speaker passion, topic shifts, key points), generates captions (via Deepgram or AssemblyAI API), and outputs files optimized for each platform (1080x1920 for Reels, 1080x1920 for Shorts). It stores files in Supabase Storage or AWS S3, generates a dashboard showing all clips, and optionally integrates with Buffer or Meta's Graph API to auto-publish. Why 100% buildable now: FFmpeg is stable and has Node bindings. Claude Vision can analyze video keyframes cheaply. Deepgram and AssemblyAI provide reliable speech-to-text. No custom ML training needed—Claude Vision identifies engaging moments, Deepgram handles transcription.
Why now?
Claude Vision (released 2024) is now fast and cheap enough for frame-by-frame video analysis. Deepgram and AssemblyAI APIs are sub-$100/month affordable. Short-form content consumption is at all-time high (TikTok, Reels, Shorts driving discovery). Long-form creators are desperate for repurposing automation.
- ▸Upload long-form video, Claude Vision analyzes keyframes and identifies 8-12 engaging segments (30-90 sec each)
- ▸Deepgram transcription and auto-caption generation, exported as .srt or burned into video
- ▸FFmpeg-powered video optimization: aspect ratio and resolution for TikTok, Reels, Shorts
- ▸Branded watermark overlay and intro/outro templates
- ▸Clip gallery dashboard with preview, approval, and bulk export/publish workflow
Target Audience
YouTube creators (500k channels with 10k+ subs), podcasters (50k regular publishers), course creators (100k), B2B companies with webinars (50k). TAM: 700k content creators producing weekly long-form content.
Example Use Case
Marcus, a YouTube creator with 500k subscribers, publishes a 45-minute podcast every Friday. Manually creating 8-10 TikTok clips takes 6 hours. ClipForge auto-generates 10 clips in 15 minutes, he approves and schedules them. He gains 3-5 more pieces of content per week without extra effort, driving a 40% increase in cross-platform views and a $2k/month sponsorship deal from incremental reach.
User Stories
- ▸As a YouTube creator, I want to automatically extract the most engaging 30-second clips from my 45-minute videos so that I can feed TikTok and Reels without manual editing.
- ▸As a B2B marketing manager, I want my webinar to be sliced into 8 clips optimized for LinkedIn and Twitter so that I can run a 2-week content series from one 1-hour recording.
- ▸As a podcast producer, I want captions and watermarks auto-added to my clips so that I can maintain brand consistency across platforms.
Acceptance Criteria
Video Upload: done when user can upload a .mp4 file without errors and it appears in the dashboard. Keyframe Analysis: done when Claude Vision analyzes 45-minute video and selects 10 segments with > 70% engagement accuracy (spot-checked by user). Transcription: done when Deepgram returns captions for full video in under 5 minutes. Clip Generation: done when FFmpeg produces optimized .mp4 files for TikTok (1080x1920), Reels (1080x1920), and Shorts (1080x1920). Dashboard: done when gallery loads in under 3 seconds and shows all clips with previews.
Is it worth building?
$49/month × 80 creators = $3,920 MRR at month 3. $149/month × 30 studios = $4,470 MRR. Combined: $8,390 MRR at month 4.
Unit Economics
CAC: $100 via Twitter/X DM outreach (20 DMs, ~1 in 10 converts, $10 founder time cost per DM). LTV: ~$156 (average retention 4 months at $49/month based on typical early SaaS churn of 15-20%/month; users who stay 12 months contribute $588 but median churn makes blended LTV ~$156-$220). Payback period: 2-3 months. Gross margin: ~65-70% (Deepgram + Claude API costs run $0.30-$0.80 per video processed; at 50 videos/month per user the variable COGS is $15-40/user/month against $49 revenue).
Business Model
SaaS subscription: $49/month (50 videos/month), $149/month (unlimited) + pay-as-you-go for Deepgram and Claude
Monetization Path
Free tier: 1 video/month, 3 clips generated. Paid: 50+ videos/month, unlimited clips, Slack integration.
Revenue Timeline
First dollar: week 4 (beta converts to paid). $1k MRR: month 2. $5k MRR: month 4. $10k MRR: month 7.
Estimated Monthly Cost
Claude Vision: $80 (assume 500 videos/month × 20 keyframes per video × $0.008 per frame). Deepgram: $100 (assume 500 hours of video = 30k minutes × $0.003 per minute). AWS S3 or Supabase Storage: $50. Vercel: $20. Stripe fees: ~$30. Sentry: $0 (free). Total: ~$280/month at launch (will vary with usage).
Profit Potential
Full-time viable at $6k–$20k MRR. High-margin product (COGS is just API fees).
Scalability
High — can add Twitch integration, auto-publish via Meta/TikTok APIs, AI-generated animations, music sync licensing.
Success Metrics
Week 1: 150 signups. Week 2: 30 paid trials. Month 2: 60 paid customers, 70% retention.
Launch & Validation Plan
Survey 40 YouTubers and podcasters on Reddit r/YouTubers and Twitter/X. Build landing page with 3-minute demo video. Recruit 8 beta testers (mix of YouTube creators, podcasters, B2B companies) for 2-week free trial. Measure: videos uploaded, clips generated, approval rate, time saved.
Customer Acquisition Strategy
First customer: DM 15 YouTube creators on Twitter/X offering 2 months free in exchange for feedback and a short testimonial video. Second: ProductHunt launch, Reddit r/YouTubers and r/podcasting, Twitter/X threads showing before/after clip galleries, TikTok account demonstrating your own tool's clips.
What's the competition?
Competition Level
Medium
Similar Products
Opus Clip (direct competitor, AI-powered highlight extraction and auto-captioning). Vidyo.ai (long-form to short-form repurposing, direct overlap). Descript (podcast and video editing with clip export, less automated). Pictory (video-to-clips with captions, older UX and slower processing).
Competitive Advantage
End-to-end automation (no manual clip selection or editing). Multi-platform optimization (TikTok, Reels, Shorts in one go). Simpler than Adobe Premiere or DaVinci Resolve for this specific use case.
Regulatory Risks
Copyright: users must own or have rights to video content. Music licensing: if music is in the video, clips inherit the license (user's responsibility). GDPR: store minimal user data, provide deletion endpoint.
What's the roadmap?
Feature Roadmap
V1 (week 3): Upload, Claude Vision segmentation, Deepgram captions, FFmpeg optimization, gallery. V2 (month 2-3): Buffer/Meta auto-publish, custom intro/outro templates, music library integration. V3 (month 4+): AI-generated voice-over, animated B-roll, team collaboration, white-label reselling.
Milestone Plan
Phase 1 (Week 1): Build Next.js scaffold, Supabase schema, S3 setup, FFmpeg keyframe extraction. Done when: user can upload video and see keyframe grid. Phase 2 (Week 2): Claude Vision agent, Deepgram integration, clip cutting logic. Done when: agent selects 10 segments and Deepgram returns full transcription in test. Phase 3 (Week 3): v0 gallery, Stripe billing, 8 beta testers on-boarded. Done when: first paying customer uploads video and downloads clips.
How do you build it?
Tech Stack
Next.js, Claude Vision API, Deepgram API, FFmpeg, AWS S3 or Supabase Storage, Stripe, Buffer API (optional) — build with Cursor for backend AI agent, v0 for clip gallery. Install real packages: @deepgram/sdk, fluent-ffmpeg, @anthropic-ai/sdk, @aws-sdk/client-s3.
Suggested Frameworks
LangChain for Claude Vision agent orchestration, fluent-ffmpeg (node-fluent-ffmpeg) for video processing, @deepgram/sdk for transcription
Time to Ship
5 weeks
Required Skills
Claude Vision API, FFmpeg, speech-to-text integration, video file handling
Resources
Claude Vision docs, FFmpeg docs, Deepgram docs, LangChain docs, AWS S3 or Supabase Storage docs
MVP Scope
Create: (1) Next.js app with Supabase auth and Stripe billing. (2) File upload endpoint that accepts video files (S3 or Supabase Storage). (3) FFmpeg service to extract keyframes (every 10 seconds) and create thumbnail grid. (4) Claude Vision agent (/api/ai/analyze) that scores keyframes for engagement (passion, topic shift, key point) and selects top 10 segments. (5) Deepgram integration (/api/transcribe) for speech-to-text. (6) FFmpeg service (/api/video/clip) that cuts segments and optimizes for TikTok/Reels/Shorts. (7) Caption rendering service (/api/video/caption) that burns .srt into video or exports .srt file. (8) Supabase schema: users, videos, clips, transcriptions. (9) v0 dashboard gallery showing clips with thumbnails, preview, and download buttons. (10) Stripe webhook for subscription lifecycle.
Core User Journey
Sign up -> upload video -> wait 10 minutes -> view clip gallery -> approve clips -> download or auto-publish -> upgrade to paid.
Architecture Pattern
User uploads video -> S3/Supabase Storage -> FFmpeg extracts keyframes (every 10s) -> Claude Vision scores each frame (agent loop) -> top 10 segments selected -> Deepgram transcribes full video -> FFmpeg cuts and encodes clips for each platform -> captions burned/exported -> clips stored in S3 -> dashboard displays gallery -> user approves -> Buffer or Meta API publishes.
Data Model
User has many Videos. Video has many Clips. Clip has one Transcription. User has one PublishingPreference (platform targets, watermark, intro/outro).
Integration Points
Claude Vision API (Anthropic) for keyframe engagement analysis, Deepgram API or AssemblyAI API for transcription, FFmpeg (via fluent-ffmpeg) for video processing, AWS S3 or Supabase Storage for file storage, Stripe for billing, Buffer API (optional) for multi-platform publishing.
V1 Scope Boundaries
V1 excludes: real-time processing, live streaming integration, multi-user team accounts, AI voiceover or dubbing, white-label.
Success Definition
A paying YouTube creator uploads a video, receives 8+ clip files optimized for TikTok/Reels/Shorts with captions within 10 minutes, approves one clip without any founder help, and renews after 30 days.
Challenges
Video processing is CPU-intensive and slow on serverless (Lambda cold starts > 30sec). Claude Vision token costs can spike if videos are long (need smart keyframe sampling). Deepgram API has rate limits.
Avoid These Pitfalls
Do not run FFmpeg on Vercel serverless functions — 512MB memory limit and 10-second timeout will kill any real video job; use a persistent worker on Render or EC2 from day one. Do not send full video frames to Claude Vision without sampling — a 45-minute video at 1 frame/sec is 2,700 frames; sample every 10-15 seconds or costs explode to $20+ per video. Do not assume Deepgram timestamps align perfectly with FFmpeg cut points — add 0.5s padding on each clip boundary or cuts will feel abrupt. Do not build auto-publish to TikTok in V1 — TikTok's Content Posting API requires app review and has strict rate limits; start with download-only and add publishing later. Do not ignore aspect ratio metadata — videos shot in 4K 16:9 need explicit crop coordinates for 9:16 output, not just a resize; missing this produces pillarboxed clips that look unprofessional. Do not store raw uploaded videos indefinitely — implement a 30-day TTL on S3 or storage costs compound fast at scale.
Security Requirements
Auth: Supabase Auth with email or Google OAuth. RLS enabled on videos and clips tables. Rate limiting: 5 uploads per hour per user to prevent abuse. Input validation: file size max 2GB, only video formats accepted. GDPR: data deletion endpoint removes user videos and clips.
Infrastructure Plan
Hosting: Vercel (Next.js frontend). Background processing: Render.com or EC2 (FFmpeg jobs). Database: Supabase Postgres. Storage: AWS S3. CI/CD: GitHub Actions. Environments: dev, staging, prod. Monitoring: Sentry. Cost: Vercel $20, Render.com $7 (starter), Supabase $25, S3 $30 (variable), Sentry $0 (free), Total: $82/month base + usage.
Performance Targets
Expected DAU at launch: 30. Video processing time: 45-minute video processed in under 10 minutes (keyframe extraction + Claude + Deepgram + clip encoding). API response time: Claude Vision under 2 seconds per frame. Dashboard load: under 3s (LCP). S3 upload: multipart upload for large files.
Go-Live Checklist
- ☐✓ FFmpeg installed and tested on Render.com instance (keyframe extraction, clip cutting)
- ☐✓ Claude Vision tested on 50 video keyframes (accuracy > 70%)
- ☐✓ Deepgram tested on 10 different audio tracks (accuracy > 90%)
- ☐✓ Clip encoding tested for TikTok, Reels, Shorts aspect ratios
- ☐✓ S3 upload and download tested end-to-end
- ☐✓ Background job queue tested (job completes without timeout)
- ☐✓ Stripe billing tested (free trial to paid conversion)
- ☐✓ Error tracking live (Sentry captures job failures)
- ☐✓ Monitoring dashboard shows processing times
- ☐✓ Custom domain configured
- ☐✓ Privacy policy and terms published
- ☐✓ 8 beta testers signed off (processed 2+ videos each)
- ☐✓ Rollback plan: revert Vercel and Render deployments
- ☐✓ ProductHunt and Twitter launch posts drafted.
How to build it, step by step
1. Scaffold project: npx create-next-app@latest clipforge --typescript, then npm install @supabase/supabase-js @anthropic-ai/sdk @deepgram/sdk @aws-sdk/client-s3 fluent-ffmpeg stripe bull ioredis. 2. Define Supabase schema: users table (id, email, plan, stripe_customer_id), videos table (id, user_id, s3_key, duration_seconds, status enum[uploading|processing|done|failed], created_at), clips table (id, video_id, start_sec, end_sec, platform enum[tiktok|reels|shorts], s3_key, caption_srt_key, engagement_score), transcriptions table (id, video_id, full_text, words_json). Enable RLS: videos and clips readable only by owner. 3. Build /api/upload: accept multipart .mp4/.mov up to 2GB, stream directly to S3 using @aws-sdk/client-s3 multipart upload (not buffering in memory), insert video row with status='uploading', return video_id. 4. Set up Bull queue (Redis-backed) on a Render.com worker service: queue named 'video-processing', one job per video_id. On upload success, enqueue job. Worker must run on Render (not Vercel) where FFmpeg binary is installed via apt-get install ffmpeg in render build command. 5. In the Bull worker, step 1 of processing: use fluent-ffmpeg to extract one JPEG frame every 12 seconds from the S3-downloaded video file (ffmpeg -i input.mp4 -vf fps=1/12 frame_%04d.jpg), upload frames to S3 under frames/{video_id}/. 6. In the Bull worker, step 2: call Anthropic Claude claude-3-5-sonnet via @anthropic-ai/sdk with each keyframe image (base64-encoded) and prompt: 'Score this video frame 0-10 for: speaker emotional intensity, presence of on-screen text/diagram, mid-sentence topic transition. Return JSON {intensity, visual_info, topic_shift}.' Batch frames in groups of 5 to reduce API round trips. Store scores in memory. 7. In the Bull worker, step 3: group consecutive high-scoring frames (score >= 7) into candidate segments, merge segments within 5 seconds of each other, enforce 30-90 second duration constraint, select top 10 by average score. Write selected segments as rows in clips table with start_sec, end_sec, engagement_score. 8. In the Bull worker, step 4: call Deepgram @deepgram/sdk transcribeFile() on the full video audio (extract audio first with ffmpeg -i input.mp4 -vn audio.mp3), request paragraphs:true, utterances:true. Store full transcript in transcriptions table. For each clip, slice the Deepgram words_json array by timestamp range to generate per-clip .srt content, upload .srt to S3 under captions/{clip_id}.srt. 9. In the Bull worker, step 5: for each clip and each platform (tiktok, reels, shorts — all 1080x1920 but with different bitrate targets), run FFmpeg: trim segment (ss start_sec -t duration), scale and crop to 1080x1920 using crop=1080:1920:(iw-1080)/2:(ih-1920)/2 for landscape source, burn .srt captions using subtitles filter, overlay watermark PNG using overlay filter. Upload output .mp4 to S3 under clips/{clip_id}/{platform}.mp4. Update clip row with s3_key, set video status='done'. 10. Build Next.js gallery page /dashboard/[videoId]: fetch clips from Supabase, render card grid with <video> preview tag (signed S3 URL), engagement score badge, platform tabs, SRT download button, and approve/reject toggle. Use v0 to generate the card component layout. 11. Add Stripe: create products for Starter ($49/mo, 50 videos) and Pro ($149/mo, unlimited) in Stripe dashboard, implement /api/stripe/webhook to handle checkout.session.completed and customer.subscription.deleted, update user plan in Supabase on webhook receipt. 12. Deploy: Next.js to Vercel (set ANTHROPIC_API_KEY, DEEPGRAM_API_KEY, SUPABASE_URL, STRIPE_SECRET in env). Bull worker to Render Web Service (set same env vars plus REDIS_URL from Render Redis add-on). Test end-to-end with a real 10-minute .mp4 before inviting beta users.
Generated
March 30, 2026
Model
claude-haiku-4-5-20251001 · reviewed by Claude Sonnet