A Complete Guide to Runway
8 minutes
Back in 2023, you might remember that hilariously unsettling AI-generated video of Will Smith eating spaghetti. Fast forward to 2025, and AI video generation like RunwayML has dramatically improved. While it's not yet convincing viewers that what they're seeing is real, the progress has been nothing short of astonishing.
RunwayML, with its Gen-3 Alpha model, allows users to generate videos from a static image, other videos, or simply a text prompt. Additionally, Runway offers AI video editing features including background removal, color grading, face blurring, and more.
But Runway isn't just limited to video; it also has a comprehensive suite of tools for generating and manipulating images, audio, and 3D assets.
Runway's extensive suite of video, images, audio and 3D tools.
Main Features & Functionality
1. Video Tools
-
AI Video Generation:
- Text-to-Video Generation: Create videos from text prompts
- Image-to-Video Generation: Bring static images to life with animated movements
- Frame Interpolation: Generate animated sequences from image sequences
-
AI Video Editing and Enhancement:
- Video Editing: AI-assisted editing and enhancement tools
- Text-Based Color Grading: Apply color presets using text prompts
- Super Slow Motion: Create smooth slow-motion effects from regular footage
- Face Blurring: Blur faces with AI
- Scene Detection: Automatically split scenes into clips
- Transcripts & Subtitles: Generate transcripts and subtitles with AI
-
Background and Object Manipulation:
- Background Removal: Automatically remove or replace video backgrounds
- Inpainting: Remove people or objects from videos
- Motion Tracking: Track objects within a video and apply effects
2. Image Tools
Runway offers a suite of AI-powered tools for image creation and enhancement, making it easy to generate, edit and refine visuals. Users can create new images from text, transform existing ones and expand compositions. Advanced editing features like Erase and Replace and Backdrop Remix allow for precise modifications, while Image Variation generates stylistic variations.
3. Audio Tools
Runway has a lip sync feature that lets you synchronise generated or uploaded audio with images or videos of people speaking. It also offers the ability to generate and clean audio, remove silence, or add AI generated subtitles or transcripts.
4. 3D Tools
Runway's 3D tools include 3D capture for multi-angle video manipulation and text-based 3D Texture Generation.
How to Get Started: Video Generation
While RunwayML has an impressive suite of features for advanced users, let's focus on its most known popular tool: video generation from an image or text prompt.
To start, navigate to Runway's home screen, click the Generate Video button from the left side panel. You should see the interface below.
Runway's video generation interface.
Step 1. Add an Image or Video (Optional)
From here, select an image or a video to use for the generated video to reference.
Runway's usage policy doesn't allow creating videos of celebrities without permission. So, as much as we'd love to bring another Will Smith eating spaghetti video to life, it's a no-go.
Step 2. Write a Prompt
Text-Only Prompting
When using text-only prompting, the following structure provides consistent results:
Prompt
Here's how an example prompt might look:
Prompt
Below the text prompt area, there's a button labelled "examples" that offers templates for a variety of prompts tailored to different use cases.
Runway's prompt templates.
Image + Text Prompting
When using a reference image, keep your text prompt simple and focused on movement rather than describing the image itself. Unlike text-only prompts, where you specify the scene in full detail, image + text prompting works best when you provide directional cues rather than redundant descriptions.
Here's an example prompt:
Prompt
Prompting Best Practices
- Be direct and descriptive: Give clear, concrete instructions. Instead of abstract or conceptual descriptions, focus on specific actions, appearances and settings.
Prompt
✅ A woman carefully paints a landscape on a canvas by a window.
❌ A woman expresses her creativity through painting.
- Keep prompts descriptive, not conversational: Unlike chat-based AI models, generative visual AI does not rely on natural conversation. Adding unnecessary pleasantries or framing prompts as commands can reduce their effectiveness. Instead, focus on direct descriptions.
Prompt
✅ A golden retriever joyfully chases a ball in a grassy park.
❌ Can you make a video of a dog playing fetch?
- Use positive phrasing: AI models work best when you describe what should be included, rather than what shouldn't. Instead of trying to exclude elements with "no" or "doesn't", reframe your description in a positive way.
Prompt
✅ The camera remains steady with smooth motion.
❌ The camera doesn't shake. No sudden movements.
Step 3. Select Settings
Choose the settings for your video:
- Aspect Ratio: Choose between 1280x768 (landscape) and 768x1280 (portrait)
- Fixed Seed: Toggle this on if you want similar results across different video generations
Camera Settings (a bit hidden - see screenshot below):
- Camera Control: Instruct Runway on how you'd like the camera to move during your video
- Static Camera: Toggle this on to keep the camera still
Where to find camera settings
Additional Features
Runway's additional features.
- Act-One: Record yourself acting out a 30 second or shorter clip and have the subject of your video mimic your facial expressions and sync to your lip and head movements.
- Expand Video: Transform your video into different aspect ratios using AI to expand the video
Image Generation (Unlimited and Enterprise Plans Only)
Frames, Runway's newest image generation model is currently limited to Unlimited and Enterprise Plan users. It excels at precise stylistic control, allowing you to generate variations that stay true to your aesthetic, whether you're aiming for retro 1980s SFX makeup, vibrant 1970s album art, dramatic landscapes or even magazine-style collages.
Prompting Best Practices with Frames
Runway suggests structuring your image generation prompts with this prompting style:
Prompt
Here's an example:
Prompt
Aesthetic Range
Runway's aesthetic range.
Tweaking the Aesthetic Range lets you choose how strictly you'd like Runway to adhere to your prompt. The lower the range, the closer it will be stylistically to the originally prompt, while higher ranges produce more extreme changes.
Best Use Cases
Product Shot Animation
Runway's video generation tools excel at creating product shot animations. Simply upload an image or a video of the product, and let Runway generate dynamic motion that highlights its features. Whether it's a rotating sneaker, a sleek car reveal or an interactive product demo, Runway can help bring static images to life.
Expressive Characters
For the artists among us, Runway's Act-One simplifies the process of bringing animated and live action characters to life. By uploading a video of yourself acting, you can transfer your facial expressions onto a fully generated character.
Traditionally, animating realistic facial expressions involves expensive motion capture equipment, multiple video references, manual rigging and intricate workflows. Runway gives you the option to eliminate these complexities by transposing an actor's performance directly onto your character.
Now, your characters can naturally mirror your emotions, opening new possibilities for storytelling, animation and creative expression.
Repurpose Footage
Runway allows creators to refresh old content without reshooting. Whether it's turning horizontal videos into vertical formats for social media, adjust lighting and color grading or even upscaling low-resolution footage, Runway's AI-powered tools make it easy to adapt and reuse footage across multiple platforms.
How does Runway compare to its competitors?
The AI video generation field is incredibly competitive and fast-moving. While RunwayML's Gen-3 Alpha was revolutionary when it came out in July 2024, it has now fallen behind state-of-the-art video generation models such as Veo 2 and Kling.
What Makes Runway Stand Out
- Video Generation Speed: Runway generates videos much faster than its competitors
- Comprehensive Suite of Tools: for generating and manipulating videos, images, audio, and 3D assets
- Ease of Use: While Runway's vast number of tools require a steep learning curve, its interface is fairly intuitive compared to many of its competitors
- Act One: This feature brings characters to life by transferring an actor's expressions, lip movements, and head gestures to an AI-generated character
Where Runway Falls Behind
- Noticeable Loss in Resolution: Generated videos often suffer from a significant drop in quality despite being standard 720p, with fuzzy areas that may be distracting
- Mediocre Physics: Inconsistencies in movement behavior can make scenes feel artificial
- Hallucinations: Many users report that Runway has a tendency to ignore prompt instructions and do its own thing
Pricing
Runway has several pricing tiers to cater to different user needs:
- Basic Plan: Free, with 125 one-time credits and limited features. Content is watermarked.
- Standard Plan: $12 per user per month (billed annually) or $15 per user per month (billed monthly). Includes 625 credits per month, 100GB storage, and more advanced features.
- Pro Plan: $28 per user per month (billed annually) or $35 per user per month (billed monthly). Offers 2250 credits per month, 500GB storage, and additional advanced features.
- Unlimited Plan: $76 per user per month (billed annually) or $95 per user per month (billed monthly). Provides unlimited video generations in Explore Mode and 2250 credits in Credits Mode.
- Enterprise Plan: Custom pricing, offering tailored solutions and advanced support options.
Conclusion
Runway has consistently been a key player in AI-driven video generation and editing, offering a broad set of tools for creators of all skill levels. While it excels in fast video generation and provides an extensive set of tools, it does have limitations such as resolution loss and hallucinations.
As AI video generation continues to evolve, Runway will need to keep innovating to stay competitive with newer models like Veo 2 and Kling. However, for those looking for a comprehensive set of video, image and audio tools or features like Act-One for character animation, it still provides valuable creative possibilities.
If you're curious to see what it can do, visit Runway's official website and try it out for yourself.
Kevin Law
Kevin Law is an English Second Language teacher with a passion for Generative AI, particularly in creating personalised and engaging learning experiences. With expertise in integrating AI tools into education, Kevin bridges the gap between traditional teaching methods and innovative technologies to enhance student outcomes. He is committed to making learning more accessible and effective for diverse learners worldwide.