From weeks to seconds: building “mega-assets” to uncover joyful moments!
Swati Sapar, Blaid Burgess, Jonathan Solorzano-Hamilton, Vinod Uddaraju
Netflix believes that great stories can come from anywhere and be loved everywhere. Creating and publishing the content to support this strategy poses a tremendous challenge: with thousands of titles available to over two hundred million members in dozens of countries, how do we help each member discover the next piece of content that will spark joy?
The answer lies in promotional assets. These assets play an important role in connecting our audience with the most relevant content. Promotional assets include both artwork and videos . Artwork includes the images that show up on Netflix service or other places such as roadside billboards.
Video assets include the trailers that give viewers a glimpse of various Netflix shows.
How can we bring more viewers to Netflix? By exciting them about our content. How can we excite viewers about our content? Through high-quality trailers! Our team builds the internal products that support the makers of our promotional videos. As engineers, we strive to improve and automate otherwise long tasks in the process of making trailers. This allows our creative partners to focus on the creative. In this article, we are going to share some of our recent successes enabling the creation of promotional videos.
Our vision is to focus on creative excellence in promotional assets that spark an interest in our viewers. We pursued two strategic goals in support of this vision:
- Give editors space to focus on creative decisions — they don’t have to do anything beyond choosing which moments to include for a given format; everything else is automatic
- Everything should be testable, including trailers and other promotional videos
The process of creating a promotional video starts with sourcing the raw material. Our creators first must cut the most interesting short-duration videos or “clips” from the main videos. Netflix’s internal tool “Clips” allows them to efficiently cut out clips from long-form videos and organize them in a useful manner.
This organizational feature allows our users to collect clips into projects and collaborate with their colleagues on projects of their choice. Users can also search for clips with certain dialogues, iconic moments, or objects that appear in a particular scene. They can see a quick preview of the videos and cut out the exact moments they need into precise clips with a precision up to 0.1microseconds. This helps them capture the most effective clips at a glance. For non-english users or content, Clips supports searching in the local language.
Once we built the collaboration platform for clips, we were faced with a new problem: enabling the creation of “mega-assets.”
Our marketing and promotional partners identified an opportunity to use “sizzle reels” to build excitement among our members around any genre. These sizzle reels would need to source brief clips from dozens of titles in our catalog, and we wanted to be able to mutate them quickly to support A/B testing.
Manually assembling these assets is tedious, expensive, and error-prone. Two particular errors show up most frequently while manually processing videos on the local machine: inaccurate placement of the “markers” that separate one clip from the next, and inaccurate metadata that identifies the source title and timecodes for the clip.
Creators determined that assembling a mega-asset takes them around 8 hours on average. Even one mistake with the marker placement could take additional days to weeks to clean up because it causes a “ripple effect,” breaking the remaining timecodes.
By using Adobe Premiere, we could meet editors where they were at and make a direct impact on their workflow. Our new “Magic Marker’’ tool implements automated marker placement and other Premiere integrations to support efficient automation for mega-assets in the creator workflow.
Considering that this is a source of error for most users, this was the best place to begin optimizing. The performance improvement of a system can only be achieved by focusing on the parts that didn’t perform well (Amdahl’s law). The simplest point of integration into Adobe Premiere is to create a Panel. Panels are simple extensions onto the already incredible Adobe products.
Our extension can programmatically go through each clip imported into Adobe Premiere and place markers at the specific points necessary for our users to build the mega-asset. We were able to accomplish this with a simple React app within our Adobe extension. Our panel gives the user an intuitive interface that they can interact with to dramatically optimize their workflow, without leaving the comfort of their Pro window.
Magic Marker is responsible for taking a process that would take 8 hours to potentially weeks down to a few seconds.
At Netflix, we believe that every member is just one story away from their next moment of joy. Our job at Netflix Studio is to help our creators tell the absolute best versions of their stories in the most efficient way possible. As engineers on the promotional assets team, we build the products that identify the best moments for our members to choose their next story. In this post we described how we met the challenge of delivering a novel type of asset: the genre-wide “mega-asset” sizzle reel. Stay tuned for a subsequent post where we’ll dive into another recent challenge that led us to experiment with a new type of mutable video asset!
Interested in solving problems like this? Studio Engineering is hiring! Check out our open roles or reach out to the authors on LinkedIn to learn more!
