The problem
Media companies sit on large video libraries, but finding a specific moment usually means knowing the right file and scrubbing through it. Metadata describes the asset, not what happens inside it — so most of the footage is effectively invisible to search.
The aim was to make that content usable again: find a scene by describing it, then turn what you find into something new.
What I built
Three extraction paths run over each piece of content: video understanding to describe scenes and shots, audio parsing to capture speech and sound, and metadata enrichment to attach titles, people and tags. Their output is combined into a single index at the level of a scene, so a query lands on a moment rather than a file.
On top of that index sit agents that turn search results into new content: resized highlights for social media, summary podcasts and articles.
The hard part
Each signal arrives at a different grain — speech by the word, shots by the frame, metadata by the asset. Getting them onto one timeline, and deciding what counts as a “scene,” is where most of the design decisions live.