Video Search — Scrubbing Hell !
“A picture is worth a thousand words” is how the old adage goes. And a video contains 24 pictures per second. Now, just imagine how much information is stored in a full-length video!
But alas, Video Search methodologies today are as old as the adage itself. Videos are cool, but less searchable. The only way to search the contents of a video is through Scrubbing. Labouring through millions of words worth of information, what a Scrubbing Hell!
The Scrubbing Hell
So, my work involves numerous zoom meetings with clients, and I record each session for later reference, when I have to prepare a monthly Report. And that is the day I have to face tens of videos to search for the content that I want. But how do I do it?
Currently I have limited options. I have a vague idea about the video which might contain the topic I am looking for, and an even more vague guess about the time-spot within the video. So I sift through all the videos in my folder and when I find the correct video (presumably), I open it up and scrub the progress bar back and forth until I reach the exact moment in the video (hopefully soon). By the end, I have performed this mundane and exhausting task so many times, that my mental CPU is overheated!
Do you see it? The problem I am facing is two-fold : Video Discovery and Video Search
This is an Enterprise-level problem. Many organisations are sitting on a huge load of video and audio content, millions of words worth, which have become a black box. You don’t know that you don’t know. The blob of binary data could contain anything!
This is a classical blind-spot problem. But why are we facing this with videos?
Make Videos Searchable
Tagging / TOC Approach
You can always tag your videos. YouTube also follows this approach. You attach meta-data around the video and make the meta-data searchable. But these methods are limited in their efficacy.
Meta-data approach and other manual methods like organisation of videos into folders and categories make these assets only somewhat searchable. Once I find the correct video, it is still a laborious task to scrub through the video to reach the time-stamp which contains the info I want.
The problem is that my video assets are not indexable, and therefore not easily searchable as my text-based assets like PDFs and Word documents.
Speech Based Indexing
This is where we have created a tool.
This approach involves indexing your video based on its speech transcript.
You can upload your video and audio assets to our platform, and we will make them discoverable and searchable for you.
From video, we extract out the audio, and then create a transcript of the extracted audio. This transcript is then indexed and mapped to various points on the original video.
Now you need only search a few words you remember hearing in the video, and the correct video would be fetched for you.
All the timestamps where the said word or phrase is spoken gets highlighted.You can simply click on the highlighted timestamp to make the video jump to the particular spot.
You can now save yourself from the Scrubbing hell of videos and make them as easily searchable as your text-based assets
Limitation of this approach though is that it depends on the quality of audio in the video, so most of the time it works great for recorded web conferences / webinars.
The Next Frontier : Object Based Search
The next battle in this domain lies in Object Recognition based video search. As I said earlier, a video contains 24 images per second, and an image consists of visual data, aka Objects.
This approach splits up the video into individual image frames, and then generates Tags and Annotations using Image Classification / Extraction algorithms. These tags are then mapped to timestamps on the original video.
You can search through these indexed tags and arrive at the correct point in the video where the tagged object is shown.
Limitations of this approach is that it works well if its an outdoor video which many real objects like a car, boy, cricket ball etc, but when it comes to recording of software application, we see this trick running out of its rope.
We do wish that some day we land into a no / less scrub search, till then happy scrubbing !
Authored by : Aashish Tyagi (aashish@stratbeans.com) & Prasoon Nigam (prasoon@stratbeans.com)
