Searching Through an Index

Search allows you to find portions of a video through natural language. Results are categorized into three types: visual embeddings, transcriptions, and optical character recognition (OCR).

To search:

# Search with natural language
resp = moonshine.search(index="your-index-name", query="red car with people shaking hands inside")

# resp = {'status': 'complete', 'output': {'embeddings': match[], 'transcript': match[], 'ocr': match[]}}

Every match object is an individual result that matches the query. A list of match objects is returned for every category.

A match object has the following data:

IndexPropertyExample Value
0Moonshine assigned video fileID name’FHXYU838JHDWK.mp4’
1Match Frame Start600
2Match Frame End725
3Match Timestamp Start’0 min, 22 sec’
4Match Timestamp End’0 min, 24 sec’

Searching With an Image

You can search through an index with an image. To search with an image:

resp = moonshine.search(index="your-index-name", image="./image.png")

# resp = {'status': 'complete', 'output': {'embeddings': match[], 'transcript': match[], 'ocr': match[]}}