​
What is Search?

Search lets you locate exact moments across thousands of hours of video using natural language or an image reference. Results are split into three specialized buckets—embeddings, transcript, and ocr— So you can quickly see why something matched.

The path to use search:

core/search

To search with natural language:

resp = moonshine.run(
    flow="core/search",
    index="your-index-name",
    query="your search query, ex: red car",
    num_args=10  # optional, defaults to 5
)

Where num_args are the max number of results returned and is optional.

To search with an image:

resp = moonshine.run(
    flow="core/search",
    image="./reference.png",  # local path or public URL
    image="path/to/image.jpg",
)

If both an image and query are supplied, the image will take precedence.

Using a Subindex will isolate a search a subset of the videos in an index. To use it:

resp = moonshine.run(
    flow="core/search",
    index="your-index-name",
    query="your search query, ex: red car",
    subindex=[
        "FHXYU838JHDWK.mp4",
        "AJDIEUJDKSLW.mp4"
    ],
)

where the subindex is a list of video IDs to search within.

​
Search Response Format

Search returns a list of match objects. The match object schema is:

​
[0]
string

Moonshine-assigned file_id

e.g. FHXYU838JHDWK.mp4

for match in resp["output"]["embeddings"]:
    file_id      = match[0]
    start_frame  = match[1]
    end_frame    = match[2]
    start_time   = match[3]
    end_time     = match[4]

​
Search JSON

TODO