For television broadcasting and video streaming, video metadata is used to provide information about specific video and audio streams or files, also known as essence. Metadata can either be embedded directly into the video or included as a separate file within a container such as MP4 or MKV.

Metadata can include information about the entire video stream or file or specific video frames. Created by cameras, encoders, and other video processing elements, metadata can include timestamps, video resolution, file size, closed captioning, audio languages, ad-insertion points, color spaces, error messages, and much more.

For the metadata to be useful, it must be recognized by whichever element or device it is intended for. For example, a set-top box needs to be able to recognize metadata about a video’s picture resolution in order to display it in the right format. A transcoder must recognize SCTE-35 metadata in order to identify targeted ad-insertion points while an HDR capable television set needs to be able to interpret metadata in order to enhance the image.

Once a live video has been captured and recorded, further metadata can be included for describing the content allowing users to search for content without having to sift through potentially long video files.