media-metadata-examples

Media Metadata Examples

This is an exploratory page to be used for storage of various multimedia data and metadata profiles, formats, and practices currently in use around the web. As this is a very, very early exploration, we should include as many types of multimedia as possible in the opening discussion - but please be sure to place your information in the correct portion of this page.

Purpose of this exploration (or : The Problem)

The purpose of the studies on this page is to determine the feasibility and demand for a flexible, semantic markup format for multimedia related paradigms.

This includes such things as:

• Providing metadata alongside linked multimedia files.

  Typically, multimedia metadata is stored within the header of the media file itself - which has massive implications for any application where metadata is to be indexed, searched, or made externally accessible. Of course, even if you could easily access the correct portions of a media file remotely, you'd still have to cope with a multitude of open and proprietary metadata formats, each with it's own distinct fieldset, nomenclature and storage method.

• What is called: Internet TV, IPTV, NewTube, vlogging, vidblogging, vidlogging, vodcasting, etc.
• What is called: Podcasting, IPradio, etc.

This study aims to make a start at solving this problem.

Contributors

These folks have contributed real-world examples and research to this document, and are interested in helping with a solution.

• Christopher Allen
• Dougal Campbell
• Dr. Ernie
• Rohit Khare
• Ryan King
• Kevin Marks
• Lisa Rein
• Mary Hodder
• Dan Glegg
• Chris Messina
• Charles Iliya Krempeaux
• Thomas Winningham
• William Sandler

If you contribute new real-world examples or research, and wish to help with a solution, please add your name as well.

Examples on the Web today

Below is a list of strawman models based on real work examples. (Links to actual examples have been omitted since many of them have been found on adult sites, and it is believed that it would be inappropriate to list these here. -- Charles)

Show Strawman #1

Single clip with a preview image. The clip is the full show.

   
    <a href="clip.mpeg"><img src="preview.png" /></a>
    

Show Strawman #2

Single clip with a preview image. The clip is the teaser of the full show. And there is a link that you can go to pay to see the full show.

   
    <a href="teaser.mpeg"><img src="preview.png" /></a>

    <a href="http://example.com/go">Pay to View</a>
    

Show Strawman #3

A show divided up in multiple clips, each with a preview image. Together the clips make up the show.

   
    <a href="clip-1.mpg"><img src="preview-1.png" /></a>

    <a href="clip-2.mpg"><img src="preview-2.png" /></a>

    <a href="clip-3.mpg"><img src="preview-3.png" /></a>
    

Show Strawman #4

A set of teasers for a show. Together they don't make up the full show. And the may or may not overlap in time. There's a "Pay to View" link there too.

   
    <a href="clip-blue.mpg"><img src="preview-blue.png" /></a>

    <a href="clip-red.mpg"><img src="preview-red.png" /></a>

    <a href="clip-green.mpg"><img src="preview-green.png" /></a>


    <a href="http://example.com/go">Pay to View</a>
    

Show Strawman #5

A single clip that comes in different formats. (Could be a teaser, a clip, or a full show.)

   
    <img src="preview.png" />
    <a href="clip.mpg">MPEG</a>
    <a href="clip.ogm">Ogg</a>
    <a href="clip.avi">AVI</a>
    

Show Strawman #6

A single preview image with the show given in 2 different methods. The first is the show broken up into a series of pieces (which makes it easier to download.) And the second is the full show in one file

   
    <img src="preview.png" />

    <a href="part-1.mpg">[1]</a>
    <a href="part-2.mpg">[2]</a>
    <a href="part-3.mpg">[3]</a>

    <a href="full.mpg">full</a>
    

Show Strawman #7

A single clip with a preview image. The title of the show is given with an image (and thus the actual machine-readable title is the alt attribute of the image) and is linked with the main URL for the show. There's also a link to older episodes of the show.

   
    <a href="..."><img src="the-name-of-the-show-png" alt="The Name of the Show" /></a>

    <a href="episode123-clip.mpeg"><img src="episode123-preview.png" /></a>

    <a href="older-shows">older shows</a>
    

Show Strawman #8

A series of clips with preview images. The title of the show is given with an image (and thus the actual machine-readable title is the alt attribute of the image) and is linked with the main URL for the show. There's also a links to older and news episodes of the show.

   
    <a href="..."><img src="the-name-of-the-show-png" alt="The Name of the Show" /></a>

    <a href="episode123-clip.mpeg"><img src="episode123-preview.png" /></a>
    <a href="episode122-clip.mpeg"><img src="episode122-preview.png" /></a>
    <a href="episode121-clip.mpeg"><img src="episode121-preview.png" /></a>
    <a href="episode120-clip.mpeg"><img src="episode120-preview.png" /></a>

    <a href="newer-shows">[next]</a>
    <a href="older-shows">[previous]</a>
    

Show Strawman #9

A series of preview images. The title of the show is given in text. A link to all the show's files via a torrent. Other metadata is also typically included too, in plain text; some is shown here. Also, sometimes metadata is put in the description.

   
    The Name of the Show

    <img src="preview1.png" />
    <img src="preview2.png" />

    <a href="show.torrent">...</a>

    Torrent Info Hash: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

    Description: blah blah blah .... here some URLs to previews...
                 blah blah blah .... this is 24 fps
                 blah blah blah

    Content: tag1, tag2, tag3

    Resolution: 320px x 240px

    Run Time: 24mins 57secs

    Size: 125.42 MB (131,509,108 bytes)

    Type: Movie Clip

    Published/Uploaded By: someone

    <a href="...">Link to page with list of files in torrent</a>
    <a href="...">Link to page with list of people on the torrent</a>
    

Show Strawman #10

An embedded object citing a specific plugin, or simply a link to the media itself, as an object parameter. The rest of the parameters include plugin OR media-specific metadata, and any URIs used are usually unique to the media content.

   
    <object 
    width='400' height='326' type='application/x-shockwave-flash'
    data='http://video.google.com/googleplayer.swf?docId=3807826142091223684&hl=en'>
    <param name='allowScriptAccess' value='never' />
    <param name='movie'
    value='http://video.google.com/googleplayer.swf?docId=3807826142091223684&hl=en'/>
    <param name='quality' value='best'/>
    <param name='bgcolor' value='#ffffff' />
    <param name='scale' value='noScale' />
    <param name='wmode' value='window'/>
    </object>
    

or:

   
    <embed 
    src="http://media.revver.com/qt;sharer=14854/137027.mov"
    pluginspage="http://www.apple.com/quicktime/download/"
    scale="tofit" kioskmode="false" 
    qtsrc="http://media.revver.com/qt;sharer=14854/137027.mov"
    cache="false" controller="true" type="video/quicktime" autoplay="true" height="272" width="320">
    

Current / Previous Standards

Still image

• EXIF
• RDF for self-describing images
• An Indexing and Querying System for Online Images Based on the PNG Format and Embedded Metadata
• Describing and retrieving photos using RDF and HTTP
• Pheed RSS extension for photo syndication

Audio Metadata Formats

• ID3v2

• iTunes
  • Album : String
  • Artist : String
  • Beats Per Minute : Number
  • Bit Rate : Number
  • Comment : Blob
  • Composer : String
  • Genre : String or Foreign ID
  • Disc Number : Number
  • Kind : Proprietary - could be implimented as MIME type
  • My Rating : Number < 5 (actually an integer from the range 0-100. 1-5 "stars" implemented by incrementing by 20)
  • Sample Rate : Number
  • Size : Number
  • Time : Number
  • Track Number : Number
  • Year : Number
  • Additional metadata used internally by the application : Date Added (Timestamp), Date Modified (Timestamp), Equalizer (foreign ID), Play Count (Number), Grouping (Internal)
  • Noteable absences : Tags, License, Copyright year. Also note common complaints about ID3 and classical music.

• XSPF (see also A survey of playlist formats and The XSPF Playlist Format, version 0)
  • album
  • creator
  • duration
  • info
  • title
  • trackNum

Podcasts

• iTunes podcast: http://phobos.apple.com/static/podcast_specifications.pdf

(HTML version of spec: local archive, updated version from Apple Chapter Tool Beta docs, online copy at Apple)

iTunes release 4.9 was widely heralded as an advance for mainstream acceptance of podcasting. However, while some advocates resist even the fragment of a trademarked name in the label for the phenomenon, the use of "itunes:" in the namespace is one of many design decisions in Apple's original specification that became focal points of debate. While Apple has a tradition of working on breakthrough features very quietly, they have been open to public input after its initial release. Many other bloggers have chronicled some of the feedback, as well as some face-to-face discussions.

A mailing list may be coming soon...

• Media RSS: http://search.yahoo.com/mrss

Media RSS has been in the works longer, but doesn't have the overnight-adoption advantage of iTunes. However, it has a much broader ambit, including video; and a much broader community of interest.

A Podcasting Microformat?

The best starting point for understanding these two proposals is W3C's summary table, by Karl Dubost.

There are (at least) two paths to consider when 'porting' these proposals into Microformats:

1. Standalone. What is the most straightforward rendering of each proposal into XHTML? This keeps the interests of the developers of consuming applications foremost: how can the migration be made as painless as possible?
2. Refactored. Given the existing core of Microformats, what is the minimal (!) necessary to add on to RelLicense, RelTag, hReview, and so forth? This favors (we'd hope) content publishing applications: how can the marginal migration costs of adding media be made as painless as possible.

Mapping the existing specs requires understanding the data definitions set forth in each specification, and finding the optimal equivalents (where available). The following is a partial summary of one possible mapping:

Comparison Table

Aspect	Atom	iTunes RSS	Media RSS	M3U	RSS 2.0	Microformat?
Authorship	<atom:author> <atom:contributor>	<itunes:author> <itunes:owner> <itunes:name> <itunes:email>	<media:credit>	nothing	<copyright> <author> <source>	<cite> hCard rel-author rev-author <address> need a way of specifying the role of the author or contributor. proposal for role By using class atribute, this allows diferent formating of authors and contributors, also is usable for <cite> and <adress> <div class="author vcard"> ...hCard code here </div> -- tonydm 21 Jan 2007
Categorization	<atom:category>	<itunes:category> <itunes:keywords>	<media:category>	nothing	<category>	rel-tag - requires a way to do sub categories?
Description	<atom:summary> <atom:content>	<itunes:subtitle> <itunes:summary>	<media:text>	nothing	<description>	XHTML (e.g. <blockquote> )
Licensing	<atom:copyright> <atom:rights>	<itunes:block> <copyright> <link>	Not explicitly mentioned	nothing	<ttl>	rel-license
Metadata		<itunes:duration> <itunes:image>	<media:thumbnail> <media:hash> <media:content url="http://www.example.com/movie.mov" fileSize="12216320" type="video/quicktime" isDefault="true" expression="full" bitrate="128" framerate="25" duration="185" height="200" width="300" />	nothing		needs a new microformat e.g., proposal for duration <abbr class="duration" title="00:27:35"> 27 minutes, 35 seconds </abbr> -- jkinberg 11:51, 19 Sept 2005 (PDT))
Preview	nothing	nothing	For Video Media: <media:thumbnail> For Audio Media: nothing	nothing	nothing	need a new microformat for this proposal for video preview <img class="preview" src="..." /> -- Charles
Rating	nothing	<itunes:explicit>	<media:adult> <media:rating>	nothing	<rating>	rel-tag
Title	<atom:title> <atom:subtitle>	<itunes:subtitle> <title>	<media:title> <title>	nothing	<title>	<h1>...</h1> <h2>...<h2> <h3>...</h3> <h4>...</h4> <h5>...</h5> <h6>...</h6> title="..." Microformats with class="title"
Show Concept	<atom:feed> <link rel="self" href="..." />	<channel> <link>	<channel> <link>	A M3U file.	<channel> <link>	need a new microformat
Episode Concept	<atom:entry> <link rel="alternate" href="..." />	<item> <link>	<item> <link>	A single URL line.	<item> <link>	need a new microformat
Media Concept	<atom:link rel="enclosure" href="..." /> also... dependent on type of data embedded in or referenced from <atom:content>	<enclosure>	<media:content>	Simply the inclusion of a URL on a line.	<enclosure>	<img> <a href=""> <object> <embed>

Video

• EXIF - (Yes, the spec deals with video taken by still cameras. -- RyanKing)
• MPEG-7 - MPEG-7, formally named "Multimedia Content Description Interface", is a standard for describing the multimedia content data that supports some degree of interpretation of the information meaning, which can be passed onto, or accessed by, a device or a computer code. (Very powerful, but you have to pay for documentation -- ChristopherA 01:53, 29 Jun 2005 (PDT))
  • An OWL ontology of MPEG-7 is available at http://rhizomik.net/ontologies/2005/03/Mpeg7-2001.owl -tmw
• SMIL - Synchronized Multimedia Integration Language (SMIL, pronounced "smile") enables simple authoring of interactive audiovisual presentations. (Not exactly video, but has a lot of useful video-related features in it. -- ChristopherA 01:55, 29 Jun 2005 (PDT))
• Video Metadata Model - a starting point

Interactive

• SVG
• SMIL
• QTVR?

Offline Media

• XSPF (media may be either offline or online)

Printed Publications (books/magazines)

• Amazon

Movies

• IMDB
• Amazon

Music

• FreeDB
• MusicBrainz
• Amazon

Container Formats

The purpose of container formats is to allow the muxing of multiple codecs into a single file or stream. For example, if you want audio, video, and text (subtitles) combined into a single stream.

The purpose of the muxing is for synchronization and so that you can stream the file and view it without having to jump around -- seek -- in the file.

Container formats include:

• AVI
• MPEG
• Ogg
• QuickTime

Container Format Comparison Table

Aspect	AVI	MPEG	Ogg	QuickTime	Microformat?
Allows Audio	Yes	Yes	Yes	Yes
Allows Video	Yes	Yes	Yes	Yes
Allows Text			Yes	Yes
Error Detection			Yes

Notes for Microformats

One of the problems with container formats is that it is often difficult and sometimes impossible to know the disposition of what they contain ahead of time (without reading the file).

For example, Ogg files use the MIME type application/ogg and generally use the extension .ogg. With only these 2 pieces of information, you cannot tell whether such an Ogg file contains video data or not.

There is room for a microformat to help with this. Perhaps something like:

<a class="moving-image" type="application/ogg" href="my.ogg">...</a>

Although, this can be done in other ways too. For example, using disposition hinting one could do something like:

<a type="application/ogg;disposition-type:moving-image" href="my.ogg">...</a>

Anothing thing for any Microformat to consider is that it may be the case that the data files that are muxed in the container format may also be offered in non-muxed forms. This is important when alternatives are offered.

Audio Formats

Audio formats include:

• AAC
• MP3
• PCM WAV
• Speex
• Vorbis
• WMA

Audio Format Comparison Table

Aspect	AAC	MP3	PCM WAV	Speex	Vorbis	WMA	Microformat?
Lossless	No	No	Yes	Yes	No
Lossy	Yes	Yes	No	No	Yes	Yes
Streamable	Yes	Yes		Yes	Yes	Yes

Notes for Microformats

• Knowing whether audio data is streamable or not can affect pre-fetching. And can be a metric used when alternatives of the same file is given.
• Having audio data marked as lossy or lossless can be useful when alternatives of the same file is made.

Playing Formats

Playing formats include:

• ASX
• Atom
• Atom+SMIL
• Atomic RSS
• Atomic RSS+SMIL
• Extended M3U
• HTML+Time
• iTunes RSS
• KPL
• M3U
• MAGMA
• Media RSS
• PLS
• RAM
• RSS 2.0
• SMIL
• WAX
• WVX

Playing Format Comparison Table

Aspect	ASX	Atom	Atom+SMIL	Atomic RSS	Atomic RSS+SMIL	Extended M3U	HTML+Time	iTunes RSS	Media RSS	M3U	RSS 2.0	Microformat?
Alternatives Playing		nothing			<smil:switch>			nothing	<media:group>	nothing	nothing	need a new microformat
Parallel Playing												need a new microformat
Sequence Playing		if you consider the Atom file a show, then nothing. if you consider the Atom file an episode, then the list of <atom:entry>'s provides a sequence.			<smil:seq>			if you consider the RSS file a show, then nothing. if you consider the RSS file an episode, then the list of <item>'s provides a sequence.	if you consider the RSS file a show, then nothing. if you consider the RSS file an episode, then the list of <item>'s provides a sequence.	if you consider the M3U file a show, then nothing. if you consider the M3U file an episode, then the URL lines provide a sequence.	if you consider the RSS file a show, then nothing. if you consider the RSS file an episode, then the list of <item>'s provides a sequence.	need a new microformat
Pre-Fetching		<link rel="enclosure" href="..." />	<link rel="enclosure" href="..." /> <smil:prefetch>	<link rel="enclosure" href="..." />				<enclosure>	<media:content>	The M3U format is just list of URL's, with each URL on its own line. Pre-Fetching would be accomplished by just added a URL to the list.	<enclosure>	rel-enclosure

Raster Graphic Formats

Raster graphic formats include:

• GIF
• PNG
• JPEG

Vector Graphic Formats

Vector graphic formats include:

• SVG

Formats & Protocols

AAC

AAC (Advanced Audio Coding) is a lossy audio format. Like MP3, AAC is part of the MPEG standard. Since April 2003, AAC has become an integral part of Apple's iPod and iTunes products. This association between AAC and Apple has become so strong among some that AAC is also sometimes call Apple Audio Codec.

References:

• http://en.wikipedia.org/wiki/Advanced_Audio_Coding

ASF

ASF is an Audio/Video format. ASF files that contain only audio data are typically called WMA files.

References:

• http://en.wikipedia.org/wiki/Advanced_Streaming_Format

Atom

Atom is a XML-based syndication format that was developed by certain members of the RSS community.

References:

• http://atomenabled.org/developers/syndication/
• RFC 3339

Atom+SMIL

Atom+SMIL is Atom with SMIL used in the <atom:content>.

(This was originally used by Charles Iliya Krempeaux for "Internet TV". Also see Atomic RSS+SMIL)

Atomic RSS

Atomic RSS is a syndication format that is RSS with Atom used as an extension module.

References:

• http://www.tbray.org/ongoing/When/200x/2005/07/27/Atomic-RSS

Atomic RSS+SMIL

Atomic RSS+SMIL is Atomic RSS with SMIL use in the <atom:content>.

(This was originally used by Charles Iliya Krempeaux for "Internet TV".)

AVI

AVI is a container format for Audio/Video formats.

References:

• http://en.wikipedia.org/wiki/AVI

B4S

B4S is a playlist format.

References:

• http://gonze.com/playlists/playlist-format-survey.html#B4S

BitTorrent

BitTorrent is a P2P protocol.

References:

• http://www.bittorrent.com/protocol.html
• http://en.wikipedia.org/wiki/Bittorrent
• http://wiki.theory.org/BitTorrentFAQ

Creative Commons RDF

References:

• http://gonze.com/playlists/playlist-format-survey.html#Creative

DAAP

References:

• http://gonze.com/playlists/playlist-format-survey.html#DAAP

Extended M3U

References:

• http://gonze.com/playlists/playlist-format-survey.html#M3U
• http://hanna.pyxidis.org/tech/m3u.html
• http://en.wikipedia.org/wiki/M3U

FLAC

Free Lossless Audio Codec (FLAC)

• http://flac.sourceforge.net/

GIF

GIF is a raster graphics format and a movie format.

HTML+Time

References:

• http://www.w3.org/TR/NOTE-HTMLplusTIME
• http://gonze.com/playlists/playlist-format-survey.html#HTML

HTTP

References:

• http://www.w3.org/Protocols/

iTunes Library

References:

• http://gonze.com/playlists/playlist-format-survey.html#iTunes

iTunes RSS

References:

• http://www.apple.com/itunes/podcasts/techspecs.html

JPEG

JPEG is a raster graphics format.

Kapsule

References:

• http://gonze.com/playlists/playlist-format-survey.html#Kapsule

KPL

KPL is a playlist format.

References:

• http://gonze.com/playlists/playlist-format-survey.html#KPL

M3U

M3U is a playlist format.

References:

• http://gonze.com/playlists/playlist-format-survey.html#M3U

MAGMA

References:

• http://gonze.com/playlists/playlist-format-survey.html#MAGMA

Media RSS

Media RSS is a syndication format that is RSS with the addition of the Media RSS Module.

MKV/MKA

Matroska MKV/MKA are video and audio containers, respectively.

• http://www.matroska.org/

MNG

MNG is a Video format.

MP3

MP3 is an Audio format.

MPEG

ASF is an Audio/Video format.

MusicBrainz

References:

• MusicBrainz.org
• MusicBrainz wiki
• http://gonze.com/playlists/playlist-format-survey.html#musicbrainz

Ogg

Ogg is a media container format.

References:

• http://xiph.org/ogg/
• http://en.wikipedia.org/wiki/Ogg
• RFC 3533 - The Ogg Encapsulation Format Version 0
• RFC 3534 - The application/ogg Media Type

PLS

PLS is a playlist format.

References:

• http://gonze.com/playlists/playlist-format-survey.html#PLS
• http://en.wikipedia.org/wiki/PLS_(file_format)

RAM

RAM is a playlist format.

References:

• http://gonze.com/playlists/playlist-format-survey.html#RAM

RSS 2.0

RSS 2.0 is the defacto standard syndication format.

References:

• http://developer.mozilla.org/en/docs/RSS

RTSP

References:

• http://www.rtsp.org/
• RFC 1889
• http://en.wikipedia.org/wiki/Real_Time_Streaming_Protocol

SMIL

References:

• http://www.w3.org/AudioVideo/

Speex

References:

• http://www.speex.org/

SVG

SVG is a vector graphics format.

Theora

Theora is an Audio/Video format/

References:

• http://www.theora.org/
• http://en.wikipedia.org/wiki/Theora

Vorbis

Vorbis is an Audio format.

References:

• http://xiph.org/vorbis/
• http://vorbis.com/
• http://en.wikipedia.org/wiki/Vorbis

WAX

WAX is an SGML-based playlist for that looks exactly like ASX and WVX, except that it can only reference to ASF or WMA, but NOT to WMV files.

References:

• http://gonze.com/playlists/playlist-format-survey.html#WAX

WMA

WMA is an Audio format.

References:

• http://en.wikipedia.org/wiki/Windows_Media_Audio

WMV

WMV is an Audio/Video format.

References:

• http://en.wikipedia.org/wiki/Windows_Media_Video

WVX

WVX is an SGML-based playlist format that looks exactly like ASX and WAX, except that it can reference ASF, WMA, and WMV media files.

References:

• http://gonze.com/playlists/playlist-format-survey.html#WVX

Comments

• See separate media-metadata-issues document.
• See separate media-info-examples document.

See Also

• video-metadata-models