media-info-brainstorming

Brainstorming for Media Microformat

This is a brainstorm for media microformat. Examples of media-info can be found here media-info-examples

Contributors

• Mary Hodder
• Lisa Rein
• Chris Newell
• Manu Sporny, Bitmunk - Digital Bazaar, Inc.
• Martin McEvoy

The Problem

There are many ways to publish media by users, but as people try to access those remixing and aggregation become more and more prevalent, having consistent records becomes important. Audio, photos and video show up in each other's publishing spaces, even though they are unique media objects. A photo might be next to the link for an audio piece, as it's visual artwork. A video can be comprised of quotes of other videos, photos and audio. Still photos can be made from videos. All three types of objects can contain subsets of media that is tagged and described. Having a single publishing format to relate them as needed in an organized structure makes sense.

Elements that come up often in practice

Here are some examples of elements that might be included because they seem to come up often in user generated media publishing, either on their own blogs or on service sites, include the following:

Base elements:

• Title
• Html URL
• Media URL
• Image URL (video and audio)
• Tags
• Description / summary
• Quotes (subsets of the object: a video quote and tags/description associated with it, a region annotation note for a photo, or the quote of a podcast or audio piece, with tags/description -- the technical detail for these subsets exists in the 'more info' section below)
• Creator
• License (defaults to copyright, if none exists, but it's there, by US law, and many other areas of the world)
• XML Feed URL
• Contributor

and for audio and visual:

• Episode
• Duration
• Interval

Other info: (This is not the same for all types of media, and is published by users in very limited ways in practice, or is captured from the device or service or in some way, invisible to the user, and therefore often depends on a service to pick it up. It should only appear in a publishing tool under 'more data' at then end, for enthusiast users.)

JPG	Video	Audio
Device	Device	Device
Ratio	Aspect Ratio	?
file size	file size	file size
.	audio codec	audio codec
.	video codec	.
.	bit / frame rate	bit rate
Portrait or Landscape	.	.
Region Annotation (subphotos: calculation of location, size)	Quotes of Video (subvideo: in and out points)	Quotes of Audio (subaudio: in and out points)
iPod compliant?	iPod compliant?	iPod compliant?
Time	Time	Date
Date	Date	Date
Inclusion in playlist?	Inclusion in playlist?	Inclusion in playlist?

Combining with other microformats

Although media items are sometimes presented in isolation they are often found in data structures that are supported by existing microformats. It should therefore be possible to use the media microformat as a child (or in some cases a parent) of these microformats.

hAtom

Podcast feeds reference a sequence of media items. A media microformat could therefore be used as a child element of hAtom.

hCalendar

A media microformat may used to describe a continuous media stream e.g. a TV channel. An event or a schedule of events on the channel can be signaled using hEvents as child elements. This combination would provide what is necessary to capture a specific event; the media stream information and the timing information.

Schema

Analysis of all Media Info examples

Common discovered properties occuring 70% or more of the time across all media info examples completed 2008-11-07

Photo	Video	Podcasting	Speech	Music
Title 93.33%	Title 100%	Title 100%	Title 100%	Title 92%
Creator 96.66%		Creator 95.65%	Creator 100%	Creator 100%
Description 73.33%		Description 95.65%	Description 100%
Photo 100%	Video 100%	Download 95.65%	Download 95.65%
	Published 91.42%	Published 95.65%	Published 83.33%	Published 77%
Comments 70%	Comments 88.57%
		Feed 100%	Feed 100%
	Tag 71.42%
				Tracks 85%

hMedia Schema

Proposed hmedia schema and description defined by common elements discovered during the analysis of all Media Info examples.

hmedia	Definition
fn	The name of a media. Reused from hcard
contributor	A Contributor is any person or organization that takes part in the creation or distribution of the Media. Reused from haudio
description	A full-text description of the Media. Reused from hreview
photo	An embedded Image or Photograph of the Media. Reused from hcard
video	The contents are an embedded video or movie of the Media. new!
enclosure	Indicates that the referred url is a download of the Media. Using rel-enclosure.
published	Represents the date that the Media was made available to the public. Reused from hatom
feedback	A user/visitor submitted comment on the media, see: Media Commentsnew!
alternate	Designates an alternate version for the document or Media typically an RSS, Atom or XML feed using the rel-design-pattern new!
category	An author-designated "category" or a keyword/subject of the Media. Reused from hcalendar, additionaly keywords may be defined using rel-tag.
item	A container element for another Media item. Reused from haudio

Media Comments

User submitted comments are highly prevalent across all media domains the following table is a definition of the discovered elements of a comment. A separate analysis was made on 25 model examples, the results are available at Media comments examples

Property	Definition
feedback	A user or visitor submitted comment contents must contain a valid hcard
hCard properties	Definition
fn	The name of the person who authored a comment
url	A url of the person who authored a comment
note	To specify a comment that is associated with the person who authored a comment
rev	Datetime of the revision of the entire coment
uid	A unique uri identifier of a comment, typically a permalink to a comment

Media Comment examples

Full media Comment example:

<div class="feedback vcard" id="comment1">
   <p><a class="url fn" href="http://somewebsite.com/">Joe Blogs<a> 
      said on <span class="rev">2008-11-08</span></p>
   <p class="note">Great video, keep up the good work</p>
   <p><a class="uid" href="http://someblog.com/article/foo.html#comment1">permalink to this comment</a></p>
</div>

Real world example found at http://www.itsjerrytime.com/?p=89:

<li class="alt" id="comment-4100">
     <cite><a href="http://www.xk9.com/woof/" rel="external nofollow">XK9</a></cite> Says:
	<br>
	<small class="commentmetadata"><a href="#comment-4100" title="">October 23rd, 2006 at 3:29 pm</a> </small>
	<p>Nah! Forget quality. It’s all about the volume, baby.</p>
        <p>Crank ‘em up and crank ‘em out.</p>
</li>

Media comment transformation:

<li class="alt feedback vcard" id="comment-4100"><!-- changed -->
	<cite><a class="url fn" href="http://www.xk9.com/woof/" rel="external nofollow">XK9</a></cite> Says:<!--changed -->
	<br/>
	<small class="commentmetadata"><abbr class="rev" title="2008-10-23T15:29:00+00:00">
		<a class="uid" href="http://www.itsjerrytime.com/?p=89#comment-4100">October 23rd, 2006 at 3:29 pm</a>
	</abbr></small><!-- changed -->
  <blockquote class="note" cite="http://www.itsjerrytime.com/?p=89#comment-4100"><!-- changed -->
	<p>Nah! Forget quality. It's all about the volume, baby.</p>
	<p>Crank 'em up and crank 'em out.</p>
  </blockquote>
</li>

Comments

Differentiating File Content from File Format

We should consider splitting file content from file format. A song (content) can be represented as an MP3 audio file (format), or an MPEG-2 music video (format). media-info should be used to describe file content with a separate microformat or link-rel design pattern to describe file format.

Example 1:

• media-info (content): A song called "Hung Up" by "Madonna"
• file-format (format): 4.5MB MP3 file encoded at 192Kbps bitrate.

Example 2:

• media-info (content): A music video called "Hung Up" by "Madonna"
• file-format (format): 35MB AVI file, MP3 audio codec encoded at 192Kbps, DiVX video codec encoded at 600Kbps

ManuSporny 06:50, 5 Apr 2007 (PDT)

Microformats and Democracy Player/Miro

Democracy (soon to be Miro) is a desktop program attempting to create a more television like experience for internet video. Microformats present several unique challenges to our project. This is my (Nick Nassar's) attempt to summarize them. My opinions don't reflect the opinion of my employer (PCF). My email address at pculture.org is nassar.

Re-reading this, I realize it sounds very negative. Please, tell me why I'm totally wrong. I hope I'm totally wrong. :)

Loose specifications are difficult to parse and create

This is a problem we're seeing with RSS that could potentially multiply with the popularity of microformats.

The RSS "standard" is very loose, and our experience is that most feeds don't even conform to that standard. Because of this, we lose the main benefits of XML: easy parsing and creation. On the parsing end, we're forced to use complex third party libraries such as Feed Parser rather than just being able to parse with a simple XML parser. This is the same problem we saw with HTML in the 90s where the "standard" involves doing the right thing in a lot of non-standard cases.

Content creators need to use complex validators like Feed Validator rather than just being able to write valid XML so they make sure they don't do things that confuse reg-exp based parsers. It's much more difficult to create a program you trust to conform to a loose spec than one you trust to conform to a more rigid spec.

For search engines and other applications where exact data isn't important, this isn't a big deal. For an application that's expected to show exactly the videos the author published, it's a lot of work to get the parsing right. Even in "valid" RSS we see people faking the length and type attributes for enclosures all the time, forcing programs like ours not to trust that information.

Microformats could easily make those problems explode. In general, microformats are more dangerous because they're even looser, and they're embedded in the XHTML, so the data is tied to the presentation. In practice, you need extra divs and spans to get XHTML formatting right. Inserting that extra div tag to get the formatting right will change the data. The formats must be well defined enough to deal with this.

When I tell this to people, they say "Nick, you're crazy. It's easy to generate valid XML." To prove my point, there are two real world examples of this on your own site. The hCalendar creator and the hCard creator don't deal with quotes and angle brackets in the input, resulting in invalid XHTML when those characters are included. I don't bring this up to pick on this site, just to point out that this sort of mistake is the norm in supposedly easy to create XML formats.

One way to mostly solve this is to write a parser, a validator, and a generator for each format when you define it. This way there's a de facto correct implementation that can act like Netscape did for HTML or the perl implementation does for perl. It would also force a real world implementation of the "standard" before it gets adopted. If it's difficult to parse correctly, you can go back and redefine it to be more easily parsable.

Strict formats are easier for everyone. As long as you insist on keeping it loose, writing parsers, validators, and generators for each format are a step forward.

HTML data

A complex problem for authors of feed readers is how to deal with HTML encoded areas such as the description. HTML can contain JavaScript, plugins, and other "dangerous" content that you don't necessarily want in your program. Feed Parser only allows white listed tags and attributes. This is generally accepted as the right thing to do, but causes problems in programs like Democracy player when content authors expect certain tags to appear but they get blocked. This might be tricky when combined with the problem of allowing extra divs and spans without altering the data itself.

A microformat spec should include details on what fields content creators can expect to show up in an application such as Democracy Player

Character Sets

A very common problem with RSS feeds is incorrect character sets. The XML header might specify utf-8, but the document is actually latin-1, which causes it not to parse. Requiring or strongly recommending that all microformat documents are UTF-8 might improve the situation.

Mime types are useful

Another problem we're seeing with RSS that could get worse with microformats is the lack of a mime type. There is no standard RSS mimetype and extension (although application/xml+rss and .rss are finally catching on), so an application doesn't know what to do with an RSS file saved to disk. We have to assume that text/xml links sent into Democracy are RSS. Feed readers can't just associate with the mime type like most programs, they need to hook in in different ways.

Combining data into XHTML means there's no way to determine what programs can handle that page without downloading it first. If an XHTML page with media information is saved to disk, we have no way of determining it has data that can be used in Democracy player without first opening it up.

This is a fundamental problem with microformats. You will never be able to load microformat documents into the correct program without parsing them.

Media specific concerns

It should always be possible to use media microformats as a feed. It would be a very poor experience if, for example, media items were duplicated in programs like Democracy whenever their HTML changed. Making them an extension of the hAtom format should cover this. Recommend that real world implementations do correct HTTP caching so programs like Democracy can avoid downloading the entire file every time.

The format shouldn't require any fields other than URL and title. As is, most sites fake the length and type attributes for RSS enclosures. Don't make people fake data they don't have.

Tag each piece of data. It's difficult for us to parse strings like "35MB AVI file, MP3 audio codec encoded at 192Kbps, DiVX video codec encoded at 600Kbp" into anything useful, but if those fields are separated out with span tags, we can actually use that data.

Make media microformats feeds. Don't require any fields. Tag each piece of data.