View source for media-info-brainstorming

= Brainstorming for Media Microformat =

This is a brainstorm for media microformat.  Examples of media-info can be found here [http://microformats.org/wiki/media-info-examples media-info-examples]

== Contributors ==

* [http://napsterization.org/stories/ Mary Hodder]
* [http://onlisareinsradar.com/ Lisa Rein]
* [http://microformats.org/wiki/User:ChrisNewell Chris Newell]
* [[User:ManuSporny | Manu Sporny]], [http://www.bitmunk.com/ Bitmunk] - [http://blog.digitalbazaar.com Digital Bazaar], Inc.
* [[User:WebOrganics|Martin McEvoy]]

== The Problem ==

There are many ways to publish media by users, but as people try to access those remixing and aggregation become more and more prevalent, having consistent records becomes important.  Audio, photos and video show up in each other's publishing spaces, even though they are unique media objects. A photo might be next to the link for an audio piece, as it's visual artwork.  A video can be comprised of quotes of other videos, photos and audio.  Still photos can be made from videos.  All three types of objects can contain subsets of media that is tagged and described.  Having a single publishing format to relate them as needed in an organized structure makes sense.

== Elements that come up often in practice ==
Here are some examples of elements that might be included because they seem to come up often in user generated media publishing, either on their own blogs or on service sites, include the following:

<strong>Base elements:</strong>
* Title
* Html URL
* Media URL
* Image URL (video and audio)
* Tags
* Description / summary
* Quotes (subsets of the object:  a video quote and tags/description associated with it, a region annotation note for a photo, or the quote of a podcast or audio piece, with tags/description -- the technical detail for these subsets exists in the 'more info' section below)
* Creator
* License (defaults to copyright, if none exists, but it's there, by US law, and many other areas of the world)
* XML Feed URL
* Contributor

and for audio and visual:
* Episode
* Duration
* Interval

<strong>Other info</strong>:
(This is not the same for all types of media, and is published by users in very limited ways in practice, or is captured from the device or service or in some way, invisible to the user, and therefore often depends on a service to pick it up.  It should only appear in a publishing tool under 'more data' at then end, for enthusiast users.)

<table border="1" cellspacing="2" cellpadding="2">

<tr><th><strong>JPG</strong></th><th><strong>Video</strong></th><th><strong>Audio</strong></th></tr>

<tr><th>Device</th><th>Device</th><th>Device</th></tr>

<tr><th>Ratio</th><th>Aspect Ratio</th><th>?</th></tr>

<tr><th>file size</th><th>file size</th><th>file size</th></tr>

<tr><th>.</th><th>audio codec</th><th>audio codec</th></tr>

<tr><th>.</th><th>video codec</th><th>.</th></tr>

<tr><th>.</th><th>bit / frame rate</th><th>bit rate</th></tr>

<tr><th>Portrait or Landscape</th><th>.</th><th>.</th></tr>

<tr><th>Region Annotation (subphotos: calculation of location, size)</th><th>Quotes of Video (subvideo: in and out points)</th><th>Quotes of Audio (subaudio: in and out points)</th></tr>

<tr><th>iPod compliant?</th><th>iPod compliant?</th><th>iPod compliant?</th></tr>

<tr><th>Time</th><th>Time</th><th>Date</th></tr>

<tr><th>Date</th><th>Date</th><th>Date</th></tr>

<tr><th>Inclusion in playlist?</th><th>Inclusion in playlist?</th><th>Inclusion in playlist?</th></tr>

</table>
<br>

== Combining with other microformats ==

Although media items are sometimes presented in isolation they are often found in data structures that are supported by existing microformats. It should therefore be possible to use the media microformat as a child (or in some cases a parent) of these microformats.

=== [http://microformats.org/wiki/hatom hAtom] ===
Podcast feeds reference a sequence of media items. A media microformat could therefore be used as a child element of hAtom.

=== [http://microformats.org/wiki/hcalendar hCalendar] ===
A media microformat may used to describe a continuous media stream e.g. a TV channel. An event or a schedule of events on the channel can be signaled using hEvents as child elements. This combination would provide what is necessary to capture a specific event; the media stream information and the timing information.

== Schema ==

=== Analysis of all Media Info examples ===

Common discovered properties occuring 70% or more of the time across all [[media-info-examples|media info examples]] completed 2008-11-07

<table border="1">
<tr>
	<th>[[media-info-examples#Analysis_of_Photo_Sites|Photo]]</th>
	<th>[[media-info-examples#Analysis_of_Video_Sites|Video]]</th>
	<th>[[media-info-examples#Analysis_of_Podcasting_Sites|Podcasting]]</th>
	<th>[[media-info-examples#Analysis_of_Individual_Publishing_of_Speech_Sites|Speech]]</th>
	<th>[[media-info-examples#Analysis_of_Music_Sites|Music]]</th>
</tr>
<tr>
	<td>Title 93.33%</td>
	<td>Title 100%</td>
	<td>Title 100%</td>
	<td>Title 100%</td>
	<td>Title 92%</td>
</tr>
<tr>
	<td>Creator 96.66%</td>
	<td></td>
	<td>Creator 95.65%</td>
	<td>Creator 100%</td>
	<td>Creator 100%</td>
</tr>
<tr>
	<td>Description 73.33%</td>
	<td></td>
	<td>Description  95.65%</td>
	<td>Description  100%</td>
	<td></td>
</tr>
<tr>
	<td>Photo 100%</td>
	<td>Video 100%</td>
	<td>Download 95.65%</td>
	<td>Download 95.65%</td>
	<td></td>
</tr>
<tr>
	<td></td>
	<td>Published 91.42%</td>
	<td>Published 95.65%</td>
	<td>Published 83.33%</td>
	<td>Published 77%</td>
</tr>
<tr>
	<td>Comments 70%</td>
	<td>Comments 88.57%</td>
	<td></td>
	<td></td>
	<td></td>
</tr>
<tr>
	<td></td>
	<td></td>
	<td>Feed 100%</td>
	<td>Feed 100%</td>
	<td></td>
</tr>
<tr>
	<td></td>
	<td>Tag 71.42%</td>
	<td></td>
	<td></td>
	<td></td>
</tr>
<tr>
	<td></td>
	<td></td>
	<td></td>
	<td></td>
	<td>Tracks 85%</td>
</tr>
</table>

=== hMedia Schema ===

Proposed hmedia schema and description defined by  common elements discovered during the analysis of all [[media-info-brainstorming#Analysis_of_all_Media_Info_examples|Media Info examples]].

<table border="1">
<tr>
	<th><code>hmedia</code></th>
	<th>Definition</th>
</tr>
<tr>
	<td><code>fn</code></td>
	<td>The name of a media. Reused from [[hcard|hcard]]</td>
</tr>
<tr>
	<td><code>contributor</code></td>
	<td>A Contributor is any person or organization that takes part in the creation or distribution of the Media. Reused from [[haudio|haudio]]</td>
</tr>
<tr>
	<td><code>description</code></td>
	<td>A full-text description of the Media. Reused from [[hreview|hreview]]</td>
</tr>
<tr>
	<td><code>photo</code></td>
	<td>An embedded Image or Photograph of the Media. Reused from [[hcard|hcard]]</td>
</tr>
<tr>
	<td><code>video</code></td>
	<td>The contents are an embedded video or movie of the Media. {{NewMarker}}</td>
</tr>
<tr>
	<td><code>enclosure</code></td>
	<td>Indicates that the referred url is a download of the Media. Using [[rel-enclosure|rel-enclosure]].</td>
</tr>
<tr>
	<td><code>published</code></td>
	<td>Represents the date that the Media was made available to the public. Reused from [[hatom|hatom]]</td>
</tr>
<tr>
	<td><code>comment</code></td>
	<td>A user/visitor submitted comment on the media. {{NewMarker}}</td>
</tr>
<tr>
	<td><code>alternate</code></td>
	<td>Designates an alternate version for the document or Media  typically an RSS, Atom or XML feed. {{NewMarker}}</td>
</tr>
<tr>
	<td><code>category</code></td>
	<td>An author-designated "category" or a keyword/subject of the Media. Reused from [[hcalendar|hcalendar]], additionaly keywords may be defined using [[rel-tag|rel-tag]].</td>
</tr>
<tr>
	<td><code>item</code></td>
	<td>A container element for another Media item. Reused from [[haudio|haudio]]</td>
</tr>
</table>

== Comments ==

=== Differentiating File Content from File Format ===

We should consider splitting file content from file format. A song (content) can be represented as an MP3 audio file (format), or an MPEG-2 music video (format). media-info should be used to describe file content with a separate microformat or link-rel design pattern to describe file format. 

Example 1:

* media-info (content): A song called "Hung Up" by "Madonna"
* file-format (format): 4.5MB MP3 file encoded at 192Kbps bitrate.

Example 2:

* media-info (content): A music video called "Hung Up" by "Madonna"
* file-format (format): 35MB AVI file, MP3 audio codec encoded at 192Kbps, DiVX video codec encoded at 600Kbps

[[User:ManuSporny|ManuSporny]] 06:50, 5 Apr 2007 (PDT)

=== Microformats and Democracy Player/Miro ===

Democracy (soon to be Miro) is a desktop program attempting to create a more television like experience for internet video.
Microformats present several unique challenges to our project. This is my (Nick Nassar's) attempt to summarize them. My opinions don't reflect the opinion of my employer (PCF). My email address at pculture.org is nassar.

Re-reading this, I realize it sounds very negative. Please, tell me why I'm totally wrong. I hope I'm totally wrong. :)

==== Loose specifications are difficult to parse and create ====

This is a problem we're seeing with RSS that could potentially multiply with the popularity of microformats.

The RSS "standard" is very loose, and our experience is that most feeds don't even conform to that standard. Because of this, we lose the main benefits of XML: easy parsing and creation. On the parsing end, we're forced to use complex third party libraries such as [http://www.feedparser.org/ Feed Parser] rather than just being able to parse with a simple XML parser. This is the same problem we saw with HTML in the 90s where the "standard" involves doing the right thing in a lot of non-standard cases.

Content creators need to use complex validators like [http://www.feedvalidator.org/ Feed Validator] rather than just being able to write valid XML so they make sure they don't do things that confuse reg-exp based parsers. It's much more difficult to create a program you trust to conform to a loose spec than one you trust to conform to a more rigid spec.

For search engines and other applications where exact data isn't important, this isn't a big deal. For an application that's expected to show exactly the videos the author published, it's a lot of work to get the parsing right. Even in "valid" RSS we see people faking the length and type attributes for enclosures all the time, forcing programs like ours not to trust that information.

Microformats could easily make those problems explode. In general, microformats are more dangerous because they're even looser, and they're embedded in the XHTML, so the data is tied to the presentation. In practice, you need extra divs and spans to get XHTML formatting right. Inserting that extra div tag to get the formatting right will change the data. The formats must be well defined enough to deal with this. 

When I tell this to people, they say "Nick, you're crazy. It's easy to generate valid XML." To prove my point, there are two real world examples of this on your own site. The [http://microformats.org/code/hcalendar/creator hCalendar creator] and the [http://microformats.org/code/hcard/creator hCard creator] don't deal with quotes and angle brackets in the input, resulting in invalid XHTML when those characters are included. I don't bring this up to pick on this site, just to point out that this sort of mistake is the norm in supposedly easy to create XML formats.

One way to mostly solve this is to write a parser, a validator, and a generator for each format when you define it. This way there's a de facto correct implementation that can act like Netscape did for HTML or the perl implementation does for perl.  It would also force a real world implementation of the "standard" before it gets adopted. If it's difficult to parse correctly, you can go back and redefine it to be more easily parsable.

''Strict formats are easier for everyone. As long as you insist on keeping it loose, writing parsers, validators, and generators for each format are a step forward.''

===== HTML data =====

A complex problem for authors of feed readers is how to deal with HTML encoded areas such as the description. HTML can contain JavaScript, plugins, and other "dangerous" content that you don't necessarily want in your program. [http://www.feedparser.org/ Feed Parser] only allows white listed tags and attributes. This is generally accepted as the right thing to do, but causes problems in programs like Democracy player when content authors expect certain tags to appear but they get blocked. This might be tricky when combined with the problem of allowing extra divs and spans without altering the data itself.

A microformat spec should include details on what fields content creators can expect to show up in an application such as Democracy Player

===== Character Sets =====

A very common problem with RSS feeds is incorrect character sets. The XML header might specify utf-8, but the document is actually latin-1, which causes it not to parse. Requiring or strongly recommending that all microformat documents are UTF-8 might improve the situation.

==== Mime types are useful ====

Another problem we're seeing with RSS that could get worse with microformats is the lack of a mime type. There is no standard RSS mimetype and extension (although application/xml+rss and .rss are finally catching on), so an application doesn't know what to do with an RSS file saved to disk. We have to assume that text/xml links sent into Democracy are RSS. Feed readers can't just associate with the mime type like most programs, they need to hook in in different ways.

Combining data into XHTML means there's no way to determine what programs can handle that page without downloading it first. If an XHTML page with media information is saved to disk, we have no way of determining it has data that can be used in Democracy player without first opening it up. 

''This is a fundamental problem with microformats. You will never be able to load microformat documents into the correct program without parsing them.''

==== Media specific concerns ====

It should always be possible to use media microformats as a feed. It would be a very poor experience if, for example, media items were duplicated in programs like Democracy whenever their HTML changed. Making them an extension of the hAtom format should cover this. Recommend that real world implementations do correct HTTP caching so programs like Democracy can avoid downloading the entire file every time.

The format shouldn't require any fields other than URL and title. As is, most sites fake the length and type attributes for RSS enclosures. Don't make people fake data they don't have.

Tag each piece of data. It's difficult for us to parse strings like "35MB AVI file, MP3 audio codec encoded at 192Kbps, DiVX video codec encoded at 600Kbp" into anything useful, but if those fields are separated out with span tags, we can actually use that data.

''Make media microformats feeds. Don't require any fields. Tag each piece of data.''