how-to-start-a-new-microformat

The New Microformat HOW-TO

Primary Author/Editors: Manu Sporny
Contributors (in order of contribution): Manu Sporny, Brian Suda

This HOW-TO provides a step-by-step tutorial in the process of creating a new Microformat from beginning to end.

Step 0: Before You Start

You must make sure that you understand the basics before you propose a new Microformat. The first mistake that you can make is to not educate yourself about what we do here.

1. Educate Yourself About How This Community Operates - Explain the logic...
   1. You can't create a Microformat until you understand some of the basics. Understanding what microformats are is the first step. Understanding what kinds of Microformats exist is the second step. Finally, understanding the microformats process is very important.
2. POSH-ify your website
   1. Making your website more POSH is important to understanding web semantics.
3. Implement a few pre-existing Microformats on your Website
   1. Implementing a microformat or two will teach you a great deal about how they are used in practice. Implement a couple on your website and use Operator (in debug mode) to see if you're marking up the information correctly.

Step 1: Determine if a New Microformat is Needed

After you understand the basics, make sure you understand the problem that you are trying to solve. Once you understand the problem you are trying to solve, make sure it hasn't already been solved.

1. Create a Problem Statement
   1. The Problem Statement is a very specific paragraph that outlines a commonly published piece of data on the web. If you need an example of an initial problem statement, one can be found here: hAudio Problem Statement. Once you have finished your problem statement, proceed to the next step.
2. Apply pre-existing Microformats to your Problem Statement
   1. There is a large list of Microformat specifications, drafts and exploratory discussions that could be applied to your Problem Statement. Make sure to check the list and see if your problem can be solved using a combination of the pre-existing microformats. If you have not found a combination of microformats that can address your problem statement, proceed to the next step.
3. Ask the microformats-new mailing list for guidance
   1. There are a number of people that have been active in the community for a long time. If you have done the previous steps and you still think that your problem is not being addressed by an existing microformat, ask the microformats-new mailing list. Make sure to include your Problem Statement and let the list know that you are considering creating a new microformat to address the issue. Be ready for a variety of answers - the community rarely agrees when a new topic arises. It may take a bit of discussion to determine if a new microformat is needed.

Step 2: Gather Examples, Similar Projects and Documenting Current Behavior

If several people on the microformats-new mailing list have stated that they are interested in working on your problem statement, and gather examples. You should also note every file format

1. Gather example URLs - Explain the logic...
   1. Example URLs that contain publishing behavior should be gathered. For example, if you are attempting to come up with a new image metadata microformat, you would collect website URLs that publish image metadata (such as Flickr).
2. Gather example file and markup formats
   1. Example file formats and markup formats that demonstrate publishing behavior should be gathered. For example, if you are attempting to come up with a new image metadata microformat, you would collect file formats that are capable of encapsulating image metadata (such as as the EXIF file format).
3. Create and populate the examples page
   1. You will want to create an examples page on the wiki. For example, if you are gathering image metadata examples, you could name the page image-metadata-examples. Place all example URLs into this page. For more guidance, take a look at the audio-info-examples page.
4. Create and populate the formats page
   1. You will want to create a formats page on the wiki. For example, if you are gathering image metadata formats, you could name the page image-metadata-formats. For more guidance, take a look at the audio-info-formats page.

Step 3: Analyze the Examples for Common Attributes

Microformats are based on sound research, logical analysis and thorough vetting of the data that you have gathered. This is one of the most tedious parts of the process, and if you do not perform a thorough job at this stage, it will come back to bite you.

1. Determine common properties
   1. It is important to define and keep track of common properties found throughout the process. If you were analyzing image metadata, for instance, you would probably define the title, summary, formatted name, and publish date properties. The list will grow as you perform more analysis. Make sure to clearly define each common property so that others will know what you mean in your analysis. For an example of defining common properties, here is the definition for the audio-info title common property.
2. Analyze each example URL
   1. Once a set of common properties have been defined, start analyzing each example URL that you have collected. Note which common properties are published on each page. You will have to add common properties to the list as you find new trends while analyzing your data. Here is an example of how you can document this analysis on the wiki: audio-info analysis documentation
3. Analyze each format
   1. Perform the same type of analysis that you performed on the example URLs on the file formats and other semantic formats you have gathered.
4. Perform aggregate statistical analysis
   1. After you have completed analysis of all URLs and formats, you should perform a statistical analysis on the aggregate data. You will need hard numbers that are backed up by data to prove your point on the microformats-new mailing list. For an example of what the aggregate analysis should look like after it is completed, the aggregate analysis of music services section of the audio-info page may be of help.

Step 4: Check your work, Start Brainstorming, and Ask for Feedback

It is important that all of your findings are thoroughly documented on the wiki. You will be able to make process easier if everybody understands the data gathered. It is important that your analysis is sound and doesn't leave any open questions.

1. Ensure that you have the basics
   1. At this point you should have an examples and formats page. The examples page should contain:
      1. A Problem Statement
      2. Common Properties and their definitions
      3. A statistical analysis of all aggregate data
2. Create and populate the brainstorming page
   1. The brainstorming page is where most of the discussion on the mailing list will be documented. It is important to propose a starting point to the mailing list. Identify the common properties that you think should be included in the microformat based on the analysis. Be as minimalistic as you can. Do not propose anything that is not backed up by hard data. An example of how to propose common properties, also known as Discovered Elements, can be found on the audio-info Discovered Elements section of the audio-info-brainstorming page.
3. Ask for Feedback
   1. Send an e-mail to microformats-new stating that examples gathering and analysis has been completed for your microformat and that you would like to start brainstorming usage scenarios, markup and work toward a microformat draft specification.
   2. Brainstorming is best done with as much community involvement as possible. There are a number of people that have been involved with microformats for years and can provide valuable direction for your microformat.
4. Discuss each Discovered Element
   1. Each discovered element that you identified in the previous step must be discussed thoroughly. See if there are any objections to your proposal and discuss alternatives.
5. Update the brainstorming page
   1. For every element, you will want to discuss re-using property names from other microformats, alternatives, benefits and drawbacks. Make sure to document the discussion in a way that is non-confrontational and fair to both sides of the argument.

Step 5: Write the First Draft

The first draft will give everybody an idea of what is being proposed. The brainstorming page can get quite large, so the first draft of the microformat will give everybody a set of properties to focus upon.

1. Create the Draft Page
   1. Create the microformat draft page and populate it with a draft template. You can copy a pre-existing specification page, such as audio-info-proposal, and delete any information not relevant to your microformat.
2. Create the Basic Schema
   1. The schema is a very brief, high-level view of the microformat. It helps readers understand what class names they should use for your microformat as well as the data types for those classes.
3. Define the Field Details
   1. The field details are very specific as to how the schema should be implemented. They contain information related to how fields should be identified, parsed, their cardinality, as well as many other field-specific pieces of information.
4. Create the XMDP Profile
   1. The XMDP profile is a machine-readable description of your microformat. Every microformat must have a XMDP profile.
5. Create Markup Examples
   1. You must show readers how to properly use your Microformat. Giving them a simple example, a more realistic example, and an example that uses every class that your microformat supports is required.

Step 6: Gather Feedback on First Draft

Once you have finished the first draft, propose the draft to the microformats-new mailing list. There will be a great deal of discussion related to naming of classes, how the data should be represented, and if the microformat should exist at all. Be patient and logically discuss each point raised on the mailing list - be sure to be nice when doing so! Remember, most of the people on the list don't know as much about your Microformat as you do.

1. Create an Issues Page
   1. You should create an issues page at this point so you can document and track issues that arise during this step in the process. The issues page will also let anybody add their concerns with the microformat without having to e-mail the list.
2. Discuss Each Issue Until it is Resolved
   1. Discuss each issue in detail and attempt to come to a resolution. Be pro-active in diffusing flame-wars before they start. If you allow a flame-war to start during discussion, it will delay acceptance of your first draft.
3. Update the First Draft
   1. Once each issue is resolved, update the first draft to reflect the resolution.

The Logic Behind The Microformats Process

Logic: Educating Yourself Before Proposing a Microformat

Based on most of the "I've got a new idea for a Microformat!" posts to the mailing lists, the proposed solution almost always results in a new Microformat not being needed. Microformats are meant to be combined with one another, therefore most problems can be solved by combining a number of Microformats. If you don't know what Microformats already exist, you will want to educate yourself on them before proposing something new.

In other words - "slow your roll".

There must be a problem to be solved. No problem, no microformat.

There are other things to try before developing a microformat. First, ask yourself these questions:

1. Is there a standard element in XHTML that would work?
2. Is there a compound of XHTML elements that would work?
3. OK, if the answer to the above two is 'no,' we can talk about a microformat.

For more details on semantic XHTML, examples of using XHTML elements, and constructing XHTML compounds, see The Elements of Meaningful XHTML (http://tantek.com/presentations/2005/03/elementsofxhtml/).

You should observe the microformats principles.

After you understand the principles, ask yourself: "are there any well established, interoperably implemented standards we can look at which address this problem?" For example, hCard and hCalendar were built on top of the IETF standards for vCard and iCal, respectively, both of which are widely interoperably implemented. The developers of those standards had already spent many years in standards committees arguing about and developing the schemas. Better to leverage all the hard work that others have done before you, than to go off as a solo cowboy inventor, and waste time repeating all their mistakes. It's also much easier to start from a well established schema, and map into into XHTML than to develop a new schema.

Also, search around on the web. Chances are that someone else has encountered the same problem as you. If you still believe that you have an unsolved problem, post something to the microformats-new (http://microformats.org/mailman/listinfo/microformats-new/) mailing list or any other public channel (see http://microformats.org/discuss/) We want to involve all interested parties in the discussion. Start the discussion BEFORE you start creating any pages on the wiki. We're not using the wiki as a general "scratch pad". If you can't summarize the problem you are trying to solve in a short email, and feel like you need a long document, you're probably trying to solve too big of a problem

Once you've found your 'problem,' ask yourself: 'is there a simpler problem here?' If so, let's solve that problem first. We want to deal with the simplest problems first and only then build up to more complex problems.

Logic: Gathering Examples First

Remember, we're paving the cowpaths (http://ifindkarma.typepad.com/relax/2004/12/microformats.html)- before you do that you have to find the cowpaths. Your examples should be a collection of real world sites and pages which are publishing the kind of data you wish to structure with a microformat. From those pages and sites, you should extract markup examples and the schemas implied therein, and provide analysis.

This collection of examples should be public, preferably on a wiki because there's no way you can do it by yourself (no matter how many of you there are). The reviews-formats page is a good example of research done before the creation of a microformat. Before developing hReview, the collaborators went out, documented current practices around reviews on web sites, and provided some analysis of the schemas implied therein.

It's quite possible during this step that you'll find someone else who has dealt with the problem you're addressing. Perhaps even solved it. Do your best to open a dialog with others who have encountered the same problem. We don't want to build walls between competing communities - we want people to work together to develop a good solution which will cover the majority of cases.