[uf-discuss] Live Writer and microformats

[Joe Andrieu]

Chris Messina wrote:
> This kind of delves into the "authoritative standards" 
> doesn't it? And how do you keep up with an ever-growing 
> repository of microformats?
> 
> Obviously it can be done -- and it's something that we need, 
> especially if we're to get support for microformats into 
> client-side software... so isn't that where remote 
> definitions/profiles come in? Sorry for showing my ignorance 
> here, but how would, for example, Flock, (or better, Lucene) 
> stay up-to-date with the most current listing/definitions of 
> all the available microformats?

Chris,

Because of the current structure of microformats, as a way of thinking and
as a process, I believe it is inherently incapable of handling a rapidly
growing list of microformats.

Someone once said to a proposal of mine that a large number of microformats
was in fact an "anti-goal".  The way the system is set up, neither the
supporting technologies nor the uF community is capable of handling a
rapidly growing set of uF.  

I'll address the latter first. We have a very slow, painstaking process,
managed by volunteers and often executed by newbies.  In fact, I expect that
the typical new microformat is likely to be championed and pushed through by
people new to the system, for structural reasons. New people hear about uF,
get excited, and find out that the uF they need/want doesn't exist. The
answer: jump into the community and find a way to contribute.  And yet, the
process we have is moderated through a mix of informal communications
channels (email, wiki, IRC, and meetups), "driven" by consensus based
decision making. And "driven" is at about the pace of molasses.

All respect to everyone who is contributing here. It's good work. A lot of
work. It takes time to go through the process and tweak out a good solution
and then champion that to the community.  It will always take effort,
intellectual investment, and time. I've seen that process create value in
ways that surprised me. Yet, it is slow. And because there are few economies
of scale or differentiation, it is most likely going to get slower as more
people get involved, clogging the central communications channels with wider
and more various needs and interests and various new microformat proposals.

Take the work Andy has been doing on the species uF.  He's prolific. He's
putting in a lot of effort and making headway in the process. He's also
relatively new, fitting the pattern I described above.  I respectfully
suggest that much of the friction with Andy can be attributed to the need to
acclimate new users to the community.  I had a similarly frustrating first
few conversations with the list.  It's the nature of finding one's way
through the social norms of a new community.  And when the typical dynamic
is that new people get turned on to uF and want to contribute, the natural
result is that a large number of uF initiatives will go through the
normalization/socialization/learning curve of the champion.  This slows down
what is already slow from the consensus decision making process and
volunteer labor force.  And as more people dive into the community, the more
divergent the forces for the cultural norm become, which can itself make it
harder to maintain a consistent, viable norm.

Second, because there is no mechanism for discovery of microformats, every
tool has to be hand-carved to handle each microformat it wants to support.
In contrast, if we required profiles, and if profiles provided a machine
readable schema of the microformat, tools could at least present users with
structured data that at least retains its typed values.  Of course, the
/semantic/ value of that auto-discovered microformat is more challenging; we
don't have a good way to define arbitrary semantics.  But at least the data
can be extracted and managed in a lossless fashion. Plus, if the format
contains well-understood components such as hCard or hEvent, at least the
application could provide reasonable interfaces and functionality for those.


As I mentioned in a previous email, I'm looking at tools where extracting uF
from web pages into the app is a common activity, and in which, as the
application provider, I may fundamentally not know about custom uses that my
users have coded.  For these custom uses, the App doesn't need to know about
the uF semantics, it just needs to hand the resultant structured data to the
custom code that does.  I would love to have a standard way to "discover" uF
and hand them off to custom code so that as a platform, I can support any uF
defined using this discoverable mechanism.

With profiles optional and lacking any sort of discovery mechanism, tools
just have to code for each uF. The way Flock, and others, stay up to date
with uF, is to simply stay in touch with the community and keep pumping the
current finished (or beta) uF versions into their development pipeline.  I
don't believe there is any way without discovery for IE or Firefox or Google
Desktop or Yahoo mail to magically hand over a parsed representation of an
arbitrary uF to an extension.

I think these are hard problems, and there are good reasons for uF to take
the simpler, 80/20 rule to get /something/ working.  The progress made here
in the timeframe you guys have been working on it is incredible. (I still
feel like a lurker and a newbie.)

So, I think it is important to realize that the fundamental structure of uF
as a process, organization, and way of thinking, inherently restricts the
pace of growth.  uF is set up to slowly forge several dozens of uF, crafting
socially accepted, shared semantics and encoding standards for use in
semantic XHTML.  Good stuff. And far more accessible and useful than most of
the other semantic web technologies out there.

If you want things to go faster or be more extensible, uF would have to
undergo some radical changes, anything from formal incorporation, funding,
and staffing (for speed), to mandatory profiles and formal schemas for
extendability.

Yes, I know there are problems with the potential solutions I mention here.
uF is the way it is for pretty good reasons.

With all that said, maybe I'm looking at things upside down.  I asked Tantek
early on how microformats scaled.  His reply was that could be better
answered over a beer.  Unfortunately, I haven't made it to any of the social
functions to chat over that beer, so maybe I could get some clarification
here.

I'm I missing something?  Or is the scaling of uF, in terms of the # of uF,
constrained as I've outlined here?  

If so, we can either accept that and include managing expectations about
that in our socialization/education/outreach efforts, or we can identify
ways to change it and proceed on that route.  I'm not advocating either,
although I clearly have a leaning, but I do think that as uF take off, there
will be more and more pressure to create more and more specialized
microformats and to do so faster and faster.  Anything we can do that
transforms that pressure into positive traction rather than frustration and
disenfranchisement is A Good Thing(tm), IMNSHO.  

At the same time, organizational development is a wicked hard problem and
one we should be very delicate in approaching, if approaching it at all.

-j


--
Joe Andrieu
joe at andrieu.net
+1 (805) 705-8651