microformats2 parsing brainstorming

Jump to: navigation, search


This page is for brainstorming, discussion, and other questions and explorations about microformats2 parsing.

For the microformats2 parsing algorithm, see:

For filing issues / problems with microformats2-parsing, see:

Contents


Parse img alt

Per https://github.com/microformats/microformats2-parsing/issues/2 currently any u-* property (e.g. u-photo, u-featured) that extracts a 'src' attr from an img tag loses any associated 'alt' text alternative, and if at some point the consuming application wants to display that u-* property as an img, they have to either omit or synthesize a fake text alternative.

It is desirable to somehow maintain that image src and alt association from the original markup, through the parsing process, up until a consuming application wishes to re-present the image with the text alternative.

There are a number of possibilities / approaches here worth brainstorming:

Include alt property in parent object

  1. explicit authoring: require the author to use a new 'p-alt' property on the image to cause parsing and extraction of the text alternative.
    • Problem(s): fails for multiple images, some of which may or may not have alt attrs or corresponding p-alt properties (and fragile, forgetting one p-alt throws off the parallel lists of u-* and p-alt).
  2. implicit p-alt: for every img that is parsed for a u-* property, the parse could generate a p-alt property with value.
    • Problem(s): fragile again for similar reasons, not all u-*s may be on img elements, or may not have alt attrs for all imgs in the source.
  3. implicit p-alt only for implied u-photo
    • This is better since there can only be one implied u-photo, and thus if there is a p-alt, it must be associated with the one u-photo
    • Problem(s): does not work for other u-* image properties e.g. u-featured

<div class="h-entry"><img src="http://example.com/photo.jpg" alt="Example" class="u-photo p-alt"></div>

{"type":["h-entry"],"properties":{"photo":["http://example.com/photo.jpg"],"alt":["Example"]}

Make photo property an object

1. use "h-image" on any u-* on img elements to imply a structure with paired photo and 'name' text alternative, e.g.
<img src="a.jpg" alt="text about a" class="u-featured h-image"/>
which would result in a u-featured property with one value, a structure of an h-image with itself having implied properties of a u-photo of "a.jpg" and a p-name of the "text about a". Similarly the author can use the object tag for the same result:
<object data="a.jpg" class="u-featured h-image">text about a</object>
In either case, the same microformats JSON would be generated, which is correct, as in both cases, there is an image with a fallback text alternative. The specific HTML used should not matter. The semantic of pairing the image with the text alternative is communicated the same way for both.
<div class="h-entry">
 <img src="http://example.com/eg.jpg" alt="Photo of an example" class="u-featured h-image">
</div>
{
"type":["h-entry"],
"properties":{
  "featured":[{
    "type":["h-image"],
    "properties":{
      "photo":["http://example.com/eg.jpg"],
      "name":["Photo of an example"]
    }
  }]
}
[1]


2. have u-* on an <img> automatically create an object if there is a non-empty 'alt' attribute.
If a u-* property is parsed on an <img> element with a non-empty 'alt' attribute, then:
Create a structure similar to the e-content nested structure that provides the "value" as the URL, and an "alt" as the text alternative.

<div class="h-entry">
 <img src="http://example.com/eg.jpg" alt="Photo of an example" class="u-featured">
</div>
{
"type":["h-entry"],
"properties":{
  "featured":[{
    "value":"http://example.com/eg.jpg",
    "alt":"Photo of an example"
  }]
}

... more brainstorming needed

img alt thoughts

Thoughts about img alt brainstorm proposals. Feel free to offer counterpoints with nested items and/or alternative preferences/opinions with (potentially multiple) top level items!

  • Tantek: I am leaning towards "Make photo property an object" brainstorm "2." because it feels more "automatic" and thus provides lower friction to more accessibility. Less (author) work for "alt" information to get passed through to the JSON result, and thus more potentially re-usable by consuming applications that want to preserve or re-emit the pairing of a photo and its fallback text alternative. -- Tantek 00:53, 19 July 2016 (UTC)
  • Aaron: I am leaning towards 2 because it takes less work on the part of publishers as well as consumers. From the publisher POV, if they add the alt attribute, that should be all they need to do, it seems odd to make them do additional work to make that show up in the parsed result. From the consumer side, some implementations will not need changing since when looking for a string value, they already use either the string directly or look for the "value" of the property if it's an object. Making consumers handle a new h- object just to read alt text seems overkill.
    • Additionally, if the alt attribute is an empty string, this should be considered the same as if it were missing, so that the photo value will be the URL string rather than the object in this case as well
  • Kevin: 2 makes sense to me as well, as this is a very specific need. If we want an image object with more substructure as 1 implies, that should be a new object type that follows the process - there is a case for that based on usage of figure/figcaption etc. but caption is not alt, and using name for it implies that it is. Kevin Marks 01:50, 19 July 2016 (UTC)
  • Bear: The thoughts given above for option 2 make the most sense as a library writer and consumer, tying this change to a parser implementation's major version change will (should) give everyone notice and time to adjust

...

  • (unanimity copied to GitHub)

When it looks like thoughts are naturally converging, we should take that emergent convergence back to the github thread for proper back/forth discussion and figuring out of details.

https://github.com/microformats/microformats2-parsing/issues/2

Parse language information

Raised by VoxPelli 18:04, 23 July 2015 (UTC)

Currently there’s no way to tell the language of parsed microformats even if those microformats has been marked up with HTML "lang"-attributes.

There are examples in the wild of people marking up pages in such a way:

Proposal is to add a new "lang" keyword to h-* and e-* objects so that the following example:

<div class="h-entry" lang="sv">
  <h1 class="p-name">En svensk titel</h1>
  <div class="e-content" lang="en">With an <em>english</em> summary</div>
  <div class="e-content">Och <em>svensk</em> huvudtext</div>
</div>

Would be parsed into something like:

{
  "type": ["h-entry"],
  "lang": "sv",
  "properties": {
    "name": ["En svensk titel"],
    "content": [
      {
        "lang": "en",
        "html": "With an <em>english</em> summary",
        "value": "With an english summary"
      },
      {
        "html": "Och <em>svensk</em> huvudtext",
        "value": "Och svensk huvudtext"
      }
    ]
  }
}

This was brainstormed on the IndieWebCamp IRC-channel where the mentioned example came up.

Additionally: consider the same for "id" attributes (use-case: rel=feed local discovery of a nested h-feed on the home page), specifically, parsing the first instance of any "id" attribute (ignoring latter duplicate id attribute values on any subsequent elements).


And alternatively: consider parsing as "html-id" and "html-lang" prefixed properties in the parsed result, e.g.


<div class="h-entry" lang="sv" id="postfrag123">
  <h1 class="p-name">En svensk titel</h1>
  <div class="e-content" lang="en">With an <em>english</em> summary</div>
  <div class="e-content">Och <em>svensk</em> huvudtext</div>
</div>

Would be parsed into something like:

{
  "type": ["h-entry"],
  "html-id": "postfrag123",
  "html-lang": "sv",
  "properties": {
    "name": ["En svensk titel"],
    "content": [
      {
        "html-lang": "en",
        "html": "With an <em>english</em> summary",
        "value": "With an english summary"
      },
      {
        "html": "Och <em>svensk</em> huvudtext",
        "value": "Och svensk huvudtext"
      }
    ]
  }
}

Language inheritance

If the "lang" attribute is not specified for a particular element, it is inherited from the nearest parent (or from the HTTP Content-Language header)

HTML5: https://www.w3.org/TR/html5/dom.html#the-lang-and-xml:lang-attributes
HTML4: https://www.w3.org/TR/html4/struct/dirlang.html#h-8.1.2

Proposal: Determine and include the inherited "lang" value on *every* microformat object that directly specifies a lang or that has an ancestor that does, e.g. if <html lang="en">, then every object in the output will have "lang": "en".

Pronouns in different languages

Language is also useful context when defining pronouns, discussed a bit here[2].

<div class="h-card" lang="en">
  <span class="p-x-pronoun-nominative">he</span> /
  <span class="p-x-pronoun-possessive">him</span> /
  <span class="p-x-pronoun-oblique">his</span>
</div>

would parse as

{
  "type": ["h-card"],
  "lang": "en",
  "properties": {
    "x-pronoun-nominative": ["he"],
    "x-pronoun-possessive": ["him"],
    "x-pronoun-oblique": ["his"]
  }
}

It could also be useful to specify multiple languages within a single h-card (pardon me if I butcher Swedish pronouns)

<div class="h-card">
  <span lang="en" class="p-x-pronoun-nominative">he</span> /
  <span lang="en" class="p-x-pronoun-possessive">him</span> /
  <span lang="en" class="p-x-pronoun-oblique">his</span>
  <span lang="sv" class="p-x-pronoun-nominative">han</span> /
  <span lang="sv" class="p-x-pronoun-possessive">hans</span> /
  <span lang="sv" class="p-x-pronoun-oblique">honom</span>
</div>

which might parse as

{
  "type": ["h-card"],
  "properties": {
    "x-pronoun-nominative": [{"lang": "en", "value": "he"}, {"lang": "sv", "value": "han"}],
    "x-pronoun-possessive": [{"lang": "en", "value": "him"}, {"lang": "sv", "value": "hans"}],
    "x-pronoun-oblique": [{"lang": "en", "value": "his"}, {"lang": "sv", "value": "honom"}]
  }
}

or alternatively, we could introduce a new microformat h-x-pronoun to wrap a set of pronouns

<div class="h-card">
  <div class="p-x-pronoun h-x-pronoun" lang="en">
    <span class="p-nominative">he</span> /
    <span class="p-possessive">him</span> /
    <span class="p-oblique">his</span>
  </div>
  <div class="p-x-pronoun h-x-pronoun" lang="sv">
    <span class="p-nominative">han</span> /
    <span class="p-possessive">hans</span> /
    <span class="p-oblique">honom</span>
  </div>
</div>


parsed as

{
  "type": ["h-card"],
  "properties": {
    "x-pronoun": [{
      "type": ["h-x-pronoun"],
      "lang": "en",
      "properties": {
        "nominative": ["he"],
        "possessive": ["him"],
        "oblique": ["his"]
      }
    }, {
      "type": ["h-x-pronoun"],
      "lang": "sv",
      "properties": {
        "nominative": ["han"],
        "possessive": ["hans"],
        "oblique": ["honom"]
      }
    }]
  }
}


Discussion:

  • Kylewm Including the "lang" attribute in h- and e- properties makes a ton of sense to me.
  • Kylewm I like the idea of introducing an h-x-pronoun container that can define all the different pronoun forms for a particular language
  • ...

Canonicalization of datetime output

Status: resolved, awaiting implementation attempt/experience.

It would be useful to choose a (more) uniform output format for datetimes to make it easier for users of the parser to consume datetimes. Microformats2 parsers already do sophisticated pattern matching to recognize date vs. time vs. datetimes, so converting this to any specific format should not add overhead.

Specifically:

  • Choose either 'T' or space as the date/time separator.
    • Prefer space as it is more human friendly/readable, which matters even for syntaxes/formats, as human still develop, debug them. Tantek 19:31, 6 January 2015 (UTC)
  • Choose either +XXYY or +XX:YY as the timezone specification (and convert 'Z' to +0000).
    • Would appreciate some study / input here as to which timezone offset syntax is more human friendly. I lean slightly toward +/-NNNN (without the colon) because in the context of seeing a time, leaving out the colon makes it less likely the offset will be confused for a time. E.g. "07:00-08:00" looks like 7-8am, even if it meant 07:00 in PST. Tantek 19:31, 6 January 2015 (UTC)
    • Space is fine - consensus 2015-01-20 meetup.
  • Parsers should not attempt make datetimes more exact than specified. They should not add time, seconds, or timezone if omitted in the original. Kylewm 04:02, 14 May 2014 (UTC)
    • Agreed. Tantek 19:31, 6 January 2015 (UTC)
    • or month, day per Tom Morris
    • consensus 2015-01-20 meetup
  • Counterpoint: PHP's builtin date parsing does not require strict formatting. And the equivalent functionality for Python is provided by the widely used python-dateutil library. Kylewm 19:02, 14 May 2014 (UTC)
    • However we cannot (must not) depend on either PHP or Python's "smart" "fixing" or Postelian "liberal handling", or any other language/framework's for that matter, as they all differ in how "intelligent" they are. Tantek 19:31, 6 January 2015 (UTC)

Perhaps just provide a guideline for these based on the above consensus.

Add meta http-equiv to microformats2 parsing model

Status: disagreement, awaiting implementation attempt/experience.

Similar to document level parsing of rel attributes, it makes sense simultaneously to parse <meta http-equiv> elements, perhaps treating "Status" in a special way (only using first number (sequence of digits) for its "value").

Use case: IndieWeb "deleted" indication inline in content for static file services that don't support HTTP return codes.

HTTP Header example:

HTML equivalent:

Related:

  • Interesting thought. Are you suggesting a top level "http-equivs:" collection similar to "rels:" in the parsed output? Should we consider "metas:" instead or in addition? Tantek 19:31, 6 January 2015 (UTC)
  • What's the use case for this? Also, http-equiv on its own is useless. http-equiv is only a supplement to the data stored in headers. And headers aren't always there: what happens in the context of someone debugging a page who pastes the source into the textarea of an mf2 parser? Without a compelling use case for including headers (and then over-riding some of them with http-equivs), I'm not sure why an implementor want to do this. —Tom Morris 00:25, 8 May 2015 (UTC)

E.g. from https://gist.github.com/aaronpk/10297489

<meta http-equiv="Status" content="410 GONE"/>
{
 "items": [],
 "rels": {},
 "http": {
 "status": 410
 }
}
  • Maybe make this an optional pass in the parser? - Tom Morris 2015-01-20
  • For now, don't bother with metas until someone provides a use-case. Tom Morris
  • Agreed on both counts. Tantek 06:56, 21 January 2015 (UTC)


MIME type

See microformats2-mime-type


Other Interpretation Parsing Notes

Note: most of these need to be written up as separate microformats2-parsing-issues

Author: Ben Ward

Microformats 2 proposes a new, all encompassing syntax modification of prefixes that will allow microformats to be parsed from pages by processors without prior knowledge of a vocabulary. The core components of this model are quite simple, are quite simple to implement, but there are a number of conflicts that emerge with the functionality of existing microformats parsers that need to be handled. This page documents a proposed model to separate these concerns clearly in a way that can be applied to the documentation of generic microformats parsing rules, and the documentation of individual vocabularies.

Collection of other unresolved parsing issues in a generic model:

This is good material for documenting as microformats-2-issues, microformats-2-faq, and perhaps some of the more technical details in microformats-2-parsing-faq.

incorporated 2015-05-28

The following brainstorms were incorporated 2015-05-28.

more information for alternates

Raised 2015-04-24 by Kevin Marks

The existing alternate parsing is omitting title - that should be added. The text would make sense to add here too.

Use-case: labels for presenting alternates

  • +1 Makes sense. Tantek 03:41, 25 April 2015 (UTC)

more information for rel-based formats

Raised 2015-04-18 by Kevin Marks

Related github test suite issue: https://github.com/microformats/tests/issues/16

Several rel-based formats have additional information that is useful beyond the link itself, which is all we capture at the moment. As I am trying to update the Universal feedparser to support mf2 based I will show examples from the testcases there.

The main change is to add a rel-urls entry for more information about the attributes and text of the urls pointed to by rel's in the document

A fork of mf2py that implements these changes is at https://github.com/kevinmarks/mf2py

rel-tag

<a rel="tag" href="http://del.icio.us/tag/tech">Technology</a> 

currently parses to:

{"rels": {"tag": ["http://del.icio.us/tag/tech"]}, "items": []} 

This loses the link text, which is useful as a label.

We add a rel-urls element to the parsed output with this extra data that can be looked up from the rels, which doesn't break backward compatibility and works better with xfn (see below)

{
    "rels": {
        "tag": [
            "http://del.icio.us/tag/tech"
        ]
    }, 
    "items": [], 
    "rel-urls": {
        "http://del.icio.us/tag/tech": {
            "rels": [
                "tag"
            ], 
            "text": "Technology"
        }
    }
}

xfn

<a rel="coworker" href="http://example.com/johndoe">John Doe</a>

currently parses to:

{"rels": {"coworker": ["http://example.com/johndoe"]}, "items": []}

This loses the link text, which is the person's name. Suggested output using the urls object:

{
{
    "rels": {
        "coworker": [
            "http://example.com/johndoe"
        ]
    }, 
    "items": [], 
    "rel-urls": {
        "http://example.com/johndoe": {
            "rels": [
                "coworker"
            ], 
            "text": "John Doe"
        }
    }
}

with multiple xfn values

<a rel="coworker friend" href="http://example.com/johndoe">John Doe</a> we get this:

{
    "rels": {
        "coworker": [
            "http://example.com/johndoe"
        ], 
        "friend": [
            "http://example.com/johndoe"
        ]
    }, 
    "items": [], 
    "rel-urls": {
        "http://example.com/johndoe": {
            "rels": [
                "coworker", 
                "friend"
            ], 
            "text": "John Doe"
        }
    }
}

rel-enclosure

<a rel="enclosure" href="http://example.com/movie.mp4" type="video/mpeg" title="real title">my movie</a>

currently parses to:

'{"rels": {"enclosure": ["http://example.com/movie.mp4"]}, "items": []}'

This loses the link text, which is the title and the attributes which give type. Suggested output:

{
    "rels": {
        "enclosure": [
            "http://example.com/movie.mp4"
        ]
    }, 
    "items": [], 
    "rel-urls": {
        "http://example.com/movie.mp4": {
            "rels": [
                "enclosure"
            ], 
            "text": "my movie", 
            "type": "video/mpeg", 
            "title": "real title"
        }
    }
}

This generalises to other rel's too, such as rel-feed and rel-alternate that have type, lang etc attributes.

(updated to include changes from feedback below) Kevin Marks 22:13, 26 April 2015 (UTC)

attributes parsed

Attributes currently parsed are:

Attributes we may consider parsing if we have a use case are

In addition there is a special attribute name text which is the text contents of the link, which is useful in rel-tag rel-enclosure and xfn, and in alternate when used for feeds. It's also clarifying for rel-me links.

Tantek suggests we use textContent for this instead, and make it a single string, not a list as name is elsewhere in mf2 parsing

feedback on more rel info

  1. "name" is bad because it misleadingly conflates with use of "name" elsewhere in microformats2.
    • Suggested alternative: textContent - since that's literally what is being returned there. Tantek 02:35, 25 April 2015 (UTC)
      • as all other mf2 keys are lowercase-with-hyphens, Tantek suggests 'text' as that isn't going to be an html Kevin Marks 07:28, 25 April 2015 (UTC)
  2. no need for array for "name"/textContent - since there is always only one at most
    • E.g. should be "textContent": "my movie" Tantek 02:35, 25 April 2015 (UTC)
    • Update: "text": "my movie" Tantek 04:39, 29 May 2015 (UTC)
  3. "urls" key is misleading - implies all URLs in the document, which is neither true, nor desired (takes much more parsing time and work and code)
    • Suggested alternative: "rel-urls". And open to better alternatives too. Tantek 02:35, 25 April 2015 (UTC)
      • If we are trying to extend the number of properties retuned from a rel without breaking the old structure why don't we call the new structure something like "rels-extended" Glenn Jones 12:29, 1 June 2015 (UTC)
        • Extension is not the point, but rather to use them complementary. One structure for look-up of any rel value, hence "rels", which returns you a list of URLs. Then you can lookup those URLs in the new mapping, by URL, hence it is called "rel-urls" - that's the point to use them in conjunction and that's why rel-urls is named what it is. Tantek 22:03, 1 June 2015 (UTC)
  1. Why is the structure of "rel-urls" different to the "alternates" structure. Should the "url" not just be added as a property and not as a key. Creating two data structures for one type of object seems inconsistent. It adds cognitive load to anyone trying to understand the JSON structure Glenn Jones 12:29, 1 June 2015 (UTC)
    • I was trying to avoid breaking the existing rels structure and use of it - I did implement a variant that put the structure inside rels, and it became cumbersome and repetitive where there were multiple rels on a url (xfn cases). Denormalising as properties of the URL made more sense. It also dedupes if there is repetitive linking to the same URL, eg a series of posts with rel-author on each. Kevin Marks 20:05, 1 June 2015 (UTC)
  2. If the rel is a "tag" then the main value we need to return should be the last path component of the URL, not the link text? Should we add another output property ie "tag" Glenn Jones 12:29, 1 June 2015 (UTC)
    • No need to return last path segment of the URL, because the URL is already there - and that's just a library/framework utility function to get the last path segment of a URL. Tantek 22:03, 1 June 2015 (UTC)
  3. As currently described, the URL from alternates is repeated in the rel-urls structure. If we are doing this, surely alternate should be in rels too? I assumed a mapping between them. Kevin Marks 20:05, 1 June 2015 (UTC)
    1. edit showing this variant: http://microformats.org/wiki/index.php?title=microformats2-parsing&oldid=65021#parse_a_hyperlink_element_for_rel_microformats
    • Yes it makes sense to drop "alternates" assuming the backcompat impact is low, put alternates in "rels" along with everything else, and direct people to use rels and rel-urls for alternates functionality. Evidence this is an acceptable even preferable approach.[3] Will add an issue accordingly. Tantek 22:03, 1 June 2015 (UTC)

Incorporated 2015-06-06

Nested h-* objects' "value" property

Status: resolved, resolution iterated, one real world implementation proven implementability, incorporated

Raised 2015-01-06 by User:Kylewm;

If a child element has a microformat (h-*) and is a property element (p-*, u-*, dt-*, e-*), the parser will add a "value" property to the resulting object. The value should attempt to be a useful representation of the object for consumers that do not have semantic knowledge of the particular h-* type. Ref: microformats2-parsing#parse_an_element_for_class_microformats.

To determine the "value", we parse the property element simply (as if it did not have a h-* class), which works well for simple h-* objects, e.g. <a class="u-like-of h-cite" href="...">...</a>

  • To handle more complex microformats, I propose that "value" for a p-* property element take on the first explicit "name" property of the nested microformat, and for a u-* property, the first explicit "url" property. Parsing will fall back on the current rules if an explicit property does not exist.
    • This makes sense to me, and fits with the use-cases and examples I've seen. Tantek 19:31, 6 January 2015 (UTC)
    • A similar (possibly simpler?) formulation would use the implied name and url rules to determine the "value" for p-* and u-* properties respectively
      • I don't think that's needed, as there are already implied rules on a property that should handle that. I'd start with just the "first explicit" scoping to be more conservative, and then see if we find any use-cases that (and existing implied rules) don't/doesn't catch. Tantek 19:31, 6 January 2015 (UTC)

For example:

<div class="h-entry">
  <div class="u-in-reply-to h-cite">
    <a class="p-author h-card" href="http://example.com">Example Author</a>
    <a class="p-name u-url" href="http://example.com/post">Example Post</a>
  </div>
</div>

The nested u-in-reply-to object would parse as

...
"in-reply-to": [{ 
  "type": ["h-cite"],
  "properties": {
    "name": ["Example Post"],
    "url": ["http://example.com/post"],
    "author": [{
      "type":["h-card"],
      "properties": {
        "url": ["http://example.com"], 
        "name": ["Example Author"]
      },
      "value": "Example Author"
    }],
  },
  "value": "http://example.com/post"
}]
...

where the outer "value" gets the in-reply-to h-cite's u-url property, and the inner "value" gets the author's p-name property.

  • Because there are no implied properties of the dt-* and e-* types, and no obvious defaults, the value rules for these types would not change.
    • A possibility for dt-* h-*: The dt-* could take either the first dt-* of the h-*, or (perhaps if no dt-* in the h-*,) the first <time> element inside. Tantek 19:31, 6 January 2015 (UTC)
    • First dt-* seems reasonable, predictable, and usable. Consensus at 2015-01-20 meetup.
    • Update 2015-05-29: no known use-cases for first dt-* or first e-*, and implementing that "would require some refactoring" (in mf2py at least per kylewm), thus until there's a use-case for first dt-*/e-* inside, let's treat "dt-* h-*" and "e-* h-*" as before. Tantek . In particular:
      • p-* h-* - value from first "name" as proposed above
      • u-* h-* - value from first "url" as proposed above
      • e-* h-* - value is already defined for e-* parsing, nothing special here
      • dt-* h-* - value from normal dt-* parsing - nothing special.
      • +1 totally agree, let's wait for use cases of e-* dt-* Kylewm 19:44, 29 May 2015 (UTC)

see also

microformats2 parsing brainstorming was last modified: Friday, January 13th, 2017

Views