Package openid :: Package consumer :: Module html_parse

Module html_parse

This module implements a VERY limited parser that finds <link> tags in the head of HTML or XHTML documents and parses out their attributes according to the OpenID spec. It is a liberal parser, but it requires these things from the data in order to work:

There must be an open <html> tag
There must be an open <head> tag inside of the <html> tag
Only <link>s that are found inside of the <head> tag are parsed (this is by design)
The parser follows the OpenID specification in resolving the attributes of the link tags. This means that the attributes DO NOT get resolved as they would by an XML or HTML parser. In particular, only certain entities get replaced, and href attributes do not get resolved relative to a base URL.

From http://openid.net/specs.bml#linkrel:

The openid.server URL MUST be an absolute URL. OpenID consumers MUST NOT attempt to resolve relative URLs.
The openid.server URL MUST NOT include entities other than &, <, >, and ".

The parser ignores SGML comments and <![CDATA[blocks]]>. Both kinds of quoting are allowed for attributes.

The parser deals with invalid markup in these ways:

Tag names are not case-sensitive
The <html> tag is accepted even when it is not at the top level
The <head> tag is accepted even when it is not a direct child of the <html> tag, but a <html> tag must be an ancestor of the <head> tag
<link> tags are accepted even when they are not direct children of the <head> tag, but a <head> tag must be an ancestor of the <link> tag
If there is no closing tag for an open <html> or <head> tag, the remainder of the document is viewed as being inside of the tag. If there is no closing tag for a <link> tag, the link tag is treated as a short tag. Exceptions to this rule are that <html> closes <html> and <body> or <head> closes <head>
Attributes of the <link> tag are not required to be quoted.
In the case of duplicated attribute names, the attribute coming last in the tag will be the value returned.
Any text that does not parse as an attribute within a link tag will be ignored. (e.g. <link pumpkin rel='openid.server' /> will ignore pumpkin)
If there are more than one <html> or <head> tag, the parser only looks inside of the first one.
The contents of <script> tags are ignored entirely, except unclosed <script> tags. Unclosed <script> tags are ignored.
Any other invalid markup is ignored, including unclosed SGML comments and unclosed <![CDATA[blocks.

Functions

[hide private]

tagMatcher(tag_name, *close_tags)

source code

replaceEnt(mo)
Replace the entities that are specified by OpenID

source code

[[(type(html), type(html))]]

parseLinkAttrs(html)
Find all link tags in a string representing a HTML document and return a list of their attributes.

source code

relMatches(rel_attr, target_rel)
Does this target_rel appear in the rel_str?

source code

linkHasRel(link_attrs, target_rel)
Does this link have target_rel as a relationship?

source code

findLinksRel(link_attrs_list, target_rel)
Filter the list of link attributes on whether it has target_rel as a relationship.

source code

findFirstHref(link_attrs_list, target_rel)
Return the value of the href attribute for the first link tag in the list that has target_rel as a relationship.

source code

Variables

[hide private]

flags = 114

removed_re = re.compile(r'(?isux)|!\[CDATA\[.*?\]\]>...

tag_expr = '\n# Starts with the tag name at a word boundary, w...

html_find = re.compile(r'(?isux)<html\b(?!:)(?P<attrs>[^>]*?)(...

head_find = re.compile(r'(?isux)<head\b(?!:)(?P<attrs>[^>]*?)(...

link_find = re.compile(r'(?isux)<link\b(?!:)')

attr_find = re.compile(r'(?isux)(?P<attr_name>\w+)=(?:(?P<qope...

replacements = {'amp': '&', 'gt': '>', 'lt': '<', 'quot': '"'}

ent_replace = re.compile(r'&(amp|lt|gt|quot);')

__package__ = 'openid.consumer'

Function Details

[hide private]

parseLinkAttrs(html)

source code

Find all link tags in a string representing a HTML document and return a list of their attributes.

Parameters:

html (str or unicode) - the text to parse

Returns: [[(type(html), type(html))]]

A list of dictionaries of attributes, one for each link tag

Variables Details

[hide private]

removed_re

Value:

re.compile(r'(?isux)<!--.*?-->|!\[CDATA\[.*?\]\]>|script\b(?!:)[^>]*>.
*?</script>')

tag_expr

Value:

'''
# Starts with the tag name at a word boundary, where the tag name is
# not a namespace
<%(tag_name)s\\b(?!:)

# All of the stuff up to a ">", hopefully attributes.
(?P<attrs>[^>]*?)

...

html_find

Value:

re.compile(r'(?isux)<html\b(?!:)(?P<attrs>[^>]*?)(?:/>|>(?P<contents>.
*?)(?:</?html\s*>|\Z))')

head_find

Value:

re.compile(r'(?isux)<head\b(?!:)(?P<attrs>[^>]*?)(?:/>|>(?P<contents>.
*?)(?:</?(?:head|body)\s*>|\Z))')

attr_find

Value:

re.compile(r'(?isux)(?P<attr_name>\w+)=(?:(?P<qopen>["\'])(?P<q_val>.*
?)\2|(?P<unq_val>(?:[^\s<>/]|/(?!>))+))|(?P<end_link>[<>])')