A Comparative Assessment of Web Accessibility and Technical Standards Conformance in Four EU States

Carmen Marincu

Barry McMullin

eAccessibility Lab

Research Institute for Networks and Communications Engineering (RINCE)

Preprint: April 2004
Final version published in First Monday, volume 9, number 7 (July 2004).
[Preprint also available in PDF Format.]

Abstract:

Design of accessible Web content is codified in standards and guidelines of the World Wide Web Consortium (W3C). Conformance with W3C's Web Content Accessibility Guidelines 1.0 (WCAG) (and/or similar, derivative, guidelines) is now the subject of considerable activity, both legal and technical, in many different jurisdictions.

This paper presents results of a comparative survey of Web accessibility guidelines and HTML standards conformance for samples of Web sites drawn from Ireland, the United Kingdom, France and Germany. It also gives some recommendations on how to improve the accessibility level of the Web content.

A particular conclusion of the study is that the general level of Web accessibility guidelines and HTML standards conformance in all of the samples studied is very poor; and that the pattern of failure is strikingly consistent in the four samples. Although considerable efforts are being made to promote Web accessibility for users with disabilities, this is certainly not yet manifesting itself in improving Web accessibility and HTML validity.


Contents




Introduction

The significant benefits brought in our society by the Internet are well known. It reduces barriers of distance and time, and creates a society in which--in principle--anyone can have access to products and services all over the world at any time.

The people who could, arguably, benefit most dramatically from this are those who, because of some disability, have restricted access to information and services in the physical world, but they can have access to the online version of the desired services using dedicated assistive technologies (hardware or software which adapts a conventional system for use by a person with disability). For example, a blind user can "read"--using a Braille display or speech synthesizer--the online version of the daily edition of her favorite newspaper or her bank statement. A user with restricted mobility can visit virtual stores from the comfort of his home. A student with cognitive disability can take her own time in understanding the taught material (Brewer, 2001). Depending on the specific disability the Internet user has, the assistive technologies will differ from slow keys and on-screen keyboard to Braille displays and screen readers (Brewer, 2001).

The typical practice of many Web content developers is to test Web site functionality only against a small number of "popular" Web browser platforms in "normal" configurations. But, although the content might seem to be rendered correctly on such a platform, this does not guarantee that it is designed correctly. By definition, users of specialized assistive technologies are not using the "popular" platforms--or at least, not using them in the "normal" configurations; rather, they must depend on equipment tailored to their particular needs. As a result, it frequently happens that sites are poorly accessible, or completely inaccessible, to such users.

This situation would be quite different if sites were designed keeping in mind "write once, read everywhere", which can be achieved by designing Web content to meet appropriate guidelines and technical standards for inter-operability. Provided both the server side and the client side conform to such guidelines and standards, the client platform can be tailored to individual user needs, and still inter-operate effectively with all conforming servers.

An important source for accessible Web design resources is the W3C's Web Accessibility Initiative who, in May 1999, published the Web Content Accessibility Guidelines (WCAG 1.0) (W3C, 1999b), which are now a reference point in achieving Web accessibility in many of the E.U.'s Member States (European Commission, 2000a,b). In Ireland, The National Disability Authority have adopted WCAG 1.0 into the national "Guidelines for Web Accessibility" (NDA, 2001). WCAG 1.0 is also a source for the "Guidelines for UK Governmental Web sites"(OeE, 2002), published by the UK Cabinet Office in May 2002, the French "Government circular of 7 October 1999 concerning internet sites by state public establishments and services" (French Government, 1999) or the German "Barrierefreie Informationstechnik-Verordnung" (German Government, 2002).

In the summer of 2002 a detailed accessibility study (McMullin, 2002) of Web sites operated by Irish organizations was conducted in the eAccess lab of the Research Institute for Networks and Computer Engineering at Dublin City University, Ireland. The main objective of the study was primarily to inform and promote Web accessibility policy in Ireland.

Prompted by the efforts of promoting Web accessibility in other EU Member states, a similar but comparative study regarding the WCAG 1.0 conformance and HTML validity of a sample of Irish, UK, French and German Web sites was conducted in May 2003. This paper presents the techniques used, the key results, and an analysis of the most common WCAG 1.0 checkpoints failures and HTML mark-up defects encountered in this study.

Web Sampling Methodology

Country Specific Web Sampling

Selecting a representative sample of Web sites corresponding to a certain country is not a simple process. For the purpose of this survey it was considered that an open directory would provide a good basis. The information provided by Open Directory Project (ODP) is structured in a hierarchical tree of categories where each site is assigned to a specific category, representing the subject of the site as closely as possible.

The main ODP catalog is for US and foreign sites in English with descriptions also in English. The Regional branch in the ODP hierarchy contains sites that provide information about a specific region, and/or when the site is directly relevant to a population within a specific geographic area in English. For example, Regional/Europe/Germany would list sites about Germany. All sites containing non-English language content are listed under the respective language under World. For example, World/Deutsch/Regional/Europa/Deutschland would list German-language sites about Germany with descriptions in German.

Since the first official language in Ireland and the UK is English, the sites relevant to the population of the two countries were considered the ones with English language content, hence, the Web sites in the Regional/Europe/Ireland and Regional/Europe/United_Kingdom categories and their sub-categories. In the case of the French and the German Web samples, the sites relevant to the population of the two countries were considered to be the ones with French/German content, hence, the sites in the World/Français/Régional/Europe/France and World/Deutsch/Regional/Europa/Deutschland/ categories and their sub-categories.

The online resources for each category considered is shown in Table1, although it should be kept in mind that the ODP content is changing constantly so it is most likely that the content available at the date of sampling (Feb 2003) had changed.

Table 1: ODP categories considered in the sampling process
Category Code URL
Ireland IE http://dmoz.org/Regional/Europe/Ireland/
United Kingdom UK http://dmoz.org/Regional/Europe/United_Kingdom/
France FR http://dmoz.org/World/Fran%c3%a7ais/R%c3%a9gional/Europe/France/
Germany DE http://dmoz.org/World/Deutsch/Regional/Europa/Deutschland/

Considering the significant difference in the number of sites available in the four categories, it was considered that the best approach when deciding the number of sites in each sample would be to use a fixed fraction or percentage of the category total. On the basis of the communication, processing, and storage resources available for the study, this was set at 5percent. The exact figures corresponding to each country sample can be seen in Table2.

Table 2: Details on the ODP content considered in the sampling process
Country Web sites available in ODP Web sites in sample
IE 5,509 272
UK 114,044 5,702
FR 30,892 1,545
DE 84,860 4,250

These country samples were selected primarily from the following ODP sub-categories:


Web Content Retrieval

Web content for each site considered in the Web samples was retrieved and stored for conformance tests and further references.

Due to the fact that sites vary significantly in size and the type of media in which resources are offered, each site's content was also subject to sampling, using the Web content mirroring robot pavuk, on the following basis:

Table3 presents statistics regarding the mirroring process. It was considered that, in order for the survey's results to be reasonably representative for a site, each site content sample should have at least 3 Web pages and at least 100kB of data. Less than half of the number of Web sites considered in the four Web samples passed this test, qualifying for the conformance tests, with small differences between the four Web samples, percentages varying from 38percent (German sample) to 48percent (French sample).

Table 3: Details regarding the mirroring process
IE UK FR DE
Sites in sample 272 5,702 1,545 4,250
Total pages retrieved 3,319 67,598 22,319 58,278
Total data retrieved 28.61 MB 552 MB 166.86 MB 377.36 MB
Sites with no data (%) 3.3% 4.5% 6.9% 3.5%
Sites with only one page (%) 33.5% 29.3% 21.2% 27.3%
Sites qualified for tests 123 2380 747 1627
Sites qualified for tests (%) 45.2% 41.7% 48.4% 38.5%
Pages qualified for tests (%) 68.9% 67.8% 73.1% 65.3%

Although the four Web samples differ considerably regarding the number of sites considered, the statistics are remarkably similar. Still, some small differences can be noticed.

The French sample had the largest percentage (almost double compared to the Irish sample) of sites for which no data could be retrieved, showing that it was a problem connecting to the Web site (ODP data obsolete--Web server not found, network disruption, server downtime). But the situation changes regarding the percentage of sites for which only the main page ("home") was retrieved, with the highest percentage in the Irish Web sample. This could show that there were problems accessing Web pages that were linked from the main page (for example the main page links to other pages with links created using scripts--Multimedia Flash or Javascript).

The results obtained in the mirroring process can already be considered as crude "accessibility indicators" since the first step toward an accessible Web site is actually retrieving the content.

Web Accessibility Conformance


Methodology

W3C, 1999b

WCAG 1.0 consists of 14 separate guidelines, each of which has an associated set of one or more individual checkpoints. There are a total of 65 checkpoints which are classified into three priority levels (1-3)

Based on these priority levels, three levels of conformance to the WCAG 1.0 can be achieved:

There are a number of software products now available to carry out automated assessments against (subsets of) the WCAG 1.0 guidelines. These have a variety of strengths and weaknesses, but are functionally very similar (by definition, as they are largely driven by the WCAG 1.0 guidelines themselves). For the purposes of this study we chose to adopt one of the more widely deployed of these products, Bobby Worldwide (Core v4.0), originally developed by the Center for Applied Special Technology (CAST), and now distributed and maintained by Watchfire Corporation. (The use of bobby in this study does not imply an endorsement by the study's authors of this particular product or its producer or provider.)

bobby implements 91 distinct tests or diagnostics, each of which maps onto a specific WCAG 1.0 checkpoint. A number of bobby diagnostics map onto (different aspects of) the same WCAG 1.0 checkpoint.

The bobby diagnostics are classified into a number of different "support" categories, as follows:

For all categories other than Full, further evaluation would be required by a human assessor to determine WCAG 1.0 conformance. Accordingly, in the work presented here, bobby is restricted to implementing just those diagnostics with Full support. There are 25 such diagnostics, which map onto (aspects of) 20 distinct WCAG 1.0 checkpoints, including some at all three priority levels, as follows (in order of priority, then WCAG 1.0 checkpoint):

Table 4: Mapping the bobby diagnostics onto WCAG checkpoints
bobby ID Description WCAG Priority WCAG Checkpoint
g9 Provide alternative text for all images 1 1.1
g21 Provide alternative text for each APPLET 1 1.1
g20 Provide alternative content for each OBJECT 1 1.1
g10 Provide alternative text for all image-type buttons in forms 1 1.1
g240 Provide alternative text for all image map hot-spots (AREAs) 1 1.1
g38 Each FRAME must reference an HTML file 1 6.2
g39 Give each frame a title 1 12.1
g271 Use a public text identifier in a DOCTYPE statement. 2 3.2
g104 Use relative sizing and positioning (percent values) rather than absolute (pixels). 2 3.4
g2 Nest headings properly. 2 3.5
g37 Provide a NOFRAMES section when using FRAMEs. 2 6.5
g4 Avoid blinking text created with the BLINK element. 2 7.2
g5 Avoid scrolling text created with the MARQUEE element. 2 7.3
g33 Do not cause a page to refresh automatically. 2 7.4
g254 Do not cause a page to redirect to a new URL. 2 7.5
g269 Make sure event handlers do not require use of a mouse. 2 9.3
g41 Explicitly associate form controls and their labels with the LABEL element. 2 12.4
g34 Create link phrases that make sense when read out of context. 2 13.1
g265 Do not use the same link phrase more than once when the links point to different URLs. 2 13.1
g273 Include a document TITLE. 2 13.2
g14 Client-side image map contains a link not presented elsewhere on the page. 3 1.5
g125 Identify the language of the text. 3 4.3
g31 Provide a summary for tables. 3 5.5
g109 Include default, place-holding characters in edit boxes and text areas. 3 10.4
g35 Separate adjacent links with more than whitespace. 3 10.5
bobbyBobby Report Explanation File

Given that evaluation is limited to only a subset of the WCAG 1.0 guidelines, and is applied to only a sample of the content of any given site, it cannot determine that any site positively satisfies the guidelines; but failure on any of these tests definitively demonstrates failure against the guidelines.

bobby did not function properly for a number of Web sites. These failures were manifested in two ways:

Table 5: Statistics regarding bobby runs
IE UK FR DE
Sites considered 123 2380 747 1627
Sites with bobby abnormally terminated 2.4% 5.8% 8.4% 6.3%
Sites with bobby "locked up" 4.9% 3.9% 6% 4.7%
Sites with bobby successfully terminated 92.7% 90.4% 85.5% 88.9%

In general, when analyzing the results, it was considered that if one diagnostic is triggered in at least one Web page of a site sample, the site should be counted in statistics regarding the specific diagnostic. It may be argued that this could make the overall results look very strict; however only a relatively small Web content sample was taken from each site considered, and it is sensible to consider that the technologies/implementations used across a site are similar, so, if the reported defect appeared at least once it is quite probable that it will appear again over the overall site's content.


Key Results


Representative Priority 1 Defects

As explained above (section Web Accessibility Conformance: Methodology), all the WCAG 1.0 Priority 1 checkpoints must be satisfied in order to accommodate use of a Web site by any significant disability groups.

The incidence of Priority 1 defects encountered by bobby are listed in Table6. The most common of these are then discussed in more detail.

Table 6: Priority 1 Defects
ID Description IE UK FR DE Overall
g9 Provide alternative text for all images 93.0% 92.5% 97.9% 92.5% 93.3%
g39 Give each frame a title 21.9% 26.9% 41.2% 50.9% 36.8%
g38 Each FRAME must reference an HTML file 21.1% 25.5% 39.3% 50.4% 35.7%
g240 Provide alternative text for all image map hot-spots (AREAs) 21.9% 19.9% 37.7% 18.5% 22.1%
g10 Provide alternative text for all image-type buttons in forms 7.9% 8.5% 12.8% 8.9% 9.3%
g21 Provide alternative text for each APPLET 8.8% 7.4% 7.0% 7.5% 7.4%
g20 Provide alternative content for each OBJECT 0.0% 0.2% 0.5% 0.2% 0.3%

Provide alternative text

The Web is, as a whole, a "graphical" environment, which has to be translated into an alternative representation when used with non-graphical browsers/technologies (such as screen-readers, braille displays). This issue is specifically addressed in Web accessibility by the general recommendation of "providing alternative content".

HTML makes provision for attaching "alternative" or ALT text to images. This is generally hidden if the image is displayed or visible to the user; but it can be picked up and spoken by a speech synthesizer, or other alternative output device, for a blind or visually impaired user. The extra design effort involved is minimal, but dramatically improves the usability of the site for the affected users.

As it can be seen in Table6, the incidence of images used without providing alternative text is large in all samples, but even more noticeable in the French Web sample. Note that bobby can check only that some alternative text is provided with each image; it cannot check whether any provided alternative text is appropriate (that is, that it effectively represents, or is an appropriate functional alternative to, the image).


g39: Give each frame a title

Frames give rise to a wide variety of problems both for general usability and for accessibility by users with disabilities in particular. The functionality provided by frames can be achieved with alternative HTML technologies, properly engineered and with good browser support and accessibility. It is our view that frames should therefore now be regarded as an obsolete technology, and should be avoided wherever possible. However, as long as frames are still in use, then it is essential that the special accessibility issues which they raise are adequately addressed (Engelfriet, 1997).

The particular diagnostic discussed here addresses the need for each HTML frame element to have an associated, textual, title attribute, which will be displayed in browsers with no frames support. This serves to provide critical orientating information about the frame organization for users who cannot directly perceive the visual layout or configuration of frames.

The percentage of sites triggering this diagnostic is quite similar in the Irish and UK samples (around 25percent) and substantially higher in the French and the German Web samples (41.2percent and 50.9percent).

It might be assumed from this that frames are better implemented in the Irish and UK sites than the German and the French ones, but this is not necessarily true. It may also mean simply that the incidence of sites using frames is double in the French and the German Web samples than in the Irish or the UK samples. More refined testing and analysis would be necessary to distinguish these two cases.


g38: Each FRAME must reference an HTML file

When frames are pointing directly to content with intrinsic accessibility barriers (such as images), an alternative content cannot be provided. The Web content should, instead, be embedded in an HTML file so that appropriate alternative content can be included too. Again, the incidence of sites triggering this diagnostic is noticeably higher in the German and the French samples.


g21: Provide alternative text for each APPLET

The APPLET HTML element normally introduces Java applets--programs that are automatically downloaded and run on the client's machine. Applets raise potential problems for accessibility, especially when they implement navigation or when they add presentation to content--such as moving text and so on. An alternative content (that would implement the same functionality) will have to be provided, since it is needed in order for the same functionality to be rendered by a browser which is not capable of handling Java code, or when the user has disabled the browser's capability to interpret Java code.

The incidence of sites that triggered this diagnostic is not that high, although this may mainly reflect the raw incidence of applet use on the Web (as opposed to the relative proportion of sites using applets which specifically violate applet related accessibility guidelines). In any case, it should be kept in mind that even one applet wrongly implemented can create serious accessibility problems for the whole site since it might obstruct the navigation from the home page or hide key information from a user whose browser can't handle Java code.

Representative Priority 2 Defects

If a Web site satisfies not only all the WCAG 1.0 Priority 1 checkpoints but also the WCAG 1.0 Priority 2 checkpoints, its content should be accessible to a broad range of disability groups. The site can claim WCAG-AA level conformance.

The incidence of Priority 2 defects encountered by bobby are listed in Table7. The most common of these are then discussed in more detail.

Table 7: Priority 2 Defects
ID Description IE UK FR DE Overall
g104 Use relative sizing and positioning (percent values) rather than absolute (pixels) 98.3% 98.2% 98.1% 98.9% 98.4%
g271 Use a public text identifier in a DOCTYPE statement 87.7% 82.5% 94.2% 76.2% 82.2%
g265 Do not use the same link phrase more than once when the links point to different URLs 79.0% 70.4% 68.2% 71.0% 70.5%
g269 Make sure event handlers do not require use of a mouse 74.6% 65.3% 72.6% 63.1% 65.9%
g41 Explicitly associate form controls and their labels with the LABEL element 62.3% 63.6% 64.5% 57.7% 61.7%
g34 Create link phrases that make sense when read out of context 36.0% 39.1% 2.4% 1.2% 21.0%
g2 Nest headings properly 14.9% 13.9% 8.8% 10.6% 12.0%
g273 Include a document TITLE 15.8% 8.9% 10.6% 12.8% 10.6%
g5 Avoid scrolling text created with the MARQUEE element 7.9% 8.3% 10.8% 8.3% 8.6%
g37 Provide a NOFRAMES section when using FRAMEs 2.6% 2.1% 4.7% 15.3% 6.9%
g4 Avoid blinking text created with the BLINK element 1.8% 2.6% 5.8% 5.5% 4.0%
g254 Do not cause a page to redirect to a new URL 3.5% 2.9% 3.0% 5.4% 3.8%
g33 Do not cause a page to refresh automatically 0.9% 1.6% 1.4% 2.4% 1.8%


g104: Use relative sizing and positioning (percent values) rather than absolute (pixels)

The visual position and size of various elements can be specified in HTML--for example, font size for text, widths of tables, or individual table cells, etc. In general, HTML allows such positions and sizes to be specified in either "relative" units (can be scaled according to some norm which the user's browser already has) or "absolute" units (not scalable). The effect of using relative units is that the browser can very flexibly adjust the visual presentation according to the available visual space on the user's device, and the user's preferences and capabilities.

The overall incidence of this defect type in the Web samples studied is 98percent which is worrying since it is a defect that can be easily corrected with a significant potential for the large category of intermediate visually impaired users. The users with intermediate visual disability are not blind, but require some facilitation, particularly the use of larger font sizes. This is already a large category of disability; furthermore, as its incidence is significantly age-related, and the relative population of seniors is growing, its importance is, if anything, increasing (European Commission, 2001).

This defect type also illustrates the general concept of universal design--designing to include the widest possible variety of users. In the current case, by designing a site that uses only relative positioning and sizes, it can automatically adapt to changing user technologies and needs--such as the growing use of Internet enabled TVs, Personal Digital Assistants (PDAs), etc., which have a much wider variety of visual display capabilities than standard computer terminals.


g104: Use a public text identifier in a DOCTYPE statement

Properly formatted HTML pages should conform to a set of strict technical specifications, to ensure compatibility between Web sites and Web browsers (Zeldman, 2001). This is true as a general principle, but is especially crucial to ensuring compatibility with the wide variety of special purpose Web browsers and assistive technologies that are necessary to address the diverse needs of users with disabilities.

In order for a Web browser to render a HTML page's functionality and structure as intended by the author, it will need to know how the document is constructed, according to which technical standard. A "Document Type Definition" (DTD) specifies the elements (and their attributes) that can be used according to a particular technical standard, and what is their intended purpose and structure. The particular DTD according to which a Web page is constructed is specified using the DOCTYPE construction. When this information is mis-constructed or missing, the Web browser is challenged to properly render an unknown mark-up structure.

In most of the cases the assistive technology is an interface between a user with disability and the technology a user with no disability will use to browse the Web. So the behavior of such assistive technology will depend on the output given by a conventional Web browser. And if the conventional Web browser is challenged in rendering a Web content, the challenge will pass on to the assistive technology. In the end, the behavior is unpredictable and may be unusable.

The highest incidence of sites that triggered this bobby diagnostic is in the French Web sample (94percent) whilst the smallest one is in the German sample (76percent).

bobby thus gives an indication of the--remarkably high--incidence of sites that do not have any DOCTYPE information specified; however, if DOCTYPE information is specified, bobby cannot check whether it is properly constructed. Additional analysis is required to determine this. As will be shown below (section HTML Technical Standards Conformance: Representative Defects), when such further analysis is conducted, it is found that the incidence of Web sites that had an absent or mis-constructed DOCTYPE statement is significantly larger again.


g265: Do not use the same link phrase more than once when the links point to different URLs

A "link phrase" is the (usually short) fragment of text in a Web page that is hypertext linked to another Web resource. For users of visual browsers link phrases are normally visually highlighted in some way (perhaps by color, underlining, etc.); and "clicking" within the link phrase causes the browser to load the linked resource. Such users can generally scan Web pages visually very quickly to pick out link phrases; and can easily read the surrounding text if they need more context to understand a particular link phrase.

For users of non-visual browsers (say using screen readers, or braille output devices) "scanning" a Web page is generally a slower and more cumbersome process. One common technique to aid scanning in such cases is to simply skip from link to link; in this circumstance, only the link phrases are directly rendered to the user, and access to surrounding text (for additional context) will be relatively slow (i.e., it will undermine the very utility of this form of scanning).

This being the case, access for such users can be significantly improved if a little care is taken in the selection of link phrases; and, conversely, poor selection of link phrases can create a significant, and generally quite unnecessary, obstacle to users. More specifically, if the same link phrase is used multiple times, in the same page, but linking to different resources, this will not be apparent to a user who is scanning only such link phrases.


g269: Make sure event handlers do not require use of a mouse

Various HTML coding techniques (commonly using client side scripting) rely on certain kinds of interaction with the user. However, depending on their individual capabilities and preferences, users may adopt a wide variety of interaction devices. In particular, the use of a conventional mouse, or even of some adapted form of screen-pointing device, may be difficult or impossible for some users. Thus, if a page is coded in such a way that certain functionality or features can be accessed only by using a particular form of interface device--such as a mouse in the current case--then that functionality will be unavailable to users with certain disabilities; worse still, such users may not even be aware that such functionalities exist.


g41: Explicitly associate form controls and their labels with the LABEL element

In the visual presentation of a Web page there can often be important relationships between different components of the page which are expressed only implicitly by their juxtaposition in the display. A common example arises in the case of HTML based forms. A form generally consists of information explaining to the user what has to be filled in, interspersed with "form controls"--text entry boxes, radio buttons, drop down lists, etc.--which the user can interact with. Typically, the relative positions in the visual display make it reasonably easy for a visual user to identify which text is associated with which control.

However: for users who are unable to use a visual display in the manner assumed by the site designer (due to blindness, visual impairment, etc.) it will generally not be possible to directly perceive these implicit, but critical, relationships. To address this, HTML provides facilities whereby a particular form control can be explicitly marked as associated with a particular text (the corresponding "label"). This coding can then be used by a suitably configured browser to help a user with a disability to recognize the correct relationships. Furthermore, coding these explicit relationships can improve general form usability; for example, the browser can associate clicks on a label as intending to activate a form control, thus providing a larger target for selection with a pointer. This may be particularly helpful to users with motor impairment which limits fine pointer manipulation, but will generally be of benefit to all users. Thus, this again illustrates the applicability of universal design.


g34: Create link phrases that make sense when read out of context

When link phrases like "click here" or "more" are used, no information regarding the document referenced is provided to users of non-visual browsers, who, in some cases can "scan" through a list of links contained in a document. ("click here to do what?").

The results of the survey regarding this diagnostic are quite interesting. The figures show a high incidence of incorrect link phrases in the Irish sample (34percent) and the UK sample (39percent), whereas the incidence is considerably smaller in the German sample (1.2percent) and the French sample(2.4percent).

Does this mean that the sites in the French and German samples provide properly formed link phrases? Or is this difference due to a limitation in bobby's capability of "matching" only link phrases in English while the German and French equivalent of "click here" ("Cliquez ici" and "Click hier") and the likes are ignored? Ideally this could be resolved through the bobby documentation or examination of the source code. However, no clarification was found in the documentation, and, as bobby is not released in source form, source code inspection was not possible. Some controlled tests of this specific point were therefore carried out. The results were consistent with the second conjecture above: it appears that, indeed, in triggering this diagnostic, bobby only matches some pre-programmed repertoire of English-language phrases (even when correct information as to the natural language of the page is present).


g2: Nest headings properly

One of the core principles in designing accessible Web content is separating structure from presentation--in order to facilitate, as much as possible, adaptation of presentation to suit the particular needs and capabilities of individual users. That means that the HTML elements should be used always and only for their intended structural purpose, and not for assumed presentational "side-effects" (which are not consistent or reliable anyway).

Users of assistive technologies often use headings as a mean of navigation within a single Web page, since scanning the whole page content can be really difficult and especially time consuming (for example when using a non-visual browser). Thus, it is important not only that HTML heading elements are used (and not implied, for example via font effects), but also that they are nested properly, in order to make explicit the structure of the page content.


g273: Include a document TITLE

The TITLE element should provide a brief summary or indication of the page content. This information is usually displayed in the title bar of the window rendering the Web content, or otherwise made available on user request. It is also used in creating bookmarks. In general then, it provides important orientation information, which is useful to all users (universal design) but particularly useful to users with a variety of disabilities.


g5: Avoid scrolling text created with the MARQUEE element

The first problem with the MARQUEE element is that it is not recognized by any HTML technical standards, so a Web content using this element will fail the WCAG 1.0 Checkpoint 3.2 : "Create documents that validate to published formal grammars.". The importance of using technical standards conformant Web content is outlined below (section HTML Technical Standards Conformance).

The second problem is that the scrolling text effect created with MARQUEE can cause problem with people with different disabilities. There are some screen readers that can't handle scrolling text. People with cognitive disabilities can't comprehend the text at the speed of the scrolling, or the motion is distracting them from the actual text.


g37: Provide a NOFRAMES section when using FRAMES

The NOFRAMES HTML element introduces alternative content for frames, content to be rendered when the Web browser does not have support for frames. As noted earlier (section Web Accessibility Conformance: Representative Priority 1 Defects), the difference between the percentage of sites that triggered this diagnostic in the Irish and UK sample and the German and the French sample may be explained by a different proportion of sites actually using FRAMES. This proportion should be reflected in the number of sites that use the NOFRAMES element.


g4: Avoid blinking text created with the BLINK element

Again, the first problem with the BLINK element is that it is not recognized by any HTML technical standards.

The second problem is that the blinking text effect created with BLINK can cause mal-functions with screen readers--which can get stuck and repeat the same text on and on. Also, people with dyslexia can be affected by blinking text, people with low vision can have problems reading the text, people with cognitive disabilities can get distracted from the actual text by the blinking effect.


g254: Do not cause a page to redirect to a new URL

The actual WCAG 1.0 checkpoint referred by this diagnostic is "do not use markup to redirect pages automatically. Instead, configure the server to perform redirects". When the server is configured to handle a page redirection, the process is transparent to the user and causes no problems. But when the redirection is implemented on the client side--using mark-up, it can disorient the user. The user may be surprised and his screen reader interrupted, lose his place if he's using a screen magnifier and so on.


g33: Do not cause a page to refresh automatically

Content developers sometimes create pages that automatically refresh (change) periodically their content. The user should be allowed to decide the time he wants to spend browsing the content of a page. If the content is automatically refreshed to soon, the user might not have read/understand the content yet and in some cases he can get confused, not knowing that the content has changed.

Conclusions of the WCAG 1.0 Conformance Tests

The WCAG 1.0 conformance level of the Web sites studied is clearly alarmingly low.

The highest percentage of sites that failed the selected bobby tests at the minimum standard of accessibility was in the French sample at 98.6percent; the lowest percentage was in the Irish sample, but this was still at a level of 94.0percent, which can hardly be considered as substantially better. Note also that, as explained above (section Web Accessibility Conformance: Methodology), bobby can automatically report only certain limited checkpoint violations. Thus, it should not be inferred, even for the Irish sample, that the balance of sites (6.0percent) are necessarily conformant with WCAG-A. Rather it is likely that many, if not all, of these remaining sites may fail other WCAG priority 1 checkpoints that could not be automatically tested by bobby.

The frequency distribution of distinct bobby diagnostics, at WCAG Priority 1, is presented in Table8. This same data is presented in chart form in Figure1. Similarly, the frequency distribution of distinct bobby diagnostics, at WCAG Priority 2, is presented in Table9, and as a chart in Figure2. As will be seen from this data, most sites in the four samples triggered between 1 and 3 distinct WCAG 1.0 Priority 1 bobby diagnostics and between 4 and 6 distinct WCAG 1.0 Priority 2 bobby diagnostics. Thus, particularly at the WCAG-AA conformance level, the failures were generally not due to violation of any single isolated checkpoint, but to a pattern of violation of multiple checkpoints.

Table 8: Frequency of WCAG 1.0 Priority 1 bobby diagnostic violations
Number of distinct diagnostics IE (%) UK (%) FR (%) DE (%)
1 48.2 47.5 31.1 32.0
2 21.9 19.3 24.3 13.9
3 14.0 17.5 21.1 32.1
4 7.9 8.6 17.2 15.2
5 1.8 1.5 4.7 2.3
6 0.0 0.0 0.2 0.1
Figure 1: Frequency Chart of WCAG 1.0 Priority 1 bobby diagnostic violations
Bar chart showing numbers of sites having different ranges of distinct Priority 1 WCAG diagnostics. See preceding table for tabular version of data (follow long description link if available).
Table 9: Frequency of WCAG 1.0 Priority 2 bobby diagnostic violations
Number of distinct diagnostics IE (%) UK (%) FR (%) DE (%)
1 0.9 1.0 1.1 1.7
2 4.4 4.6 4.1 6.4
3 7.0 15.3 13.6 18.4
4 24.6 26.8 32.4 29.9
5 32.5 26.8 32.4 28.1
6 21.9 17.4 12.5 11.6
7 7.9 5.8 3.0 3.3
8 0.9 1.6 0.9 0.6
9 0.0 0.4 0.0 0.1
Figure 2: Frequency Chart of WCAG 1.0 Priority 2 bobby diagnostics
Bar chart showing numbers of sites having different ranges of distinct Priority 2 WCAG diagnostics. See preceding table for tabular version of data (follow long description link if available).


HTML Technical Standards Conformance

Create documents that validate to published formal grammars.

An HTML page is properly built when its mark-up conforms to a standard technical specification. Each standard is specified by a Document Type Definition (DTD) document which contains descriptions of the entities, elements and attributes that can be part of an HTML document, and how they can be interrelated. Because most of the existing Web browsers are able to render--to at least some extent--Web pages which don't conform to a DTD, many of the failures in the HTML code can pass unnoticed by most users. But such code defects can be a real impediment in access by users with disability helped by special purpose Web browsers and dedicated assistive technologies. They also complicate, and therefore inhibit, ongoing development of such niche technologies.

Methodology

There are different tools that can be used in order to validate HTML code against its description in the corresponding DTD. The output is usually a list of problems encountered (diagnostics) with suggestions as to how could they be fixed. A list of such tools can be seen on the WAI's Web site.

The tool chosen for the HTML conformance tests in this study is onsgmls. onsgmls is a parser and validator of SGML/XML files (HTML and XHTML), part of the OpenSP collection of SGML/XML processing tools.

onsgmls implements 438 individual diagnostics which can be triggered when an element or attribute in the HTML content is not used according to its specification in the HTML page's DTD.

Each diagnostic is assigned a "severity level" as follows:

The study presented in this paper will discuss only the onsgmls diagnostics with an "error" severity level, showing clear non conformance between the HTML content and its specification in the DTD.

As already explained, the validation process involves comparing the use of each component in an HTML document with its specification in the DTD of the HTML standard used in the document. Thus, in order to generate consistent validation results, onsgmls needs a properly specified DTD, normally identified via a DOCTYPE declaration in the beginning of the HTML page (W3C, 1999a, Section 7.2).

Accordingly, if the DOCTYPE declaration is missing or unrecognized, the Web page will immediately and automatically fail the validation test. It would be possible to configure onsgmls to assume a default DTD against which the document could be tested further in such cases. However, this approach was not adopted in the current study as it was judged that it would seriously confound or distort the results otherwise obtained (since we could not then distinguish between defects which were due to genuinely incorrect mark-up in a target document and apparent defects due only to a mis-match between the document and the arbitrarily selected default DTD).

In summary, it was found that over 82percent of all pages retrieved had missing or otherwise unusable document type information. Over 98percent of all sites considered had at least one such page. This is a striking result in itself; but it also makes it difficult to extract any further, more detailed information since, in the absence of a usable DOCTYPE declaration, no further validation is possible.

Recall that, in the analysis of the bobby (Web accessibility guidelines) conformance test results, it was considered that if a given diagnostic is triggered in at least one page of a site, the site should be counted in statistics regarding that diagnostic (see section Web Accessibility Conformance: Methodology). The same general approach was adopted in the analysis of the HTML technical standards conformance test results. However, given the extremely high proportion of sites having at least one page without a usable DOCTYPE, this was subject to further refinement. For the purposes of presenting and ranking the detailed HTML validation results the site samples were reduced by:

The effect of this is discussed further below, and summarized in Table10.

Key Results


Representative Defects

As already indicated, for the sample as a whole, by far the most common HTML mark-up defect was a missing or unusable DOCTYPE declaration, as summarized in Table10.

Table 10: Incidence of missing or unusable DOCTYPE declaration
IE UK FR DE
Pages processed by onsgmls 2,288 45,857 16,312 38,062
Pages with missing or unusable DOCTYPE 1,774 (77.5%) 35,529 (77.5%) 14,264 (87.4%) 26,950(70.8%)
Sites processed by onsgmls 123 2,380 747 1,636
Sites with missing or unusable DOCTYPE on at least one page 121 (98.4%) 2,345 (98.5%) 746 (99.9%) 1,584 (96.8%)
Sites considered for further analysis (having at least 3 pages/100Kb with usable DOCTYPE) 36 (29.3%) 663 (27.9%) 140 (18.7%) 603 (36.9%)

A correctly specified document type declaration is obviously crucial in the validation process of a Web page. But more importantly than this, when the document type information is correctly specified in an HTML document, the Web browser knows how the document is constructed and its content and functionality can therefore be rendered consistently and as intended. An HTML document without a usable DTD is a challenge to Web browsers to behave consistently or reliably because the mark-up structure is unpredictable.

More than that, starting with Internet Explorer 5.0 on Macintosh (released in March 2000) there is now a trend for the mainstream browsers to more strictly implement standards-conformant behavior. This means that the way that the Web content is rendered by the browser depends on the precise document type which is declared (using a correctly structured DOCTYPE declaration). For backwards compatibility a feature called "DOCTYPE switching" is sometimes implemented. In this case, even when a correctly structured DOCTYPE declaration is present in the Web page, the content may be rendered either according to the HTML standard specified--"standards mode"--or in a backwards compatibility way--"quirks mode"--in which the behavior is unpredictable for an arbitrary browser and/or operating system. More than that, the "DOCTYPE switch" feature is not guaranteed to be kept for long. It seems likely that future browser implementations will progressively drop it in favor of the "standards mode" only rendering behavior.

As mentioned before, in the absence of a usable DOCTYPE declaration, the validation results are not consistent or meaningful, and the Web pages that triggered this diagnostic were therefore removed from further, more detailed analysis. Given this, and to ensure that the results generated by a Web site are in some sense representative, each site was required to have to have at least 3 pages with a usable DOCTYPE declaration, including at least 100KByte of data, in order to qualify for more detailed evaluation. The number and the percentage of sites--from the ones considered by onsgmls in the first stage--are shown in Table10.

Due to the specific hierarchical structure of HTML documents, defects in mark-up will usually influence the validation process of elements within the element with incorrect mark-up. In the end, the diagnostics triggered by onsgmls during a validation process are closely inter-related, such that if one defect is repaired and the validation process repeated, the validation results can be significantly different.

The HTML conformance tests triggered 35 distinct diagnostics in the Irish sample, 48 distinct diagnostics in the UK sample, 45 distinct diagnostics in the French sample and 52 distinct diagnostics in the German sample. Although the distinct diagnostics triggered by the four samples therefore varies, the most common ten diagnostics (as measured by the proportion of sites triggering them at least once) are the same for all the four samples. Further, the ranking of these by relative site incidence varies with an average of just three places between the four samples, with the most common three diagnostics having the same rank ordering in all four samples. There is therefore very considerable underlying consistency in the pattern of defects across all the samples.

The ten most common mark-up defects are listed in Table11 (starting with the most frequent). Each of these will now be discussed in more detail.



Table 11: Representative mark-up defects. Note that this table is based on the number of sites triggering each diagnostic at least once. Percentages are relative to the reduced site samples (after filtering for usable DOCTYPE declarations, per Table10).
Diagnostic IE UK FR DE Overall
Undefined attribute 92% 91% 83% 91% 89%
Element not allowed by document type 81% 77% 72% 80% 76%
End tag for not opened element 75% 74% 60% 69% 69%
Required attribute not specified 53% 75% 73% 77% 69%
Missing required end tag 56% 53% 46% 54% 52%
Non SGML character number 56% 41% 47% 44% 47%
General entity not defined and no default entity 47% 49% 41% 45% 45%
Attribute value must be literal 44% 38% 39% 35% 39%
End tag for element not finished 31% 38% 37% 46% 38%
Attribute value not allowed 28% 39% 42% 36% 36%

Undefined attribute

In HTML each element can be described by specific attributes specified in the element's DTD declaration. The "undefined attribute" diagnostic is triggered when, during the validation, an element appears as being described by an attribute that is not specified in the element's DTD declaration. Some situations in which this diagnostic can be triggered are:

Element not allowed by the document type

An element is used in the HTML document within another element, but this should not contain it according to the DTD. These defects are usually due to a misconstruction of nested elements, for example a list item element (LI) used directly within a paragraph element (P) when it should only be used directly within a list element (OL or UL).

End tag for element not opened

The content of an HTML element is delimited by a "start tag" and an "end tag" and the way that the elements can nest is described in the DTD. If onsgmls encounters improperly nested elements, it considers that the outer element is implicitly closed before the start tag of the inner element and it triggers a "missing end tag" diagnostic if the end tag is required in the structure of the outer element. Later on in the content of the document, when the original end tag of the outer element is encountered, onsgmls considers it as belonging to no opened element.

Required attribute not specified

This signals that the DTD declares that a certain attribute is required on some element, but it is not present. Some specific features addressing Web accessibility for users with disability are implemented with the help of compulsory attributes, providing details on the content of an element when the element can't be rendered effectively or directly for any particular user. Thus, this defect is particularly likely to be directly correlated with accessibility problems.

Whilst this diagnostic's incidence is quite similar in the UK, French and German Web samples (around 75percent), it is considerably smaller in the Irish Web sample at 53percent.

Missing required end tag

This diagnostic is triggered when the DTD declaration specifies that the end tag is required for a particular element, but it is missing from the HTML code in the Web page content. Some situations in which this diagnostic can be triggered are:

This diagnostic's incidence is quite similar in the UK, Irish and German samples (around 54percent of the sites considered for further analysis) and smaller in the French Web sample (46percent ).

Non SGML character number

In general, authoring tools may encode HTML documents in the character encoding of their choice, providing the encoding is correctly labelled. The information about the character encoding of a document can be specified via HTTP headers and/or using an embedded META http-equiv element. However, due to the implementation characteristics of the system used in this survey, character encodings specified via HTTP headers are lost after the mirroring process discussed above (section Web Sampling Methodology: Web Content Retrieval).

This diagnostic appears to be commonly triggered when the authoring tool which generated the Web page uses a character encoding of its own choice but the information regarding the character encoding is not made available in the HTML content. As a result, in order to validate the document, onsgmls will have to use heuristics to determine the character encoding. Although this practice is standards conforming (W3C, 1999a, Section 5.2.2), its results are necessarily unpredictable.

From the point of view of practical browsing, if the character encoding is not reliably communicated then the rendering of specific characters will be unpredictable, and may be incorrect. This would potentially slow down and confuse comprehension. This can impact all users, but may have a disproportionate effect on particular disability groups. For maximum accessibility it is recommended that an open standard character encoding be used (such as ASCII, UTF-8, ISO-Latin-9 etc.) and that this should be explicitly declared via both HTTP headers and an embedded META http-equiv element.

General entity not defined and no default entity

A given character encoding (method of converting a sequence of bytes into a sequence of characters) may not be able to express all characters of the document character set (a set of abstract characters used in a document and their integer references) (W3C, 1999a, Section 5). For such encodings, or when hardware or software configurations do not allow users to input some document characters directly, authors may use SGML character references.

The & (ampersand) character is a special character in SGML, marking the start of a character reference. When the & is used as such in an HTML document as part of a text (e.g., "Smith&Sons") onsgmls will look in the document character set for the abstract character represented by the "&Sons" character reference and, of course, it won't find it so it will trigger the above diagnostic. This should be avoided by "escaping" literal use of the & character--that is, by using the character reference & everywhere in the HTML document where the & character is not intended as the start of a character reference.

The issue of & character used as such instead of its character reference & arises especially in URLs. When the Web content is generated dynamically (using a server-side application), the arguments based on which the Web page is determined may be embedded in URLs, separated from each other with & characters. Links to such URLs should escape the & character as discussed before.

Attribute value must be literal

By default, SGML requires that all attribute values be delimited using either double quotation marks (ASCII decimal 34) or single quotation marks (ASCII decimal 39). Still, some "relaxed" standards allow attribute values not included in quotation marks, provided that they follow some rules.

This diagnostic is triggered when the value of an attribute is not included between "double quotation marks" although it should be because the specific use wouldn't qualify for the exception rules (e.g. : <table width=90%> instead of <table width="90%">)

End tag for element not finished

This diagnostic is triggered when an end tag is found for an element, when one or more of its inner elements are still open. In most cases this appears to be an error propagated from other HTML defects already detected by onsgmls, as discussed previously (section HTML Technical Standards Conformance: Representative Defects).

Attribute value not allowed

This diagnostic is triggered usually when the value of an attribute is not one of the values permitted for that attribute in the specified DTD.

Conclusions of the HTML Technical Standards Conformance Tests

The level of conformance to HTML technical standards of the Web content considered for tests is alarmingly low (almost nonexistent) in all the four Web samples.

Most of the HTML defects on the sites studied are probably not apparent to the majority of Web users--because, presumably, the developers have specifically tested them against some (small) selection of "popular" browser platforms. However, because they do not conform to technical standards for inter-operability, their rendering is--at best--unpredictable. This is likely to have a disproportionate affect on users who rely on specialized, tailored, client technologies--specifically, users with disabilities. Content may thus fail to be rendered, may be garbled, or may be otherwise inaccessible to such users. Worse, precious development effort in individualizing assistive technologies may have to be spent on attempting to compensate for these server side defects, rather than improving the client side functionality that the user really needs. In the worst case, this effort may have to be wasted repeatedly for each different client accessing each different (non-conformant) server. Whereas, conformance to technical inter-operability would substantially reduce or eliminate this waste.

On the sites that had usable document type information, the lowest number of distinct HTML standard violations was triggered by the sites in the Irish Web sample (35) and the highest one in the German Web sample (52). The sites in the French sample triggered 45 distinct HTML standard violations while the sites in the UK Web sample triggered 48 distinct HTML standard violations. Although the number of distinct diagnostics triggered by the four samples varies, the most common 10 diagnostics are the same for all the four samples, and their ranking by relative incidence varies with an average of just 3 places, the most common three diagnostics being the same.

The frequency of distinct HTML standard violations detected by onsgmls is shown in Table12. This same data is presented in chart form in Figure3. In this data the percentage is calculated in relation to the total number of sites that had usable document type information as shown in Table10.

Table 12: Frequency of HTML standard violations detected by onsgmls on Web content with usable document type information
Number of distinct diagnostics IE (%) UK (%) FR (%) DE (%)
0 2.8 4.4 11.4 4.8
1-5 19.4 19.6 17.9 13.9
6-10 36.1 39.1 35.7 43.8
11-15 36.1 31.1 26.4 28.5
16-20 2.8 5.7 8.6 8.6
21-25 2.8 0.2 0.0 0.3
Figure 3: Frequency Chart of HTML standard violations detected by onsgmls on Web content with usable document type information
Bar chart showing numbers of sites having different ranges of distinct onsgmls diagnostics. See preceding table for tabular version of data (follow long description link if available).

Most sites in the four samples had up to 15 distinct HTML standard violations detected by onsgmls. But the majority of these defects could be repaired relatively easily. Although it might seem that it would involve a considerable amount of work, many of the detected defects are inter-related--so that correcting one substantive HTML code defect could eliminate a number of reported diagnostics.

It would be recommended that Web content authors should become familiar with the HTML standard technical specifications and integrate in the publishing Quality Assurance service one of the existing HTML validation tools in order to ensure that Web content is valid before making it public online. These tools not only list the defects encountered but also give references as to how these defects might be repaired. One such tool is the W3C's online HTML Validator which is easy to use, doesn't require any local installations and can validate either online or uploaded Web content.

Of course, may Web page authors use HTML authoring tools or content management systems, VLE/MLE etc. In such cases the HTML mark-up may be hidden from the author. If the HTML mark-up is found to be invalid, the authors can get easily discouraged from providing valid HTML code: they may have no knowledge or understanding of the HTML specification, hence they don't have the knowledge to repair the HTML code (even if the authoring tool allowed such intervention--which it commonly does not!). In this case it is strongly recommended that the authors raise the accessibility issue with the Web content authoring tool developer or vendor, maybe even providing the validation results.

Discussion

This section provides a brief critical review of the study, and of the tools and methodology adopted.

First, the project has demonstrated that this sort of largely automated survey of selected accessibility indicators is technically feasible; and that once the appropriate tools have been developed and integrated, the technical resources to carry out such a survey are comparatively modest. This contrasts with surveying approaches that rely on significant manual intervention.

However, the project has also identified a number of limitations and/or open issues, many of which will require further research.

Conclusions

Although the number of sites in each sample was very different, the results of the survey were remarkably similar.

We conjecture that the similarity regarding the HTML technical standard conformance may be largely due to defects in common Web authoring tools or content management systems. However, the poor conformance to Web accessibility guidelines is presumably due to lack of information or misunderstanding of their importance on the part of content designers and authors. It seems that the "write once, read everywhere" concept is still quite far from reality, even though significant efforts in promoting Web accessibility are being invested in the studied countries.

The results of this survey do not represent the "exact" level of Web accessibility on the Irish, UK, French or German Web sites, but they demonstrate a widespread lack of concern with accessibility guidelines and technical inter-operability. Not only that, but, as mentioned in the Introduction the results of this survey are similar to the results of an earlier survey of a sample of Irish sites, using similar technologies (McMullin, 2002). While it may be argued that the results are still generated based on sample of sites, the fact that samples with such different number of sites generated essentially the same results is suggestive that this situation is probably typical of the Web as a whole in these countries. This is disappointing because it decreases considerably the potential that the Web could offer for significant improvement in service and opportunity for users with disabilities.

This study signals that, despite very laudable goals in documents such as the E-Europe Action Plan (European Commission, 2001), the current commitment to accessibility of the Irish, UK, French and German Web for users with disabilities is, at best, aspirational--and, at worst, cynically inadequate.

This is doubly unfortunate. It is not just that Web technology is not being applied--as it could be--positively to improve opportunities and capabilities for users with disabilities; but on the contrary, as Web services become more pervasive and essential, to the extent that they remain inaccessible this will actually impose progressively more disadvantage and exclusion on groups with disabilities in our society.

It is hoped that the results of this study will serve to highlight these issues, and to further encourage the many agencies and organizations who are already actively promoting and supporting voluntary improvements in Web accessibility around EU. Ultimately however, there must surely also be a role for compulsion--legislation and regulation--to fully guarantee and vindicate the rights of all citizens to equal treatment in a digital democracy.

Acknowledgments

The work described here received financial support provided from AIB PLC.. The work was carried out in the Research Institute for Networks and Communications Engineering (RINCE), established at DCU under the Programme for Research in Third Level Institutions operated by the Irish Higher Education Authority.

About the Authors

Ms.Carmen Marincu is currently completing her Ph.D. research studies at the eAccessibility Lab of the Research Centre for Networks and Communications Engineering (RINCE) at Dublin City University (DCU).

Dr.Barry McMullin is a senior lecturer in the School of Electronic Engineering of Dublin City University (DCU), and directs the eAccessibility Lab.

Correspondence Email: Barry.McMullin@rince.ie

Bibliography

Brewer, J. (2001),
"How People with Disabilities Use the Web", World Wide Web consortium (W3C), at http://www.w3.org/WAI/EO/Drafts/PWD-Use-Web/, accessed 2003-10-15.
Engelfriet, A. (1997),
"Using Frames and Accessible Web Sites", Web Design Group, at http://www.htmlhelp.com/design/frames/, accessed 2003-09-18.
European Commission (2000a),
"eEurope : An Information Society For All", European Commission, at http://europa.eu.int/information_society/eeurope/2002/news_library/pdf_files/initiative_en.pdf, accessed 2002-10-21.
European Commission (2000b),
"eEurope Targets 2001/2002", European Commission, at http://europa.eu.int/information_society/eeurope/2002/action_plan/eaccess/eu/targets_2001_2002/index_en.htm, accessed 2004-07-16.
European Commission (2001),
"eEurope 2002: Accessibility of Public Web Sites and their Content: Communication from the Commission to the Council, the European Parliament, the Economic and Social Committee, and the Committee of Regions", Commission of the European Communities, at http://europa.eu.int/information_society/eeurope/2002/news_library/pdf_files/communication_accessibility_en.doc, accessed 2003-09-22.
French Government (1999),
"Circulaire du 7 octobre 1999 relative aux sites internet des services et des etablissements publics de l'Etat", L'Action de l'Etat et societe de l'information, at http://www.admi.net/jo/19991012/PRMX9903708C.html, accessed 2004-07-16.
German Government (2002),
"Barrierefreie Informationstechnik-Verordnung", DE, at http://www.webforall-heidelberg.de/html/deutsch/bitv.php, accessed 2003-11-12.
McMullin, B. (2002),
"Users with Disability Need Not Apply? Web Accessibility in Ireland", at http://www.firstmonday.dk/issues/issue7_12/mcmullin/index.html, accessed 2003-11-12.
NDA (2001),
"Guidelines for Web Accessibility", Irish National Disability Authority (NDA), at http://accessit.nda.ie/, accessed 2003-07-07.
OeE (2002),
"Guidelines for UK Governmental web sites", Cabinet Office - Office of the e-Envoy, at http://e-government.cabinetoffice.gov.uk/Resources/WebGuidelines/fs/en, accessed 2003-11-12.
W3C (1999a),
"HTML 4.01 Specification", World Wide Web consortium (W3C), at http://www.w3.org/TR/html401/, accessed 2002-10-21.
W3C (1999b),
"Web Content Accessibility Guidelines 1.0 (WCAG 1.0)", World Wide Web consortium (W3C), at http://www.w3.org/TR/WCAG10/, accessed 2002-07-08.
Zeldman, J. (2001),
"To Hell With Bad Browsers!", A List Apart, number 99, at http://www.alistapart.com/articles/tohell/, accessed 2003-09-18.

Copyright

Final version published in First Monday, volume 9, number 7 (July 2004), under First Monday Copyright Conditions.

Page Administrative Information

Maintainer: eaccess@rince.ie