These days I’m the Library Web Developer at CNIB, but this role follows on from a decade or more in academic research based on design for accessibility.
This blog post is designed to give an introduction to how I became involved in accessibility, how that coloured my thinking, and the research strategy I then followed.
In 2003, a relative with Multiple Sclerosis (MS), a progressive disease of the central nervous system, asked for help in choosing a mobile phone suitable for his disability. The evaluation of existing mobile phones that ensued became a master’s thesis investigating the accessibility issues surrounding hand-held devices, which in turn became the starting point for this research.
Selecting a device for someone with Multiple Sclerosis required a detailed comparison of the usability of a range of possible phones for that user, culminating in a ranking from “best accessible” to “least accessible”, which in turn implies that whichever the chosen definition of accessibility, accessibility must be measurable, and to some extent quantitative. It must also, in some way, be related to the characteristics of users, and how users interact with the device. How measurable and how quantitative such a definition can be, and the required fidelity of any associated user model, eventually became the central issues of the research.
The approach taken to construct an effective user interface for a given individual in order is one of abstraction followed by guided reification to match user need. The meaning of reification in this context is to make concrete that which is abstract; a photograph for example is a reification of a place and time, with all of the implied lens distortion and selectivity in choosing the point of view of the image.
In the case of my research, users, devices, context, and content are all considered in very abstract terms, as are potential means of interaction with the user. This abstract view of interaction is then used to support the concept of dynamic self-adaptation of presentation of content by user interfaces, meeting user need by adjusting interaction modalities to match user capability.
The existing, practical, model for this approach was an adaptation of the Shlaer-Mellor Object Oriented Analysis and Design method, Shlaer and Mellor (1988 and 1992), used on commercial software projects during the 1990s.
The Shlaer-Mellor method makes a distinction between the process of analysis, and that of design and implementation, best described as implementation through translation, as opposed to the classical approach of iterative elaboration. In practical terms, this means that a computer program can be automatically generated (reified) from a generic, almost context independent abstract model of information and behaviour, using a rule-based translation of the analysis. In this approach, the rules describe an optimal implementation for the given device and operating context through use of “archetypes”.
Describing self-adaptation of a user interface to support a specific user, device, and context is a natural extension of the Shlaer-Mellor method, and personal experience in the use of the method during the 1990s heavily influenced the approach taken to expressing users, devices, context, and content.
As the research progressed, each of the problem domains of users, devices, context, and content was considered in terms of a Shlaer-Mellor style translation, with some small-scale implementations to help ground the abstract models produced in practical realizations.
My research had, indeed has, a working thesis:
The word “entity” was chosen because of the number of different contexts in which a person interacts with their environment, both physically and intellectually. The aim was to define accessibility in sufficiently general terms using “entity” that it could, in principle, embrace all such contexts. In terms of this research however, it was the interaction between people and computer software that was of sole concern.
Each hypothesis is numbered for reference and is followed by a short description.
This forms the central hypothesis of my research, and follows from an assumption that the ontology of any given problem domain can be expressed in terms of the associations between the semantic terms of that domain, an approach that is taken in systems modelling using Object Oriented Analysis. Further, since in this case it is the encounter between problem domains that is of interest, the encounter itself must be a problem domain capable of expression through association relationships between at least some of the terms in the two related domains, a “bridge” in terms of the Shlaer-Mellor Object Oriented Analysis method.
When constructing a user interface, it is not the general, but the specific case of an individual that is of interest. Similarly, when selecting from a range of products, it is not the general case, but the specific case of the product’s capacity to support the given user that is of interest. The general solution to this in computing is user and device profiling, albeit using narrowly defined, context specific solutions. Such profiles can be found in specific populations of the relational models described in 1.4.1. As such, profiling is entirely an artifact of the model of the encounter between user and entity.
This hypothesis addresses the validity of the question, “Is it more accessible?” It is based on the assumption that having expressed accessibility in terms of a framework of relational models, and that specific user and device profiles can be described by populating the frameworks, it is then possible to compare the results of the encounter between capability and capacity. Such a comparison relies on the idea of relational models being expressible as directed graphs so that entailment of the graphs can be used to rank accessibility.
1.5 Research Questions
In addition to the general approach described in Section 1.2, the approach to the day-to-day progress of the research was to ask questions, often with limited expectation of being able to fully answer them; the interest in many cases was the implications of such questions for the research, rather than their answers per se. Chronologically, the questions approximate to:
a) Is it accessible?
b) Is it more accessible?
c) Who can use it?
d) How can we interact with it?
e) What are we interacting with?
f) How do we interact with it?
g) Is it more accessible?
As is clear from (b) and (g), the same questions appeared a number of times within the research as new subject areas were addressed, and previous research reviewed.
1.6.1 Is it accessible?
“Is it accessible?” is the perennial mantra of accessibility research. We ask the question of the built environment, of public transport, of goods and services, and of computer systems and their programs, amongst others. With such a wide range of possible subject areas to address, even defining what we mean by the word “accessibility” can be a challenge, and one that is typically resolved through narrow definition. For example, international standard ISO/IEC 24751, ISO (2008), defines accessibility to be,
“The ability of the learning environment to adjust to the needs of all learners”
It is sometimes said that we are the sum of our experiences, and that is certainly true of the approach taken to this research, which builds upon 14 years personal experience in the real-time software industry, mostly within the defence and communications sectors; the most recent experience was with creating user interface design methods at Nokia Inc. Significantly, the accumulated experience also included time as a quality assurance engineer, auditing large software development projects, and consequently, the world-view brought to the research is one of systems, processes, and workflows.
It was this world-view that, in 2003, informed my work on the question, “Is a mobile phone accessible to users with Multiple Sclerosis (MS)?” The conclusion reached was that whilst none of the tested phones were fully accessible, in principle they could be made so for certain operating contexts. Constraints included:
- The symptoms experienced by the user.
MS has a range of symptoms and related physical impairments, but not all sufferers experience all of the symptoms, and those they do experience varies over time, sometimes even over a single day. This in itself challenges any notion of a single static user profile used to evaluate accessibility. That is to say, there are multiple population sets for each user, given the relational models described in section 1.4.
- The severity of the symptoms experienced by the user.
Again severity varies over time so that, for example, the accuracy of touching a point on a surface may vary, even for a single user in a single day.
- Where and when the phone is being used.
Experience showed that disabled users tended to evolve coping strategies for their disability, and one of my test users solved some of his mobility problems (he experienced severe hand tremors) by jamming his phone into the side of his wheelchair in order to stabilize the device whilst he pushed buttons, or opened folding phones. Consequently results varied depending on the chair he sat in for experiments.
- Which features of the phone are considered for evaluation.
Is a phone accessible if users cannot themselves connect the phone to its charger, or turn it off and on? Is the phone accessible if the user cannot play games that are pre-installed on the phone?
- The choice of interaction modalities offered by the phone.
One test user had no sense of touch and found pressing keys reliably very difficult, especially when multiple letters of the alphabet were assigned to the same physical key, which was the case with the majority mobile phones in 2003 when the initial research was done. However, one of the phones evaluated, the Nokia 6108, designed primarily for Chinese and Japanese input, allowed for character-by-character handwriting recognition (using Latin characters) and thanks to the quality of the audio feedback provided by the phone, the test user was able to enter text messages reliably if rather slowly.
1.6.2 Can it be made more accessible?
The main research 2004, immediately after successful completion of the master’s thesis, with the initial intention of looking to widen that research beyond MS, and to consider some potential improvements to mobile phones in order to support a more inclusive user base. To that end, existing research into user interface design, which explicitly supported disabled users, was reviewed together with existing work on novel user interfaces for both wearable computing and low-attention user interfaces. In reviewing this work, it was the ways in which content was adapted to match user need for differing operating contexts, with the contextual issues encountered during the master’s work that was to the fore.
Playing into the investigation of existing user interface research was a professional background in real-time software development and user interface design in product areas such as in-car infotainment systems where multiple applications compete for a user’s limited attention; this experience led to a favouring of solutions based on abstract user interfaces, and on transcoding between visual, sonic and haptic design spaces.
Three questions arose when considering the inspected solutions:
- What, exactly, is being solved?
- How effective is chosen solution?
- How does the chosen solution compare with the alternatives?
The desire to create a generally more accessible mobile phone, rather than solving problems for a specific disability or impairment, required that the research step back from detailed solutions in order to consider a more abstract model of accessibility. The target of the research then became the search for accessibility frameworks capable of answering these questions given the concrete solutions considered so far, with the specific goal of being able to answer the question, “Is it more accessible?”
1.6.3 A framework for accessibility
Four related problem domains were identified as being associated with such a framework:
- A description of the user base.
- A description of the device or program under consideration.
- A description of the operating context of that device or program.
- A description of the content being accessed by the users.
One software analysis method for dealing with these kind of related problem domains is the semantic decomposition found in the Shlaer-Mellor Object Oriented Analysis and Design (OOA/OOD) method (Shlaer and Mellor, 1988 and 1992), where individual problem domains are considered and modelled in isolation with the relationships between the domains modelled separately, a technique known as “bridging”. Whilst the technique is intended to decompose complex problem domains into a hierarchy of simpler problem domains, this research re-uses the concept to express an accessibility framework based on the four problem domains identified in the list above. The motivation to do so came from personal commercial experience with Shlaer-Mellor, where the method was used to express abstract models of telephone switching and routing, to express models of remote configuration management in trunk telephony, and to express models of aircraft and crew rosters.
1.6.4 The user base
The first of the problem domains to be considered in detail was the user base. Beginning with Nesbitt’s work on the multi-sensory design space, Nesbitt (2001), individual problem domains were considered and modelled based upon visual, sonic, and haptic user capability. Models of capability rather than impairment were chosen since a user interface is constructed based on what a use can do, rather than what they cannot. The models themselves were expressed as Shlaer-Mellor Information Models. The fidelity of the models on accessibility was considered, as was how such models may be populated for a specific user.
1.6.5 Devices and their use
Device capability and the user’s operating context were then similarly considered and modelled with the intention of expressing a user profile in terms of specific devices and operating contexts. Device capability was expressed in terms of the visual, sonic, and haptic design space, identifying requirements upon a user’s capability to interact with the device for a given operating context. Accessibility at this stage of the research was considered to be the matching of a user’s capabilities to device capability; the better the match, the more accessible the device. There was also the expectation that comparison between any two devices for the same user and the same operating context would be possible, answering the question, “Which is more accessible?”
1.6.6 Interaction modalities and content
Matching user capability and device capability does not in itself say whether content on a device is accessible to the user; a device is only as useful as the interaction modalities that it supports. If those modalities are inappropriate to express the given content, then the content will still be inaccessible. For example, expressing a short mobile phone text message is quite different to expressing tabular content. Not only does tabular content require greater cognitive capability by the user than plain text, it also requires more sophisticated interaction modalities to support presentation and interaction. To resolve this issue of appropriateness of modalities, the relationship between interaction modality and capability was considered, as was the mapping between content and modality.
Of the many possible models of expressed content, this research focused on the Dexter Model of hypertext as described by Halasz and Schwartz (1994), and the Amsterdam Model of hypermedia as described by Hardman, Bulterman, and Rossum (1994). These models were chosen because of the ubiquity of Web content in modern life. It also allowed for a comparison between the prevailing model of Web accessibility given by the World Wide Web Consortium’s (2008) WCAG standards and the accessibility framework under development as part of my research. An analysis of the Dexter and Amsterdam models resulted in a re-working to allow for better expression of alternative content, and of dynamic augmentation and adaptation of content. The re-worked hypermedia model was then placed within a workflow to examine how the accessibility framework may influence development of accessible content, and rendering of content based on populated user profiles.
1.6.7 A formal model of accessibility
The original research from the early 2000s completed with a proposed semi-formal re-definition of accessibility based upon the developed framework, and reflection upon a future based on an even more formal definition of accessibility, capable of describing accessibility in a purely quantitative manner based on graph theory, game theory, and intelligent agents.