by Erik Pukinskis
Theories have poured into HCI from many disparate fields, including cognitive psychology, artificial intelligence, ergonomics, sociology, anthropology, graphic design, and computer science. When it comes to theories devised within the field of HCI itself, researchers have created everything but consensus. Many “theories” simply take the form of goals: usability, calmness, fun, etc. Others take the form of methods: ethnography, cooperative design, etc. Still others offer predictive models of interaction: GOMS, Fitt’s Law, etc. This cacophony of independent approaches can be fatiguing for HCI practitioners and researchers. A history of failed attempts at applying these theories and a perceived lack of consensus or cooperation has created an atmosphere of distrust and independence in the HCI community.
This essay is titled with the first two lines of Thomas Erickson’s poem, “Theory Theory: A Designer’s View.” The poem exposes our conflicting images of theory: is it a robust mixture or an incomprehensible mess? Is it a happy club or an elitist clique? Erickson asks what, if any, is the role of theory in design and then begs the question by suggesting that theory is to be used when it is useful. This essay will attempt to explain the difficulties Erickson highlights, and offer a more thorough analysis of the relationship between HCI theory and practice.
The following pages will look at a number of applications of HCI theory and attempt to understand the difficulties encountered and how such difficulties are a reflection of the theories being used.
GOMS is a fairly simplistic model for predicting the time cost of a given task by breaking it down into basic interactions (key presses, mouse moves, etc.) It has been used to successfully evaluate interfaces in text-editing, VLSI layout, graphical editing, spreadsheets, computer command abbreviations, high-functionality oscilloscopes, and video games. (Myers, 1994). The most often cited application of GOMS, however, is a study by Attwood et al (1996) which evaluated workstations for telephone operators and made recommendations based on their speed, as assessed by GOMS.
Unfortunately, GOMS is only really useful in analyzing behavior the user has already mastered. (John 1995) Additionally, the goals of HCI are always changing—speed and usability were once paramount, but researchers have suggested calm computing (Galloway, 2001) and fun (Carroll, 2004) as alternative goals, to name just two. Speed seems at best indirectly related to these and at worst orthogonal to them. Because it is a purely predictive, rather than descriptive or generative theory, GOMS has a difficult time maintaining relevance as these goals shift.
The distributed cognition approach occupies a very different position in HCI. Distributed cognition is an outgrowth of the cognitive psychology movement, which portrays the mind as a symbol processor. The distributed cognition approach suggests that this symbol processing occurs not only in the brain, but is shared with other people and the environment they share. Distributed cognition is often attributed to Edwin Hutchins (1995) and has been applied to HCI by him and other researchers at UCSD. Kirsch (2001) suggests that by using distributed cognition concepts, it is possible to do a deeper analysis of a work environment—one which gets at the structure of the environment which is invariant across many office settings. Hutchins himself has had success using distributed cognition as a theoretical paradigm and a methodology for informing the design of airplane cockpits. (Hutchins and Klausen, 1996) However, Hutchins is a founder of the distributed cognition movement, and Kirsch is an “insider.” The fact that they can successfully apply it is not an endorsement of its portability.
There are, however, examples of distributed cognition being used by people outside the “inner circle” to inform design. Busby and Hughes (2003) applied the theory to failures described in incident reports from the UK Health and Safety Executive and found many examples of failures which were not due to execution or planning on the part of the individual, but due to distribution issues alone. This suggests that distributed cognition is simple enough to be applied, despite its abstract nature.
Rogers (2004) describes attempts in the early 1980s to use paired-associate learning to discover the most effective possible command names for a word processor. As has been the case in many early applications of psychological theory, the results of these studies were conflicting, suggesting a variety of factors which might affect the learnability of command names.
The difficulty which such blind application of theories from outside HCI is that those theories are derived from one set of behavior (generally observed in a laboratory) and they are applied to a completely different sample of behavior in HCI (in a more naturalistic setting). We have no guarantee that such generalization is permissible. Additionally, in this case there is no consensus that the specific findings or conclusions of the paired-associate learning studies were accurate to begin with.
Even today, the most widely accepted psychological theories are rejected by a portion of the theoretical psychology community. Before cognitive psychology—arguably today’s dominant theory of mind—achieved current degree of prominence, J. J. Gibson (1966) was laying the foundation for ecological psychology, which rejects many of the core assertions of the cognitive movement, most notably the idea that the individual is a self-contained information processor and the idea that cognition is symbol processing. Cognitive psychology has subsequently bent to attempt to account for more ecological phenomena, most notably in the distributed cognition approach, (Hutchins and Norman, 1988) but disagreement remains about the presence of symbols and representations in cognition. One possible explanation, therefore, of the difficulty practitioners have had applying cognitive psychology to HCI is that it may not offer an accurate account of cognition.
This indecision coming from the disciplines which feed into HCI is a major cause of mistrust amongst HCI practitioners. In Erickson’s poem, he states that he is often “theory leery.” This may be the result of putting energy into learning a theory, only to have it be obsoleted by three more conflicting theories which must also be painstakingly deciphered.
Ecological psychology—the afformentioned contender to cognitive psychology—has also spawned HCI applications. Vicente and Rasmussen (1990) created the Ecological Interface Design (EID) framework, which Sharp and Helmicki (1998) used to design a system for the assessment of tissue oxygenation in a Neonatal Intensive Care Unit (NICU) environment. Their design was found to significantly improve the accuracy of physicians’ diagnoses. Unfortunately, there are several problems with this line of research. First, many factors undoubtedly contributed to this improvement, including the researcher’s desire to create a usable interface, so the role of the EID framework is questionable. Second, the role of ecological psychology is also somewhat dubious in this case. The EID framework seems to be merely inspired by the sense of ecological psychology, without actually integrating any of its strong theoretical assertions.
This is what appears to be Erickson’s “dilemma.” HCI theory can be quite enjoyable and important in its own right, but not necessarily useful when one must design interaction. Theory is abstract by definition, and often practitioners are shut off from the “ivory tower” in which theories are laid out in full view. Ecological psychologists, in particular, make some very bold claims with scholastically cloistered rationales. An excellent example is Swenson and Turvey’s 1991 paper arguing that perception-action cycles represent an optimal entropy producing state. Erickson suggests that many such cases represent “clear concepts clad in fancy clothes,” and intimates that they are intentionally obfuscated.
The fact is, theories are abstract. They often strive for the unnatural; the low-level explanation which defies conventional wisdom. Theories exist in the physics community, for example, which not only lack obvious practical applications, but in many cases can’t even be applied experimentally. Theory is seen as a search for truth. Eventual practical gains are an assumption, not a promise—though historically the practical gains almost always come. Quantum physics, for example, is often overlooked by the architects of engineering curricula as too theoretical. Yet researchers are arguing that it provides important background for understanding many topics in engineering, including nanotechnology, photonics, superconductivity, and matter interferometry. (Wittmannn et al, 2002)
The model of Vicente and Rasmussen’s approach to design and theory points at a possible solution to Erickson’s dilemma. Sharp and Helmicki (1998) were able to carry out a successful EID-inspired redesign, despite the fact that neither of them seem to be an expert in HCI or psychology. The tenuous connection between EID and ecological psychology makes this a somewhat poor example, but the idea of having intermediate design frameworks which apply HCI theories in specific design methodologies is a good one. It creates an intermediate layer between theory, which is inevitably abstract, and design, which is inevitably practical.
Also occupying this area between pure theory and pure practice are some of the more methodological theories of HCI. Jakob Nielsen, one of the most successful champions of HCI, has long suggested heuristic methodologies (Nielsen and Molich, 1990) for their cost-effectiveness and ease of application. Such methods tend not to provide a coherent, predictive theoretical framework, but rather useful guiding principles, like using “simple and natural dialog” and “prevent errors.” These serve to focus the efforts of designers, but will not explain either behavior or interaction, except to say that designs that follow the heuristics will tend to be more usable.
Clearly GOMS, distributed cognition, the information processing approach, ecological psychology and Nielsen’s heuristics serve vastly different purposes for HCI practitioners. GOMS is useful in situations where you want to hone the performance of expert users. Distributed cognition is useful where you want to understand how the behavior of one user fits into their environment and the behavior of those around them. Other theories have still further uses. Erickson describes this as a “bricolage” or “ménage” of theory, and suggests keeping what is useful and discarding the rest. (“I’ll keep what’s good, discard dogmatic.”)
The approaches to HCI described above primarily explain and predict user behavior, and provide design guidance. Later on, I will suggest an approach to developing more generative theories, but in this section, I will suggest a way of dividing pure theories and theory-driven methodologies from practice-driven methodologies.
One feature separating theory from practice is that practically derived methods often employ actual human beings as models of human psychology. A theoretically-driven method of evaluating an automobile interface would be using the external cognition approach to identify external and internal representations of information, and distinguish computation done in the interface from computation done in the head and computation shared between the two. One might also look for examples of computational offloading or re-representation. In this case, theory is providing a model of interaction and a model of cognition.
By contrast, a more practice-driven methodology might suggest creating a mockup of an interface, trying it out on a real human being, and seeing what happens. Sometimes designers do not even go this far. A software engineer might simply say to his or her self, “does this interface make sense to me?” and use themselves as a model of human behavior. Even now, many in the HCI design world devote considerable energy to training designers to think like a user. In such a case, a skilled designer is simply a person who trained themselves to be a good model of their target audience when they need to be. This short-circuits the gap left by the lack of good predictive theoretical models.
The final point Erickson seems to make is that theory is not created in isolation, it is grounded in reality (“theory’s situated, no pure abstraction.”) A good example of this is Kirsch’s search for “deep structures” in his application of distributed cognition to workplaces. He suggests that a theory must have “the right psychological properties for inhabitants to act appropriately.” Even though he is looking for an abstract account of the deep structure, it must capture very real concerns at the practical level.
There is an emerging picture here, of the relationship between theory and practice in HCI. We have seen pure theory which is largely isolated from practice (the debate over the symbol processing nature of cognition) applications of theory with little methodological influence (Busby and Hughes’ application of Distributed Cognition,) applications inspired by theory but missing much of the theoretical debate (Sharp and Helmicki’s adaptation of ecological psychology) and applications steeped entirely in practice (Nielsen’s heuristics). What seems to be changing over the last twenty years is that the middle of this continuum is getting more and more crowded. As psychological theories improve and grow more socially relevant (distributed cognition) and relevant to tools (activity theory and external cognition) they only become more accessible. And intermediate frameworks like the EID framework can fill in the gaps.
Erickson devotes some energy in his poem to illuminating the disconnect between theorists and the rest of the world. He paints a picture of an ivory tower of theorists, content to share understanding only with the persistent few (“who’s to blame if the others? lack verve?”) and making jokes among themselves (“a ludic take on structuration”.) I have attempted to describe the effects of this situation on practical adoption of theories and suggest some possible solutions. Such practical adoption of theory is critically important for HCI, as it gives theories relevance and increased value in industry, which increases support for the field. I would argue two things, however: first that the tight circle of deeply entrenched theorists is also critically important, and second, that such a circle does not yet exist.
The reason this ivory tower does not exist is that HCI theories are generally fairly divergent. Kuutti (1996) characterizes the field as consisting of “fragmented subareas that evince little coherence or connection.” Don Norman’s The Design of Everyday Things (Norman, 1988) is as close as we seem to have to a viewpoint everyone can agree on, but that book’s legacy seems to be mostly the widespread acceptance of the importance of thinking carefully about technology in situ.
There is a lot of work to do to create the kind of theoretical focus that would enable the emergence of an ivory tower. This discussion was opened by commenting on the surfeit of theories in HCI coming from other fields. A large portion of the above discussion is devoted to psychology. And psychology, sociology, anthropology, graphic design, even linguistics generally place their lens over the humans and their environment. Yet, the name human-computer interaction gives the computer equal billing. HCI proponents have been battling technologists for decades to shift the focus of software development from the technology to the user. User-centered design guarantees relevance to the user being centered upon, but it runs the risk of forgetting massive portions of the computer’s abilities, which might meet needs which don’t even exist yet, or would never by discovered through ethnography or design.
One example is the invention of the mouse. Douglas Englebart has acknowledged the influence Vannevar Bush’s “As We May Think” (Bush, 1945) had on his work. (Griffin, Online) Bush’s prescient work predates the invention of the transistor, yet it draws together the calculator, photograph, facsimile, vocoder, stenotype, and pencil and paper and unifies them into a single argument for a fundamentally new kind of machine, one which extends the function of the human mind; an idea machine. Bush’s background was in mathematics, and his years of experience were spent building wondrously technological computers. Bush’s vision somehow got planted in the back of Engelbart’s brain.
Engelbart then spent considerable energy devising a “system approach to human intellectual effectiveness… a framework that provides orientation as to the important factors of the system, the relationships among these factors, the types of change among the system factors that offer likely improvements in performance, and the sort of research goals and methodology that seem promising.” (Engelbart, 1962) Engelbart was thus profoundly influenced by both the theory of mind and the theory of computation, and he made perhaps the single most significant advance in HCI by inventing the mouse. This is the type of advance which can occur with a deep understanding of both computers and people
It should be noted that Kuutti (1996) suggests the precise opposite—that direct manipulation interfaces were invented out of necessity in the sixties, without any understanding of the phenomena. It seems difficult to justify this position when the backgrounds of the Engelbart and Bush are taken into account. Another supporting example is the father of interactive computing, J.C.R. Licklider, who was originally a psychologist, and established at MIT what was arguably the first cognitive science department (though MIT did not recognize it as such.) In many ways, Licklider deserves just as much credit for the invention of direct manipulation as Engelbart does, and he has just as theoretical a background. It seems that Kuutti attributes the invention of direct manipulation interfaces to practice because she has not acknowledged its inventors.
In the spirit of J. C. R. Licklider and Vannevar Bush, I suggest that incorporating hybrid theories of computation and cognition into HCI is an important step, which will help make HCI theories more generative. I have also suggested focusing on the creation of intermediate design frameworks which can bridge the gap between theory and practice, which I believe will help make HCI theories more prescriptive. The deep understanding of both psychology and computer science seems to have led to great leaps in HCI in the past, and I believe the field runs the risk of missing these leaps if we stray too far into user-centeredness.
In the end, many of the issues highlighted in Erickson’s poem are inevitable. HCI is a young field, and thus there are many competing theories to consider. What is important is that the field does not give up on theory and substitute methods for models. We have seen applicability improve in distributed cognition, external cognition and activity theory, and the successes described in this essay point to a promising future for theory in the world of HCI practice.
Busby, J.S., Hughes, E.J. (2003) The role of distributed cognition in the causation of accidents. Int. J. Risk Assessment and Management, Vol. 4, No. 1, pp.36-51.
Bush, V. (1945) As We May Think. Atlantic Monthy, July 1945
Carroll, J. M. (2004) Beyond Fun. ACM Interactions, Volume 11, Number 5 (2004), Pages 38-40
Donald, N. A. (1988) The Design of Everyday Things. Currency Doubleday, New York, New York, 1988.
Engelbart D. (1962) Augmenting Human Intellect: A Conceptual Framework. Summary Report, Stanford Research Institute, on Contract AF 49(638)-1024, October 1962.
Galloway, A. (2001). The Origins of Ubiquitous Computing and Calm Technology. Available at: http://www.purselipsquarejaw.org/research_design/notes/ubicomp_origins.html
Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Greenwood Press, Westport, CT, 1983, ISBN 0-313-23961-4. First edition: 1966.
Griffin, S. Internet Pioneers: Doug Englebart Online?. Available at: http://www.ibiblio.org/pioneers/englebart.html Accessed 11/13/2004
Hutchins, E. (1995) Cognition in the wild. Boston, MIT Press, 1995
Hutchins, E. and Klausen, T. (1996). Distributed Cognition in an Airline Cockpit. In Communication and Cognition at Work, eds. D. Middleton and Y. Engeström, Cambridge: Cambridge University Press.
Hutchins, E. and Norman, D. A. (1988). Distributed cognition in aviation: a concep paper for NASA (Contract No. NCC 2-591).Department of Cognitive Science. University of California, San Diego.
John, B. E. (1995) Why GOMS? interactions, vol. 2, no. 4. pp. 80-89.
Kirsh, D. (2001) The Context of Work. Human Computer Interaction, 2001
Kuutti, K. (1996) Activity Theory as a potential framework for human-computer interaction. research. In Nardi, B. (ed), Context and Consciousness. Chapter 2, 17-44.
Myers B.A. (1994) Challenges of HCI Design and Implementation, ACM Interactions, 1994, 1(1), 73-83.
Nielsen, J., and Molich, R. (1990). Heuristic evaluation of user interfaces, Proc. ACM CHI'90 Conf. (Seattle, WA, 1-5 April), 249-256.
Rogers, Y. (2004) New Theoretical Approaches For HCI. ARIST: Annual Review of Information Sciences and Technology, no. 38, 2004.
Sharp, T., Helmicki, A. (1998). The application of the ecological interface design approach to neonatal intensive care medicine. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Chicago, IL. October 5-9. Santa Monica, CA: HFES.
Swenson, R., Turvey, M. T. (1991) Thermodynamic Reasons for Perception-Action Cycles. Ecological Psychology, 3(4), 317-348 Copyright c 1991, Lawrence Erlbaum Associates, Inc.
Whittmann, M. C., Steinberg, R. N., Redish, E. F. (2002) Investigating student understanding of quantum physics: Spontaneous models of conductivity. American Journal of Physics, Vol. 70, No. 3, pp. 218–226, March 2002