MoDyS-Project: How to model interruption & resumption in human machine interaction

Interrupt me, please, I have to improve my work!

....

Multitasking: Benefit or Burden?

Doing too much makes you slower and dumber, states Collin Allen in the August 2005-Edition of Psychology Today. Nevertheless, plenty of research has been spent on the development and implementation of systems in the field of human machine interaction. Most of them concentrates on people`s “natural ability and predisposition to multitask” (Cherry, 1953). This burgeoning popularity of in-vehicle technology (McCarley et al., in press.) poses the question of how people manage multiple tasks simultaneously and how we can describe the underlying cognitive processes. One major problem in system design is based on the fact that we think in parallel, but act in serial. Besides this asynchronous parallelism (Edmonson, 1989), our cognitive capacities (Cowan, 2005) reveal limitations restricting the process of human information processing.

Task Switching or Task Resumption

Psychological literature provides a pool of denotations for the management of parallel tasks or goals. Some talk about task switching (...), whereas others use the term multitasking for the management of multiple tasks. Taatgen (2005) refers to dual task scenarios by proposing the term parallelization. To clarify, in this article we solely focus on the management of parallel tasks by defining it as multitasking in human machine interaction. Attention, however, can only be spent on one task at a certain moment in time. Altmann & Gray (1999) refer to this phenomenon as serial attention as the process of focusing mentally on one item at a time.

The nature of interruptions

A plethora of studies on multitasking is provided by psychology-related literature. Multitasking seen as the concept of interruption and resumption throws a different light on it. Basicly, one task is interrupted by another. Most of the time, the interrupted task is the primary task (main priority), the interrupting task is the secondary task. An interruption starts with an (external or internal) alert. A secondary task starts and ends (either with or without another alert) and the primary task is recovered. The time between the alert and the begin of the secondary task is defined as interruption lag, the time between the end of one and the resumption of another task is the resumption lag. The increase of literature on interruption and resumption demonstrates the increasing interest in this field (see also www.interruptions.net). McFarlane (2002) gives an detailed overview of relevant components. Interestingly, already in the middle eighties the management of multiple activities concentrated on interruptions. For Miyata & Norman (1986), the areas memory, attention and action are psychological concepts relevant in the study of multiple tasks or activities. They further differ task-driven processing (no sensitivity to external events) from interruption-driven processing (existing sensitivity to external events).

Conscious or unconscious?

Based on Miyata & Norman (1986), a task depends on conscious control if it is new or ill-learned. The authors refer to situations in which affective components (e.g., if teh task is critical, dangerous, includes a conflict) play a key role. .Following Rasmussen (1983), behavior on a skill-based level does not require attention, thus it is processing automatically without conscious control. The fundament of this research attests that two tasks referring to the same modality (see Wickens, XXX) can only be performed simultaneously if one of them dynamically is carried on without conscious control. To be more concrete, driving thus can be interrupted by availing people`s ability to continue because for a (more or less short) moment, attention can be directed away from the street (driving continues automatically) and directed to another (visual) task. An interruption, either external or internal (McFarlane, 1998), includes attention and, accordingly, consciousness. Interruptions lead to a switch of the focus of consciousness, but a person definitively has to be aware of the interrupting task.

Interruptions and performance

In the 20s, a student of Lewin found that uncompleted tasks are remembered better than completed one. This phenomenon entered psychological theories, and until today it is known as the Zeigarnik effect (Zeigarnik, 1929) named after the russian scientist who discovered it. Some students simulate this technique by planning pauses before completing a chapter, for instance. Cliffhangers in television work the same manner (of course, this is also related to what Hitchcock calls suspense). However, many research has referred to this well-established cognitive bias. In a modern office, for instance, interruptions (e.g. e-Mail notifications) are common features inherent in daily work.

The role of Prospective Memory in interruptions

Related to interrupting a task is a concept knwon as prospect memory. We refer to the management of several tasks simultaneously as multitasking, which itself is a sequence of interruption and resumption. At that point, it is time to stop and think of the nature of errors people are doing when managing multiple tasks. After an interruption, it is necessary to remember the task to be recovered. A failure to remember a task that needs to be performed in the future has been referred to as a prospective memory failure. Mary Czerwinski Eric Horvitz Susan Wilhite. Ellis (...) uses the term prospective remembering to describes the process and skills required to support the fulfillment of an intention to perform a specific action in the future. ELLIS, J et al (2000). Ellis [2000] emphasizes that interruptions of a task are the most frequently cited reason for failures in prospective memory [Ellis et al, 2000]. Prospect memory includes two necessary components:
(a) retrospective rehearsal (what was I about to do?)
(b) prospective goal encoding (what did I want to do?). [Trafton et al]. ]
At the age of Gestalt psychology, a Lewin student found a cognitive bias which to date is established as the Zeigarnick effect. The story behind it is both simple and fascinating: observing waiters coincidently, it turned out that unfinished tasks can be remembered more easily than finished, completed tasks. Applied to the field of task interruption, does interrupting improve performance? Positions vary: some allude to the Zeigarnik effect by showing that task interruption is task improvement. On the other side, there is evidence for a decrease in performance by interruption. At least, the complexity of the both (interrupted and interrupting) task seems to play a key role.

Aspects of a prospective (interrupting) task

Time-based vs. event-based tasks

The occurance of an interrupting task appear twofold:first, a task can occur at a certain moment in time (or after a certain amount of time). We refer to this as time-based task, in contrast to tasks that have to be performed synchronously when an event appears (event-based task). This distinction is very prominent in the area of prospective memory (Smith, R.E. & Bayen, U.J, 2004; ...).

Internally- vs. externally-driven

That

Cognitive modelling

The overall aim of our research group is to build (cognitive) models simulating user behavior in dynamicly changing environments. To do this, we use the ACT-R architecture (Anderson & Lebiere, 1998). The results of the simulation, i.e. the data provided by the ACT-R model(s) builds a baseline for the engineering process: prospective design (see also: www.prometei.de) is a technique sparsely applied so far. User modeling, in our eyes, starts before testing mock-ups of systems. Based on psychological theories, ACT-R models do not substitute but amend and support the design and implementation of systems adaptive to user specification.

Literature

The Literature part is related to

...on cognitive modeling

...on interruption (and resumption)

...on prospective memory

...various (miscellaneous)

...on cognitive modeling

Anderson, J. R. and Lebiere, C. (1998). The atomic components of thought. Mahwah, N. J.: Lawrence Erlbaum Associates.

Anderson, J. R., Taatgen, N. A., Byrne, M. D. (submitted) Learning to achieve perfect time sharing: Architectural implications of Hazeltine, Teague, & Ivry.

Byrne, M. D., & Anderson, J. R. (2001). Serial modules in parallel: The psychological refractory period and perfect time-sharing. Psychological Review, 108, 847-869.

Kushleyeva, Y & Salvucci, D. D. (2003) Human Multitasking: Towards an ACT-R Task-Independent General Executive. In Proceedings of the 25th Annual Meeting of the Cognitive Science Society. pg 1369.

Salvucci, D. D., Kushleyeva, Y., & Lee, F. J. (2004). Toward an ACT-R general executive for human multitasking. In Proceedings of the Sixth International Conference on Cognitive Modeling.

Salvucci, D.D. (2002). Modeling driver distraction from cognitive tasks. In Proceedings of the 24th Annual Conference of the Cognitive Science Society (pp. 792-797). Mahwah, NJ: Lawrence Erlbaum Associates.

Salvucci, D. D. (2001). An integrated model of eye movements and visual encoding. Cognitive Systems Research, 1(4), 201-220.

Salvucci, D. D., Boer, E. R., & Liu, A. (2001). Toward an integrated model of driver behavior in a cognitive architecture. Transportation Research Record, No. 1779.

Salvucci, D. D. (2000). A model of eye movements and visual attention. In Proceedings of the International Conference on Cognitive Modeling (pp. 252-259). Veenendaal, The Netherlands: Universal Press.

Salvucci, D. D. (2005) A multitaksing general executive for compound continuous tasks. Cognitive Science, 29, 457-492.

Taatgen, N. A. (1999). Learning without limits: from problem solving toward a unified theory of learning. Doctoral Dissertation, University of Groningen, The Netherlands.

Taatgen, N.A. (2005). Modeling parallelization and speed improvement in skill acquisition: from dual tasks to complex dynamic skills. Cognitive Science, 29, 421-455.

...on interruption (and resumption)

Altmann E. M. and Gray W. D. (2000) Managing attention by preparing to forget. In: Proceedings of the IEA 2000/HFES 2000 Congress. Santa Monica: Human Factors and Ergonomics Society, 152-155.

Altmann E. M. and Trafton J. G. (2002) Memory for goals: An activation-based model, Cognitive Science, 26, 39–83.

Altmann E. M. and Trafton J. G. (2004) Task interruption: Resumption lag and the role of cues. In: Proceedings of the 26th annual conference of the Cognitive Science Society (CogSci 2004).

Arroyo E., Selker T. and Stouffs A. (2002) Interruptions as multimodal outputs: Which are the less disruptive? In: Proceedings of 4th IEEE International Conference on Multimodal Interfaces (ICMI'02), Institute of Electrical and Electronics Engineers, 479-483.

Cypher, A. (1986). The structure of users' activities. In D. A. Norman & S. W. Draper (Eds.), User centered system design: new perspectives on human-computer interaction. (pp. 243-263). Hillsdale, NJ: Lawrence Erlbaum Assoc.

Cutrell E. B., Czerwinski M. and Horvitz E. (2000) Effects of instant messaging interruptions on computing tasks. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2000 Extended Abstracts). New York: ACM Press, 99-100.

Cutrell E., Czerwinski M. and Horvitz E. (2001) Notification, disruption, and memory: Effects of messaging interruptions on memory and performance. In: M. Hirose (Ed.), Human-Computer Interaction – INTERACT 2001 Conference Proceedings. Amsterdam: IOS Press, 263-269.

Czerwinski M. and Horvitz E. (2002) An investigation of memory for daily computing events. In: X. Faulkner, J. Finlay, F. Detienne (Eds.), People and Computers XVI – Memorable Yet Invisible: Proceedings of HCI 2002. London: Springer-Verlag, 229-246.

Czerwinski M., Christman S. and Rudisill M. (1991) Interruptions in Multitasking Situations: The Effects of Similarity and Warning. Technical Report JSC-24757, NASA Johnson Space Center, Houston, Texas.

Czerwinski M., Chrisman S. and Schumacher B. (1991) The effects of warnings and display similarities on interruptions in multitasking environments, SIGCHI Bulletin, 23 (4), 38-39.

Czerwinski M., Cutrell E. and Horvitz E. (2000) Instant messaging and interruption: Influence of task type on performance. C. Paris, N. Ozkan, S. Howard and S. Lu (Eds.), Proceedings of OZCHI 2000: Interfacing Reality in the New Millennium, Academic Press, 356-361.

Czerwinski M., Cutrell E. and Horvitz E. (2000) Instant messaging: Effects of relevance and time. In: S. Turner and P. Turner (Eds.), People and Computers XIV: Proceedings of HCI 2000, Vol. 2, British Computer Society, 71-76.

Czerwinski M., Horvitz E. and Cutrell E. (2001) Subjective duration assessment: An implicit probe for software usability. In: J. Vanderdonckt, A. Blandford and A. Derycke (eds.), Proceedings of Joint AFIHM-BCS Conference on Human-Computer Interaction IHM-HCI'2001, Volume 2, Toulouse: Cepadues-Editions.

Czerwinski M., Horvitz E. and Wilhite S. (2004) A diary study of task switching and interruptions. In: Human Factors in Computing Systems: Proceedings of CHI'04. New York: ACM Press, 175-182.

Dodhia R. M. and Dismukes R. K. (2003) A task interrupted becomes a prospective task. Poster presented at the 15th Annual Convention of the American Psychological Society.

Dodhia, R. M. and Dismukes R. K. (2003) A task interrupted becomes a prospective task: Encoding and retrieval manipulations. Poster presented at the 44th Annual Meeting of the Psychonomic Society.

Gillie T. and Broadbent D. (1989) What makes interruptions disruptive? A study of length, similarity and complexity, Psychological Research, 50 (4), 243-250

Hazeltine, E., Teague, D. & Ivry, R.B. Simultaneous Dual-Task Performance Reveals Parallel Response Selection after Practice. In JEP:HPP.

Hazeltine, E., Teague, D., & Ivry, R. B. (2002). Simultaneous dual task performance reveals parallel response selection after practice. Journal of Experimental Psychology: Human Perception & Performace, 28, 527-545.

Kushleyeva, Y., Salvucci, D. D., & Lee, F. J. (2005). Deciding when to switch tasks in time-critical multitasking. Cognitive Systems Research, 6, 41-49.

Lee, F.J. & Taatgen, N.A. (2002). Multi-tasking as Skill Acquisition. Proceedings of the twenty-fourth annual conference of the cognitive science society (pp. 572-577). Mahwah, NJ: Erlbaum. Fairfax, VA: August, 2002.

McCarley, J.S., Vais, M., Pringle, H., Kramer, A.F., Irwin, D.E., & Strayer, D.L. (In press). Conversation disrupts scanning and change detection in complex visual scenes. Human Factors.

McCarley, J.S., Vais, M.J., Pringle, H., Kamer, A.F., Irwin, D.E., & Strayer, D.L. (2004) Conversation distupts change detection in complex traffic scenes. Human Factors, 46, 424-436.

McFarlane D. C. (1997) Interruption of People in Human-Computer Interaction: A General Unifying Definition of Human Interruption and Taxonomy (NRL Formal Report NRL/FR/5510-97-9870), Washington: US Naval Research Laboratory.

McFarlane D. C. (1998) Interruption of People in Human-Computer Interaction. Doctoral Dissertation, George Washington University, Washington.

McFarlane D. C. (1999) Coordinating the interruption of people in human-computer interaction. A. Sasse and C. Johnson (Eds.), Proceedings of Human-Computer Interaction (INTERACT’99), Amsterdam: IOS Press, 295-303.

McFarlane D. C. (2002) Comparison of four primary methods for coordinating the interruption of people in human-computer interaction, Human-Computer Interaction, 17 (1), 63-139.

McFarlane D. C. and Latorella K. A. (2002) The scope and importance of human interruption in human-computer interaction design, Human-Computer Interaction, 17 (1), 1-61.

Monk C. A., Boehm-Davis D. A. and Trafton J. G. (2002) The attentional costs of interrupting task performance at various stages. In: Proceedings of 46th Annual Meeting of the Human Factors and Ergonomics Society (HFES 2002), Santa Monica: Human Factors and Ergonomics Society, 1824-1828.

Monk C. A. (2004) The effect of frequent versus infrequent interruptions on primary task resumption. In: Proceedings of 48th Annual Meeting of the Human Factors and Ergonomics Society (HFES 2004), Santa Monica: Human Factors and Ergonomics Society, 295-299.

Monk C. A., Boehm-Davis D. A. and Trafton J. G. (2004) Recovering from interruptions: Implications for driver distraction research, Human Factors, 46 (4), 650-663.

Monk C. A., Boehm-Davis D. A. and Trafton J. G. (2004) Very brief interruptions result in resumption cost. In: Proceedings of the 26th annual conference of the Cognitive Science Society (CogSci 2004).

Morey, C.C., & Cowan, N. (in press). When do visual and verbal memories conflict? The importance of working-memory load and retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition.

Morey, C.C., & Cowan, N. (2004). When visual and verbal memories compete: Evidence of cross-domain limits in working memory. Psychonomic Bulletin & Review, 11, 296-301.

Miyata Y. & Norman D. (1986) Psychological issues in support of multiple activities. In: D. A. Norman & S. W. Draper (Eds.), User Centered Systems Design: New Perspectives on Human-Computer Interaction, Hillsdale: Lawrence Erlbaum Associates, 265-284

Nagata S. F. (2003) Multitasking and interruptions during mobile web tasks. In: Proceedings of the 47th Annual Meeting of the Human Factors and Ergonomics Society, Santa Monica: Human Factors and Ergonomics Society, 1341-1345.

O’Conaill B. and Frohlich D. (1995) Timespace in the workplace: Dealing with interruptions. In: Human Factors in Computing Systems: CHI'95 Companion, New York: ACM Press, 262-263.

Speier C., Valacich J. S. and Vessey I. (1997) The effects of task interruption and information presentation on individual decision making. In: K. Kumar and J. I. DeGross (Eds.), Proceedings of the XVIII International Conference on Information Systems, Atlanta: Association for Information Systems, 21-36.

Speier C., Vessey I. and Valacich J. S. (2003) The effects of interruptions, task complexity, and information presentation on computer-supported decision-making performance, Decision Sciences, 34 (4), 771-797.

Strayer, D. L., & Johnston, W. A. (2001). Driven to distraction: Dual-task studies of simulated driving and conversing on a cellular phone. Psychological Science, 12, 462-466.

Trafton J. G., Altmann E. M., Brock D. P. and Mintz F. E. (2003) Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal, International Journal of Human-Computer Studies, 58, 583-603.

Turnbull D. G., Chewar C. M. and McCrickard D. S. (2003) Are cognitive architectures mature enough to evaluate notification systems? In: Proceedings of the 2003 International Conference on Software Engineering Research and Practice (SERP'03).

Van Bergen A. (1968) Task Interruption, Amsterdam: North-Holland.

Xia L. & Sudharshan D. (2002) Effects of interruptions on consumer online decision processes, Journal of Consumer Psychology, 12 (3), 265-280.

Zeigarnik, B. (1927). Das Behalten erledigter und unerledigter Handlungen. Psychologische Forschung, 9, 1-85.

Zeigarnik, B. (1967). On finished and unfinished tasks. In W. D. Ellis (Ed.), A sourcebook of Gestalt psychology, New York: Humanities press.

...on (prospective and working) memory

Brandimonte, G.O. Einstein & M.A. McDaniel (Eds.), Prospective memory: Theory and Application (pp. 23-51). Mahwah, NJ: Erlbaum.

Cowan, N. 2001, The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral & Brain Sciences. Special Issue, 24(1), 87-185.

Cowan, N. (2005). Working-memory capacity limits in a theoretical context. In C. Izawa & N. Ohta (eds.), Human learning and memory: Advances In theory and applications. The 4th Tsukuba international conference on memory. Erlbaum. (pp.155-175)

Cowan, N., Elliott, E.M., Saults, J.S., Morey, C.C., Mattox, S., Hismjatullina, A., & Conway, A.R.A. (in press). On the capacity of attention: Its estimation and its role in working memory and cognitive aptitudes. Cognitive Psychology.

Cowan, N., Johnson, T.D., & Saults, J.S. (in press). Capacity limits in list item recognition: Evidence from proactive interference. Memory.

Einstein G. O., McDaniel M. A., Williford C. L., Pagan J. L. and Dismukes R. K. (2003) Forgetting of intentions in demanding situations is rapid, Journal of Experimental Psychology: Applied, 9 (3), 147-162.

Ellis, J. & Kvavilashvili, L. (2000). Prospective memory in 2000: Past, present and future directions. Applied Cognitive Psychology, 14, 1-9.

...various (miscellaneous)

Amelang, M. & Bartussek, D. (1990). Differentielle Psychologie und Persönlichkeitsforschung. Stuttgart: Kohlhammer.

Broadbent, D. E. (1958). Perception and Communication. Oxford: Pergamon.

Edmondson, W. H. (1989). Asynchronous parallelism in human behaviour: A cognitive science perspective on human-computer interaction. Behavior and Information Technology, 8(1), 3–12.

Gopher, D. (1994). Dual-task performance. The Blackwell dictionary of cognitive psychology, 111-116.

Hockey, G. R. J. (1986). A stress control theory of adaptation and individual differences in stress management. In G. R. J. Hockey,A. W. K. Gaillard, & M. G. H. Coles (Eds.), Energetics and human information processing (pp. 285-298). Dordrecht: Matinus Nijhoff.

Hockey, G.R.J. (1997). Compensatory control in the regulation of human performance under stress and high workload: A cognitive energetical framework. Biological Psychology, 45, 73-93.

Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149.

Mattes, S. The lane-change-task as a tool for driver distraction evaluation. In Proceedings of IGfA, 2003.

Rauterberg, G.W.M., & Felix, D. (1996). Human Errors: Disadvantages and Advantages. In Proceedings of the 4th Pan Pacific conference on occupational ergonomics - PPCOE '96 (pp. 25-28). Hsinchu: Ergonomics Society Taiwan.

Rasmussen, J. (1983). Skills, Rules, and Knowledge: Signals, signs, symbols and other distinctions in human performance models. IEEE Transactions on Systems: Man and cybernetics, 13.

Wickens, C. D. (1984). Multiple Resource Model of Human Performance: implications for Display Design. Williamsburg, VA.

• http://www.prometei.de/
  official website of the research training group prometei
• http://interruptions.net
  including a bunch of literature in interruption (and resumption)
• http://act-r.psy.cmu.edu/
  official ACT-R homepage

• http://www.pages.drexel.edu/~cgt22/
  on driver distraction
• http://www.zwisler.de/scripts/reason.html
  on human error
• http://www.home.zonnet.nl/waard2/mwl.htm
  mental workload
• http://en.wikipedia.org/wiki/List_of_cognitive_biases
  a list of cognitive biases
• http://psychclassics.yorku.ca/
  classics in the history of psych

• http://www.home.zonnet.nl/waard2/mwlch7.htm
  literature on driver`s mental workload