Note: The ® symbol indicates a registration for the convention
  Title:  User Experience and Usability Testing Within the Sociotechnical Process - Methods, Challenges and Pitfalls
AllAcademicCode:  1396134
    Time: 8:30 AM - 4:00 PM Tuesday, Oct. 23
Description:  User experience methods support work among designers, programmers and users to analyze work and learning processes with the aim to improve technologies. The full day workshop has the goal of introducing user experience (UX) projects including client communication, goal negotiation, conducting the study, methods, results and presentation of recommendations. Participants will develop an understanding and get the opportunity to apply a set of methods with hands-on material.
Full Abstract:  Back to List

1. Introduction
The goal of user experience (UX) is to improve technology from users’ perspectives. To be successful in design and development, researchers and designers can pre-define environments, workflows, and future users (Herrmann, Loser, & Jahnke, 2007). However, real contexts of technology usage may differ from what is anticipated. This disconnection can lead to situations where the full potential of the technology is not realized due to different reasons, a) users cannot adopt the new IT system due to technical problems, b) the technology may not be flexible enough to meet diverse users’ needs, c) socio-cultural contexts lead the technology adoption in a different direction from designers’ intention (Orlikowski, 1999). Social, organizational and technical aspects are crucial for future success of technology integration (Herrmann & Fischer, 2011). To address these issues, UX provides different methods to analyze sociotechnical systems and recommend design revisions for a technology integration.

2. Theoretical Lens
In this proposed workshop, we want to encourage participants to view the use of technologies as a new social practice of instructors, teachers and others. Each new technology prompt changes of existing work or learning. This, however, is not only a design interface problem. While bringing new technology into context, it will create new work or processes; it is an interwoven co-evolutionary growth of the social and technical aspects (Fischer & Herrmann, 2011). This wicked problem has been studied by sociotechnical approaches that support human behavior through technological and organizational change since its inception in the early 1950s (Trist & Bamforth, 1951). While Human-Computer-Interaction research investigates the person-tool relationship, the sociotechnical design sheds light on broader organizational or contextual issues (Grudin, 1994) such as new forms of work because of the new infrastructure. Human- or User-centered design (Norman & Draper, 1986) is not context free, which means that while in a certain context a technology works, in another it may fail. In order to study the new technologies, and how users use it, it might be fruitful to view them as a sociotechnical prompt that can trigger the development of new practices of users.
Whereas the technical elements can be designed and engineered, the social component involving participants and their interactions are often hard design, at least not in the same capacity as the technical component (Jahnke, 2016). We especially highlight the interplay between communicative human-human interaction enhanced with technologies (the Social), human-computer-interaction, and interaction between the technical elements (the Technical) (Herrmann, 2006). These three components are interdependent and evolving as Herrmann (2012) states: “rules can be designed, but conventions evolve” but interactions are never totally random nor predetermined. As such, Grudin (1988) states difficulties in design and development: a) It is difficult to know all the requirements of a system; b) users have diverse perspectives and aims that may not be clear to designers; c) it is a dynamic system with boundaries that change over time.
To address these challenges, UX provides a set of methods. Human-Computer-Interaction (HCI) focuses on improving interfaces (Shneiderman et al., 2016), e.g., Is the interface usable to perform a specific task? Sociotechnical methods (Prilla & Jahnke, 2012) try to address organizational aspects of work and learning, e.g., Is the technology useful and how does it changes existing work or learning processes toward new forms of work and learning?

3. Workshop Outline
In this workshop, we present work from our Information Experience Lab (founded in 2003) and share user experience and usability methods within the sociotechnical process. More specifically, we will introduce into user task testing (Wixon, 2009), System Usability Scale (Sauro, 2011), focus group interviews, ethnographic methods and observations (Patton, 2005; Whitehead, 2005), Persona methods, and work process identifying methods, e.g., SocioTechnical WalkThrough (Herrmann et al., 2007; Prilla & Jahnke, 2012).
The target audience includes researchers, designers, IT developers students, instructors and practitioners in the field of learning and performance systems design. The workshop is organized as two parts. The workshop is open to 6-12 participants.

Morning session
--Welcome, Agenda and Warming-up: Participants share their thinking “what is UX, usability?”, participants describe their background and projects (if applicable)
--Introducing the project process (three sample projects): discrepancy of typical textbook knowledge vs. our experiences working with clients
--Considerations of how to bring (useful) technology into work: How organization, humans and technology affect each other.

Afternoon session
--Introduction into methods of user task testing, SUS, focus groups interviews, and Persona development
--Group work: 3-5 participants, hands-on, present their solutions
--Lessons Learned, discussion of possible collaborations

We will close the workshop with take-away messages and reflections about the workshop.

--Patton, M. Q. (2005). Qualitative Research. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of Statistics in Behavioral Science. Chichester, UK: John Wiley & Sons, Ltd.
--Sauro, J. (2011). Measuring Usability with the System Usability Scale (SUS): MeasuringU. Retrieved Oct 15, 2017 from
--Whitehead, T. (2005, 7 17). Basic Classical Ethnographic Research Methods. Retrieved from Cultural Ecology of Health and Change:
--Wixon, D. (2009). Task Analysis. In Jacko, J.A. & Sears, A. (1st Eds.), Human-Computer Interaction (34-40). Boca Raton: Taylor & Francis Group.

Back to List
Handout(s) :  Open New Page for the Resource.
Note: The 'Resource' section is at the bottom of page
Sponsor:  Fee-Based Workshops       Session Type: Workshop
    Presenter(s):  ® Isa Jahnke, University of Missouri-Columbia, Neeley Current, University of Missouri, ® Hao He, University of Missouri Columbia, ® Yen-Mei Lee, University of Missouri-Columbia, ® Minh Pham, University of Missouri, Columbia, USA, Carl Hewitt, University of Missouri, ® Gayathri Sadanala, University of Missouri, Columbia, ® Joi Moore, University of Missouri,