UPDATE:The position papers accepted for presentation at the workshop can now be downloaded here.
UPDATE: Preliminary programme now online (see below).
Preliminary Programme
9:00-10:30 Introduction This session will introduce the workshop participants and their research. Every participant will be given approximately five minutes to introduce themselves. If you want to use any slides in the process, please email them to Chris prior to the event. We will also introduce the different research questioned that were submitted.
10:30-11:00 Coffee break
11:00-12:30 Example problems We will present a small number of example 'problems'/systems for small groups to work on - either to explore one of the submitted research questions or to identify relevant issues. At the end of the session, we'll have a brief plenary discussion about the results from this exercise and set up break out groups for the first afternoon session.
12:30-13:30 Lunch break
13:30-15:00 Break-out groups In session, small break out groups will discuss one of the research questions to identify potential routes to address the challenges posed by them.
15:00-15:30 Coffee break
15:30-17:00 Synthesis session Based on the reports from the break-out groups, we will try to summarise the findings in a consistent way. We'll discuss the report paper to be published in the workshop proceedings as well as future steps to be taken.
19:30 Workshop dinner If there is sufficient interest, we will organise an informal workshop dinner at a casual restaurant near the conference site.
Original Call for Papers
Scope
A significant number of ubiquitous systems are being built to support human users in performing a variety of tasks. These applications cover a large range of scenarios, including safety-critical medical applications, customised support in the workplace, and leisure-related applications. Users interact with these systems via implicit or explicit means, e.g. by moving about in an environment or using custom-built devices. At the same time, many ambient applications deploy sensors to gather information about the context, in which those interactions take place, and system behaviour can change depending on contextual factors. All this adds up to a fairly complex situation, which poses new challenges for evaluation, potentially pushing the boundaries of traditional evaluation methods and opening up opportunities for novel approaches.
The goal of this workshop is to bring together researchers with an interest in this area to discuss the current state of the art in evaluating ubiquitous systems with real users, to identify shortcomings and benefits of traditional evaluation methods and to explore novel approaches. We are inviting position papers addressing issues such as:
- Experiences from conducting user studies with ubiquitous systems
- Factoring in contextual factors in evaluation
- Benefits and shortcomings of lab-based/in-field user studies in Ubicomp
- Novel approaches to evaluate ubiquitous systems with users
- Comparing different methods for evaluation
- Evaluating implicit interactions
- Evaluation of user experience prior to deployment/completion
- The role of prototyping in evaluating ubiquitous systems
- Cultural/technological probes
- Virtual and simulated environments as platforms for user studies
This list is indicative of the topics covered but we explicitly invite papers discussing other related questions. Submissions should consist of three parts:
- A position paper discussing a related topic (up to two pages)
- A short research statement of the submitting author describing their particular interests and previous research relating to the workshop topic (up to half a page)
- A particular issue/question that should be discussed at the workshop
All three parts should be saved as a single PDF file and should be sent to Chris Kray (c.kray@ncl.ac.uk). The submission deadline is 20 August 2007.
Format
The workshop will be organised as a one-day event, focussed on discussion. We will use the first session to introduce participants (via brief talks of about 5 mins), which will be followed by an initial round of discussion. One outcome of this discussion will be the (sub-)set of questions to discuss more thoroughly in break-out groups. During the second session, we will break up into smaller groups of about 5 members each to discuss the previously identified questions in-detail. We will all come together again for the third session, where individual groups will report on their discussion. We will conclude on a plenary discussion of the points made and on how to proceed with the topic in general. We aim to limit the workshop size to about 20 participants.
Expected outcome
There are two main outcomes we aim to achieve in organising this workshop: community building for researchers interested in this topic and an initial research agenda, outlining open questions and promising new research directions. We will also discuss the option of initiating a special issue of a related journal on the topic of the workshop.
Important Dates
| 20 September 2007 | Submission deadline |
| 31 September 2007 | Notification of acceptance |
| 20 October 2007 | Camera-ready copy due |
| 21 October 2007 | Position papers available on workshop web page |
| 7 November 2007 | Workshop at AMI 07, Darmstadt, Germany |
Organising Committee
| Christian Kray | Newcastle University, Newcastle upon Tyne |
| Lars Bo Larsen | Aalborg Universitet, Aalborg, Denmark |
| Patrick Olivier | Newcastle University, Newcastle upon Tyne, UK |
Program Committee
| Ian Bishop | University of Melbourne, Melbourne, Australia |
| Cornelia Boldyreff | University of Lincoln, Lincoln, UK |
| Keith Cheverst | Lancaster University, Lancaster, UK |
| Mark Dunlop | Strathclyde University, Glasgow, UK |
| Connor Graham | Lancaster University and University of Melbourne |
| Paul Holleis | Ludwig-Maximilians-University, Munich, Germany |
| Judy Kay | University of Sydney, Sidney, Australia |
| Jesper Kjeldskov | Aalborg University, Aalborg, Denmark |
| Ingrid Mulder | Telematica Instituut, Enschede, The Netherlands |
| Julian Newman | Glasgow Caledonian University, Glasgow, UK |