Allocation of function: scenarios, context and the economics of effort
Andy Dearden1, Michael Harrison2 & Peter Wright2
1.	Armature Ltd.
1 Apex View
Leeds
LS11 9BH
UK
Email: ADearden@armature.com
Tel: (+44) 113 259 5131
Fax: (+44) 113 259 5150	2Dept. of Computer Science
University of York
York
YO1 5DD
UK
Email: {mdh,pcw}@cs.york.ac.uk
Tel: (+44) 1904 433376
Fax: (+44) 1904 432767	

Abstract
In this paper, we describe an approach to allocation of function that makes use of scenarios as its basic unit of analysis. Our use of scenarios is driven by a desire to ensure that allocation decisions are sensitive to the context in which the system will be used and by insights from economic utility theory. We use the scenarios to focus the attention of decision makers on the relative costs and benefits of developing automated support for the activities of the scenario, the relative impact of functions on the performance of the operator’s primary role and on the relative demands placed on an operator within the scenario. By focussing on relative demands and relative costs, our method seeks to allocate the operator’s limited resources to the most important and most productive tasks within the work system, and to direct the effort of the design organisation to the development of automated support for  those functions that deliver the greatest benefit for the effective operation of the integrated human-machine system.
1.	
Introduction
In this paper, we describe an approach to allocation of function that takes scenarios as its basic unit of analysis. Our aim is: to ensure that the allocation recognises the many different ways in which humans and machines may co-operate; to make the allocation decision sensitive to the envisaged working context of the system; and to recognise the limited resources that are available to the human agent(s) in the system and to the organisation that is designing and developing the system. 
To achieve this aim we take the traditional question of function allocation, i.e. 
“to which agent, human or machine, should we allocate this function?”, 
and reframe the question as:
“To which functions, required of this system in this context, should we allocate the limited resources of this agent, human or development organisation?”
Our proposed method is currently undergoing evaluation with British Aerospace Military Aircraft & Aerostructures plc. 
1.1.	Defining the allocation problem 
In this paper, we define allocation of function by the following statement.
‘Allocation of function is an early stage of the design of a human-machine system. The input to allocation of function is a specification of the functions that the human-machine system must deliver within its intended working context. The output from allocation of function is a specification, at an appropriate level of abstraction, of the functionality of the automated subsystems that will be required. The goal of allocation of function is to design a system for which:  the performance (including considerations of safety and reliability) is high; the tasks of the operator are achievable and appropriate to the operator’s role; and the development of the system is technically and economically feasible.’
Some of the terms of this definition require elaboration.
Functions 
We take the system ‘functions’ to be activities that the integrated human-machine system is required to be capable of performing. The description of the system at a functional level should not assume any particular division of work between human and machine. Our interpretation of  functions as device independent inputs to allocation is consistent with the view propounded by Cook & Corbridge (1997). For instance, ‘monitor fuel levels’ would be an acceptable function required of the human machine system for an aircraft, but ‘monitor fuel display’ would not. To keep the distinction between functions that are requirements of the whole human machine system and human activities that are consequences of design decisions, we shall refer to work eventually assigned to a human operator as a set of ‘activities’ (our ‘activities’ are thus equivalent to ‘tasks’ in Cook & Corbridge (1997)). Some authors prefer to use the term ‘task’ rather than ‘function’ when discussing the inputs to allocation, when reviewing their work we retain their nomenclature. 
Early stage of design
UK Ministry of Defence standards (MOD, 1989) place allocation explicitly as an element of preliminary design, prior to the detailed design of equipment and construction of prototypes. A proposal for allocation of function may be necessary when initially tendering for a large development contract, to demonstrate that the proposed concept is feasible. Goom (1996) points out that the ‘gestation period’ for a complex system can be up to 15 years, hence allocation decisions may need to be made more than 10 years before delivery of the final system.
Working context
We take the working context of the system to be partially defined as a set of assumptions about the situation into which the system will be deployed. In the case of military systems, these assumptions may  include statements about the type of mission that the system will be required to perform, the kinds of threats it is likely to encounter, and the supporting infrastructure that can be assumed to be available. They may also include statements about the relationship of the human operator to other human agents and agencies (see operator’s role, below). There may also be some initial design assumptions about the physical characteristics of the workspace, e.g. that the human will be sitting in a restricted space such as an aircraft cockpit. These assumptions may be stated at the beginning of the allocation process, however, we recognise that such assumptions may be questioned within the allocation and design process, and may be revised during the design conversation with reference to the views of the commissioning organisation.
An appropriate level of abstraction
Given the very early stage of design that we are considering, the specification output by the allocation of function process necessarily describes the requirements for the automation at a high level of abstraction. However, the specification must be sufficiently precise to allow a design team to compare and contrast the relative difficulty of developing the proposed automated system; the relative demands upon the human that would follow from different allocation decisions; and the relative performance that might be achieved by a human-machine system based on alternative allocation decisions.
The operator’s role
By operator's role we mean a high level description of the responsibilities of the operator. In some definitions,  role can be reduced to the aggregation of tasks that a person performs,  but many tasks that people perform they do not feel responsible for and many things they feel responsible for are not associated with tasks that they themselves perform. Instead, our definition rests on the concept of accountability (McCarthy et al. 1998). Activity within an organisational context is divided not only in terms of who will carry out the tasks and actions that constitute the activity, but also in terms of who will be called to account for the success or failure of that activity. Accountability can be locally negotiated by those involved in the task performance, or it can be globally imposed by others in the organisation not directly involved in task performance. Accountability can also be distributed in an organisation.
1.2.	Structure of this paper
In section 2, we review existing approaches to allocation and suggest two major failings, namely: that existing methods disregard important details of the work context that is being designed; and that existing methods obscure some of the organisational trade-offs that need to be made. In section 3, we present a new method of allocation that re-frames the allocation problem in terms of the resources available to the design organisation and to the human operator. In section 4, we present an case study that applies the method. In section 5, we discuss the new method and relate it to the review in section 2. Finally, in section 6, we present our conclusions and outline further work.
2.	Existing approaches to allocation
In this section, we review existing methods of allocation and seek to demonstrate the need for a new approach.
In examining existing approaches, we shall focus on three specific aspects:
1)	the way in which alternative allocations are characterised;
2)	the criteria that are advocated as relevant to the allocation decision; and
3)	the procedures that are used to search the space of possible allocations.
We then review critiques of allocation suggesting that existing approaches are unsatisfactory. In particular, we are concerned with:
4)	the number of iterations required within allocation processes.
This leads to a diagnosis that suggest two reasons for the problems identified, namely that current approaches to allocation:
5)	disregard important aspects of the working context; and
6)	obscure the trade-offs that the development organisation must make.
2.1.	Characterising allocation alternatives
In their extensive review of function allocation, Older et al (1997) draw a distinction between techniques that restrict the alternatives for a function to either allocation to the human or to the machine, and those techniques that permit an allocation in which the function is shared between human and machine.
The early work of Fitts (1951) and allocation approaches using MABA-MABA (Men are better at …. Machines are better at ….) lists are typically restricted to considerations of either the human or the machine performing each individual function.
Since Fitts’ early work, it has become apparent that many functions in complex systems require sharing of the function between human and machine. Such sharing may take place in many different ways. Billings (1996) identifies 5 different levels of automation between totally manual operation by a human and fully autonomous operation by a machine. Sheridan & Verplanck (1978) offer a set of 8 intermediary levels that can occur in a supervisory control system. Goom (1996) observes that: 
“In modern defence systems, the majority of functions need both humans and machines to undertake complementary tasks”[p53]
Given this observation, any method that restricts the allocation alternatives to a binary decision (human or machine) or a ternary decision (human, machine or shared) will be inadequate.
Techniques developed more recently recognise the possibility of sharing a function and consider the different ways such a sharing could be configured. Ip et al. (1990) use ‘Task Allocation Charts’ to break down each function into a series of subtasks that can then be allocated to either a machine or a human. Meister’s (1985) method uses a similar scheme, configuring a set of subtasks required to implement a function and then considering alternative ways these could be shared between human and machine. Some techniques offer generic decompositions of a function, allowing each element to be allocated to either a human or a machine. For example, Shoval et al. (1993) use a three stage model of perception, cognition and action for each function.
2.2.	Selecting and combining criteria in allocation decisions
Having identified some alternative options for allocating functions, a method needs to offer some guidance as to criteria that are relevant to the decision. In the historical development of allocation methods, we see a gradual enrichment of the set of criteria applied. 
Traditional MABA-MABA lists focus exclusively on the relative performance of human or machine with respect to each function. More recent work has identified multiple criteria that might be applied in allocation decisions. Meister (1985) suggests seven criteria (performance, reliability, number of personnel, workload, cost, maintainability and safety) that should be considered. In a review of techniques, Older et al. (1997) list seventeen different criteria that have been applied. Papantonopoulos & Salvendy (1993) provide a list of twenty-six criteria suggested by other authors.
More recent work has identified the need to consider the overall structure of the work that the human has been assigned. In the development of military systems Goom (1996) states:
“One of the principle lessons that has emerged from the application of MANPRINT has been the need to identify the jobs of the users”[p.48]
In the method proposed by Clegg et al. (1996), the allocation of tasks is preceded by an initial selection of an ‘organisation of work’ which defines, amongst other elements, the various ‘roles’ that the humans will have within the work system. Consistency with the human’s identified role is then used as a criterion when allocating tasks.
2.3.	The search procedure
Given a characterisation of the option space for allocation and a set of criteria, a method requires some procedure for exploring a range of allocation possibilities in order to find ‘good’ or ‘optimal’ allocations. Two basic strategies are evident in the literature as surveyed by Older et al. (1997). We consider these two strategies below.
2.3.1.	Allocating individual functions in isolation
The majority of techniques for allocation during systems consider each function in turn and attempt to identify a ‘best’ allocation for that function. Once a hypothetical allocation for each function has been generated, the set of suggested allocations is examined to check whether it meets the design constraints. If not, the process iterates and some allocations are changed. Most authors assume that multiple iterations will be required (Beevis & Essens, 1997).
An example of this type of procedure is described by Pulliam & Price (1984). The procedure takes each function and considers a set of alternative engineering concepts that might be possible to implement it. Each of these concepts is then analysed by applying a series of qualitative questions to the proposal, such as: ‘is automation mandatory?’, ‘if mandatory, is automation feasible?’, ‘is automation technically preferable?’, ‘should control tasks be allocated to man to keep him occupied, interested and informed?’. In the later stages of the procedure, the set of hypothetical allocations is tested ‘deductively’ by asking questions such as: ‘does man meet core performance requirements ?’; ‘does man meet human performance requirements ?’; ‘are cost trade-offs acceptable ?’; ‘is the human factors structure adequate ?’; ‘is cognitive support adequate ?’; ‘is job satisfaction optimal ?’. If these questions receive negative answers, the procedure iterates.
The same iterative pattern is apparent in the methods outlined by Meister (1985), Gramopadhye et al. (1992), Clegg et al. (1996) and by Corbridge & Cook (1997). The approach advocated by Clegg et al. (1996) makes some attempt to limit the number of iterations by considering two questions that relate allocation for each task to other tasks in the system, namely:
‘would the automation of this task constrain other choices ?’
and
‘are there any aspects of this task that make it critical to the performance of the system ?’
2.3.2.	Global optimisation techniques
An alternative search strategy, is to apply optimisation techniques to search for the best global allocation (i.e. to optimise the allocation of all the functions in the system simultaneously). This approach is evident within work on dynamic allocation. The only instance surveyed in Older et al. (1996) of this approach being advocated for allocation during systems design is in a preliminary proposal made by Denley (1989). Unfortunately, Denley’s suggestions have not been further developed.
In work on dynamic allocation, the optimisation strategy is applied either to a small set of functions, or to a large set of instances of the same basic function (or task). For example, Shoval et al. (1993) describe the development of a navigational aid for blind users that allows different global allocations of  three tasks ‘perception’, ‘cognition’ and ‘action’ between the human and the machine. The system monitors the performance of the user and the complexity of the environment and uses this information to select an ‘optimal’ mode of operation. In the system, considered by Rencken & Durrant-Whyte (1993) there are, at any one time, a large set of functions to be performed which may be drawn from one of two types: either (a) identifying a new object or (b) re-capturing an existing object. Each of these functions may be allocated to a machine, allocated to a human, or left in the queue. The algorithm searches for an allocation of all the functions in the system that minimises a global cost function. 
A natural question to ask is whether the optimisation techniques used in dynamic allocation could be scaled to support the design of large complex systems. This is unlikely because the ‘cost’ or ‘quality’ functions used in generating the global optimisation require parameters to be set for each function. In a system involving a large number of heterogeneous functions, and which takes into account a rich set of criteria, a very large number of such parameters would have to be defined and set. Hence, attempting to scale the approach merely shifts the complexity from determining the best allocation for each function to determining the correct parameters and the appropriate values to use.
2.4.	The need for iteration
A basic problem dealing with each function in isolation is that it seems to require a number of iterations of the allocation process before a satisfactory solution can be found. Beevis & Essens (1997), reporting on a NATO workshop devoted to function allocation, touch on the question of why successive iterations are recommended by almost all authors who seek to allocate each function in isolation: 
“This may suggest predictive weakness in available function allocation techniques;…”
Unfortunately, they immediately retreat from exploring this issue by continuing:
“… more likely it reflects the many criteria that are involved in the decision”[p8, ibid.]
In contrast to Beevis & Essens’ apologies for successive iteration, Fuld (1993, 1997) presents a more radical position. According to Fuld: 
“…..the more I’ve tried to apply formal allocation (in my case, to nuclear power plant design), the more convinced I have become that it simply doesn’t work, organisationally or analytically as a central focus of the design process.”(Fuld, 1993, p21).
As a result of this observation, Fuld concludes that: 
“… the allocation concept is a poor framework for generating design hypotheses.”(ibid., p22).
In place of the effort to allocate functions by a formal analytic process, Fuld recommends that designers should be allowed to develop proposals for the form of a system, with human factors expertise being focussed on the review and testing of the emerging design. Thus:
“The correction of misfits occurs throughout the design process, but in the hypothesise-and-test model, it is testing that systematically identifies misfits…….I suggest that human factors practitioners subsume their concern for large-scale allocation problems within practical human-factors review and verification activities (that is testing for adequacy and lack of significant misfit) in the emerging design.”(ibid., p22).
The potential hazard with Fuld’s critique is that his arguments might lead organisations to ‘throw out the baby with the bathwater’ by abandoning any attempt to think about the work allocated to the human until the form of the design has reached a highly concrete level, e.g. a high level specification for the whole system, or even an early prototype.
Below, we demonstrate that the predictive weakness identified by Beevis & Essens (1997), is a likely result of any procedure that considers each function in isolation. In the second half of the paper, we present an alternative approach that we claim reduces the need for large numbers of iterations over allocation decision, whilst enabling human-factors concerns to be brought into these early stages of design.
2.5.	Isolating functions disregards work context
In recent years, there has been a growing awareness of the importance of contextual factors in interactive systems design. This trend is evidenced by the current interest in alternative theoretical perspectives such as Activity Theory (Nardi, 1995), distributed cognition (Hutchins, 1995; Palmer et al, 1993), ethnomethodology and situated action (Suchman, 1987) and in new design approaches such as Contextual Design (Holtzblatt & Beyer, 1997) and Scenario-Based Design (Carroll, 1995; Kuutti, 1995). Proponents of contextual approaches highlight the complex dependencies and interactions between work activities in any work system. Allocating individual functions in isolation fails to respect these interdependencies. The example below shows how an understanding of contextual factors may alter allocation decisions.
Fields et al. (1997) consider the work that may be necessary in an aircraft flight-deck following a bird-strike incident. At the start of the scenario the aircraft is flying low over water, using three engines (this is a normal context for the particular aircraft under consideration). When the bird strike occurs, engine fires are detected in both engines on one wing, leaving only one engine operating. In the formal description of the work, there are defined emergency procedures for: shutting down the burning engines; extinguishing the fires; re-starting the non-working engine; reconfiguring hydraulic systems that are pressurised by the engines. Other activities such as altering the aircraft trim and reducing drag are also required but are not so formally defined. 
If functions are considered in isolation, there may be strong arguments for automating the function of shutting down the engine. The function is highly proceduralised and therefore easy to automate (automation is low cost), the function may impose high workload (automation will reduce workload), an automated system is less likely to make an error (automation would improve reliability) during execution of this safety-critical function (automation could improve safety). However, when the specific work context is examined, the resolution of the conflicting goals (shutting down the engine vs. maintaining thrust to gain altitude) is highly contingent and the pilot may be held responsible for the management of the emergency. This suggests that the pilot should have final authority over the timing of these actions. Hence, the allocation recommended changes when the function is considered in context.
Further, Wright et al. (1998) present an analysis of the same scenario that highlights the fact that although formal procedures are provided for each of the failures, an important aspect of the pilots’ work involves switching between the different procedures, scheduling their actions and prioritising some activities over others.
Such co-ordination, prioritisation and scheduling activities are not directly associated with any of the original list of functions identified at the system level. Rather, they are emergent within the work context. As Dekker & Wright (1997) point out, allocation is not a ‘zero-sum game’, automating a function does not necessarily result in a reduction in workload equivalent to the human work necessary for that function. Rather, it transforms the work that the human must perform. Such emergent activities must be considered when making allocation decisions, but they can only be identified by considering the set of functions that must be delivered concurrently within the scenario.
2.6.	Isolating functions obscures organisational trade-offs
From the perspective of the development organisation, allocation decisions serve to guide the organisations’ investment in research and development during the early years of the project. In designing the system, the organisation seeks to provide the best possible system, for a  competitive price. To achieve this objective resources must be directed at those developments that are expected to deliver the greatest benefit for the effort expended.
Below, we develop a simple economic model to represent this type of problem. In the model we shall treat the notion of the ‘best possible’ system by using a general concept of ‘quality’, where quality is assumed to be some aggregated measure of the various benefits that may be offered by a system.
A simple economic model of allocation
A system is required to deliver a set of functions fi , i = 1 .. k. 
For each function, two possible design options are generated. Option A for each function, denoted fiA , delivers adequate level of quality QiA for a cost CiA. A second option for each function, option B denoted by fiA , delivers a higher quality, QiB , at a higher cost CiB.
The overall performance of the system is modelled as a weighted sum of the quality measures appropriate to each function, i.e.
EQUATION 1 HERE
The allocator must decide which option should be chosen for each function, given that the overall cost of the system will be given by:
EQUATION 2 HERE
Clearly the cheapest (and lowest quality) system that can be constructed uses option A for all functions, and the most expensive, highest quality system uses option B for all functions. However, the problem for the allocator is to find the system that provides the highest quality per unit cost, given by:
EQUATION 3 HERE
Now consider the choice between two alternative systems that differ only with respect to the option chosen for a single function fj, i.e. consider the question of which option (A or B) should be chosen for function j. 
Let: 
EQUATION 4 HERE
and
EQUATION 5 HERE
System 1 (using option A for function j) provides a better quality to cost ratio than system 2 (using option B) if and only if:

EQUATION 6 HERE						(Equation A)

What is apparent in this equation is that the decision over which option to select is not determined by the quality/cost ratios of the two options for function j. Rather, it is determined by a relationship between: on the one hand, the ratio of the marginal quality improvement and marginal costs associated with switching from option A to option B; and the other hand the overall quality/cost ratio for all the other functions in the system. Therefore, the ‘best’ allocation for function j cannot be discovered by considering that function in isolation. This result is consistent with the theory of marginal economic utility (see any basic economics text, e.g. Parkin & King, 1995).
The following simple example demonstrates the conclusion of the model.
A system must be designed to deliver 3 functions, A, B & C. 
Two design options are generated for each function, as shown in table 1.
FUNCTION	Option 1 – highly automated	Option 2 – limited automated assistance	
Function A
	Cost: £2 million
Performance: 3 units	Cost: £1 million
Performance: 2 units	
Function B	Cost: £2 million
Performance: 3 units	Cost: £1 million
Performance: 2 units	
Function C	Cost: £6 million
Performance: 2 unit	Cost: £4 million
Performance: 1 unit	
Table 1: an example allocation problem
Considering function C in isolation, we might think that option 1 is superior to option 2 because the ratio of performance to price (in £ millions) is 2/6 = 1/3 for option 1, and only 1/4  for option 2. Conversely, considering either function A or B in isolation, option 1 offers a performance/price ratio of 3/2 vs. a ratio of 2/1 for option 2, hence we might expect that option 2 should be preferred. 
In fact, for the system as a whole, the highest quality per unit price that can be obtained uses option 1 for functions A and B, and option 2 for function C. This combination provides a total performance of 3 + 3 + 1= 7 units for a cost of £8 million. This solution cannot be discovered by any procedure that considers each function in isolation.
2.7.	Summary
In this section, we have examined existing methods for allocation, and identified two important weaknesses of any procedure that attempts to allocate individual functions in isolation. Namely, that such procedures: 
1)	disregard complex interdependencies in work context that is being designed; and
2)	obscure the trade-off decisions that the development organisation must make.
In summary, our analysis suggests that existing approaches to allocation of function may have been encouraging designers to ask the wrong question. We suggest that rather than asking:
“to which agent, human or machine, should we allocate this function ?”
systems designers should be considering the capabilities of the operator, and the design organisation and asking:
“to which functions, required of the system in this context, should we allocate the limited resources of this agent, human or development organisation?”
In the second half of this paper, we present a new approach to allocation that takes this resource allocation perspective as its starting point.
3.	Function allocation as a question of resources
In this section, we present a new method that treats allocation of function as a resource allocation problem. To enable the allocation team to identify the trade-offs that face the human and the design organisation, the method proceeds by examining a collection of scenarios. Each scenario may involve many functions that must be delivered concurrently. The scenario helps to expose the complexity of the working context that is being designed, and reveals some of the organisational trade-offs implicit in developing new automation. The method is being developed and evaluated in collaboration with British Aerospace and has been tailored for use in the design of single-seat aircraft. Further work is planned to extend the method to allocation for multi-person crews. 
Within each scenario, the method considers the relative merits of the set of design options for the set of all the functions that are relevant to the scenario. By taking this larger, holistic, view the method aims:
•	to allow decision makers to consider many different ways in which the human and machine might interact to deliver any individual function;
•	to ensure that allocation decisions are informed by the working context including aspects of the operator’s role and the possibility of emergent work activities;
•	to explicate some of the organisational trade-offs between the design options suggested for different functions.
3.1.	Overview of the method
Figure 1 shows the structure of the proposed allocation process. 
The process begins by identifying the functions that must be delivered at the systems level, and identifying any functions that necessarily must be allocated entirely to the machine (e.g. maintaining level flight in a aerodynamically unstable aircraft), or for which there is an externally governed limit on the degree of automation that will be acceptable (e.g. authorising the use of weapons in military aircraft). Since these processes are common in other allocation methods (see, for example, Corbridge & Cook (1997), Pulliam & Price (1985), Meister (1985)) we shall not discuss them in this paper. 
Our method diverges from previous methods at the third step where the role of the operator is defined taking into account the mandatory allocations.
A small number of scenarios are developed that describe situations in which the new system might be used.
The method then treats each scenario in turn. Within each scenario the following steps are conducted.
1)	The set of functions required in the scenario is identified.
2)	A subset of functions is identified as candidates for total automation. This is achieved by considering the research and development that would be required to achieve total automation, and examining  the relationship between the function and the operator’s  role. The comparison of alternative candidate functions is supported by using a two dimensional grid. 
3)	A set of possible design options for partial automation of the remaining functions is generated.
4)	These options are assessed by rating the research and development ‘costs’ associated with them, and the  possible ‘benefits’ that they might offer. In particular, the method considers benefits (and possible disbenefits) with respect to workload, performance, and situation awareness.
FIGURE 1 ABOUT HERE
5)	Options for partial automation are selected using a two dimensional grid. 
6)	The emergent consequences of the proposed allocations are considered, and any emergent activities identified. If emergent activities are identified then steps 4 and 5 are repeated, with any proposals that may be available to partially automate the emergent tasks.
After each scenario has been considered, contradictions in the allocation suggested by different scenarios are used to identify functions for which dynamic allocation may be appropriate. 
The allocation suggested for the whole system is reviewed, and if necessary the process can be iterated introducing additional scenarios, or refining existing scenarios. 
In the rest of this section, we focus on the steps of defining the operator’s role, authoring scenarios, and analysis of individual scenarios. 
3.2.	Defining the role of the operator
The first  step of the method after identification of the required functions and mandatory allocation is to define the ‘role’ of the operator. The importance of having a high-level understanding of the operator’s role has been previously identified by Clegg et al. (1996) and Goom (1996).
To develop the role statement, a hierarchy over the set of functions is constructed in which functions that appear to be related to the same high level goal are grouped into a small number (10 to 20) of ‘abstract functions’. It should be possible to give these abstract functions names that are meaningful to operators from the domain. In the context of a military aircraft, some possible abstract functions are ‘maintaining the mission plan’, ‘executing planned manoeuvres’, ‘responding to immediate threats’, ‘co-ordinating plans with other agents’. This set of high level abstract functions is used to create a statement of the role of the operator that indicates the abstract functions that will be his or her primary responsibility. This process is, necessarily, informed by knowledge of technological capabilities, human capabilities, social / political constraints, and concepts of accountability.
The role statement acts as a key element of the design philosophy. An example of a role statement developed for Boeing civil transport aircraft is: 
“The captain is the final authority for the operation of the airplane. Both crew members are responsible for the safe conduct of the flight. Flight crew must prioritise tasks according to the following hierarchy: safety, passenger comfort, and flight efficiency” (Higgins, 1996 [p. 6.2]).
In the context of military aircraft, a possible statement of the operator’s role in a modern military jet could be
The pilot is the final authority for the use of offensive weapons and for flight manoeuvres. The pilot is responsible for mission planning, tactical and strategic decision making and co-ordination with other aircraft.  The pilot is responsible for the management of systems to maximise the probability of successful mission completion.
The statement of operator role may include specific reference to some of the abstract functions identified, e.g.  co-ordination with other aircraft, but it may also use more general concepts of work within the domain, e.g. tactical and strategic decision making.
One possible criticism of this step in our allocation procedure is that it appears to move the problem of allocation rather than solve it. Instead of allocating functions, are we not simply allocating ‘abstract functions’ and thus open to the very criticisms that we have made in the foregoing discussion (section 2.5)? The advantage that we see in defining the operator’s role at this higher level of abstraction, where a small number of abstract functions (10 to 20) are identified, is that designers can consider the complete set of abstract functions together. When all the abstract functions are considered as a set, the design team can compare and select between alternative sets of abstract functions that could be used as the basis for the role, rather than considering each abstract function in isolation. This avoids the difficulties inherent in isolating functions (abstract or otherwise). 
3.3.	Creating scenarios
Because the aim of the scenarios is to force developers to confront the limitation on the resources (of both the human operator and the design organisation), scenarios should focus on situations where workload is likely to be high, where the complex cognitive work may be required, or where performance, safety or reliability might be compromised. 
Examples of useful scenarios for the civil aircraft domain could be situations immediately following systems failures or complex situations arising during approach and landing, e.g. a change of approach route. In the military aircraft domain, some scenarios would be set in combat situations. A scenario in the nuclear power domain, combining high workload with complex cognitive tasks, could be a loss of coolant accident in which latent failures hinder diagnosis and recovery.
Sources for developing scenarios may be: experiences of practitioners in previous systems; incident and accident reports on previous systems; frequent conditions that may occur in normal operation.
Scenarios are described using the set of headings below which were adapted from Fields et al. (1997).
1.	A list of humans in the system, their organisation and their primary roles.
2.	A rationale for developing the scenario.
3.	A description of the situation and environment in which the scenario occurs. In the military aircraft domain, the situation description could include information about: speed and direction of travel, intended future flightpath, location and direction of hostile forces, current fuel levels, or previous damage.
4.	A description of the assumed scenario baseline, i.e. the assumptions about the equipment available within the scenario. Is this a scenario based around a previous system, or is it a hypothetical scenario involving some technologies that have been proposed for the new system? The baselines used for scenarios should be restricted to technologies that are either already available, or are judged to involve ‘low risk’ research and development. 
5.	A description of  exceptional circumstances that could lead to the scenario evolving differently, e.g. possible variations in the behaviour of agents external to the human-machine system under consideration, possible uncertainty in the environment or possible additional system failures. 
Each scenario is analysed to produce a list of actions that the human-machine system must carry out. Actions in the list should treat the human-machine system as a ‘black-box’, i.e. they should not implicitly or explicitly assume any particular allocation of functions. Thus, each action should correspond to one (or possibly more than one) of the system functions. Any action that cannot be related to a (previously identified) system function should lead to the introduction of a new function to the system. The list of functions is taken forward to the next stage of the allocation process.
Because the scenarios are the main driver for the allocation process, it is important to ensure that a wide enough range of scenarios is considered to provide confidence in the allocation decisions made. On the other hand, the effort required for the allocation process increases as more scenarios are analysed. A basic minimum requirement is that every function identified in the initial requirements specification should be represented in at least one scenario. After this minimal number of scenarios has been considered, it may be useful to create additional scenarios to validate earlier allocation decisions. One possible strategy would be to monitor how often a prior decision is challenged by a new scenario, and to cease generating scenarios when the frequency of such challenges drops below some acceptable limit.
3.4.	Selecting candidates for total automation
For each function identified in a scenario the following two questions are considered.
•	Firstly, how feasible is it to construct a totally automated system to deliver this function with an acceptable level of performance, reliability and safety? The answers are rated on an ordinal scale ranging from.
a)	current systems produced by this organisation or immediately available to this organisation meet this objective;
b)	current systems on other (e.g. competitor) platforms meet this objective but we do not have immediate access to these solutions;
c)	this objective is likely to be achievable by means of a low cost, low risk research and development effort;
d)	this objective may be achievable on the basis of a high cost or high risk research and development effort;
e)	this objective is appears to be infeasible within the project timetable. 
•	Secondly, how closely related is the performance of this function to the identified role of the operator ? This question is designed to determine how desirable a totally automated solution will be from a human factors perspective. The answers are rated as follows.
a)	this function is separable from the operator’s role and does not interact with the performance of the operator’s role in this scenario;
b)	this function provides information or involves control for which it may be desirable to co-ordinate actions with activities that are within the operator’s role;
c)	this function provides information or involves control which critically affects the operator’s performance of their role;
d)	this function is central to the operator’s role in this scenario.
The two ratings for each function can be recorded by entering each function into the matrix shown in table 2.
State of Automation Research 
vs. Relation to pilot role.	Existing with immediate access
(a)	Existing in competitor systems

(b)	Low risk/ low cost R&D

(c)	High risk or high cost R&D

(d)	Infeasible

(e)	
Separable      (a)	1	2	3	4	13	
Role related information or control           (b)	5	6	7	8	14	
Role critical information or control           (c)	9	10	11	12	15	
Central to role
                     (d)	16	17	18	19	20	
Table 2: A matrix for selecting functions for total automation.
The matrix is used to determine which functions should have totally automated solutions developed for them. Functions appearing in cells at the top left of the matrix are good candidates for total automation. Functions appearing close to the bottom right are poor candidates. 
The designers select a list of functions, starting at the top left of the matrix, for which totally automated solutions will be applied. The remainder of the functions will be passed to the next stage of analysis. If a function is proposed for total automation which requires high risk research and development, then the function should still be carried forward to the next stage of analysis so that a possible ‘fall back’ allocation, involving lower project risks, can be identified.
This selection strategy reflects our economic model by treating the feasibility of automation as a measure of cost, and using the closeness of the function to the operator’s role as a (negative) measure of benefit, i.e. the more closely a function is related to the operator’s role, the lower the benefit that will be delivered by total automation. 
The pattern of functions that are selected for total automation may indicate something about the philosophy of the design organisation. For instance, an organisation that is highly technology driven might prefer to automate functions in cells 5, 6, 9 and 10 before considering functions in cells 3 and 4. In contrast, a human-centred design organisation might prefer to allocate research resources to functions in cell 4, rather than accept automation of functions in cells 5 and 6.
3.5.	Selecting candidates for partial automation
The analysis of options for partial automation proceeds through the following sequence of steps.
1)	The design used to deliver each function in the scenario baseline is recorded. The recording of the baseline design can be made using any appropriate representation, e.g. free text descriptions as used by Meister (1985), or a level of automation on an ordinal scale such as those outlined by Billings (1996), Sheridan & Verplanck (1978). Within our current work with BAe we are developing a generic framework for describing functions specifically designed to support designers in generating new design options (see Dearden et al., 1998).
2)	The designers generate a set of new design options for partial automation for each function. The technical feasibility of these options is considered using the same ordinal scale as for total automation. Options that are judged to be infeasible are discarded. The options of total automation and ‘no change’ from the baseline are also included at this stage.
3)	The designers consider the expected effect of each option on three parameters within the scenario: workload, situation awareness and performance (including safety and reliability). With respect to each parameter the design option is rated on an ordinal scale from: large improvement, improvement, no change, deterioration, large deterioration. The option describing the ‘baseline case’ will be rated as ‘no change’ with respect to all three parameters. A reduction in workload is taken to be an improvement in this scheme. Note that this rating should consider how the design option affects the workload, awareness or performance within the scenario, rather with respect to the individual function. For example, if the performance with respect to a function was improved, but that function was not particularly important within the scenario, then that might be judged as an improvement, but not a large improvement. Similarly a 50% reduction in workload achieved for a function that represents only a small part of the total workload for the scenario would be considered an improvement, but not a large improvement. If total automation is considered as an option and the function was judged in the previous stage to be closely related to the pilot’s role, then this assessment should be reflected in some deterioration with respect to situation awareness and / or performance. 
4)	The scenario is examined to see which aspects of the situation are causing the greatest concern. Is there a pressing need to reduce the operator’s workload within the scenario (given the tools available in the scenario baseline), or should a higher priority be attached to improving performance or to situation awareness? 
5)	When all the options for all the functions have been rated and the primary concern identified, the options are entered into a second matrix (see table 3). The design options used in the baseline should also be included in the table (in cells 17, 18 or 19). The designers start searching from the top left of the matrix selecting new design options. If a design option for a function is selected, then all other options for that function are deleted from the table. Any design options appearing in cells twenty-one to twenty-four should be removed from consideration. If a design option is selected from the ‘high risk research and development’ column, then an alternative, low risk solution should also be considered as a ‘fall-back’ position.
After a number of options for functions in cells one to three and five to seven have been selected, the designers should re-evaluate the scenario and consider whether or not the primary concern should be changed as a result of the decisions made so far. For instance, consider a scenario in which high workload is the primary concern. If new partially automated solutions are selected that significantly reduce the expected workload then a different concern such as performance or situation awareness may now be more significant. If the primary concern is changed, then options for the remaining functions are re-arranged in a  new matrix reflecting the changed priorities. Option selection then proceeds as before, starting with the options that provide the greatest improvement for the new primary concern. This procedure iterates until one design option has been selected for every function.
This selection strategy reflects our economic model by considering the ratio between the marginal cost of developing automation (represented here by the assessment of the research and development effort required for the proposed design option), and the marginal benefit of the automation (represented by the comparison of each option to the scenario baseline in terms of workload, situation awareness and performance). The method handles these three different aspects of benefit independently, selecting one measure as the primary focus at any stage, rather than attempting to aggregate them into a single measure. This approach avoids any requirement to establish ratio scale measures for each factor, or to make strong assumptions about their independence.
One possible outcome of the procedure is that some functions cannot be successfully allocated without making use of options from the high risk research and development column, or from a row involving a ‘large deterioration’ with respect to a secondary concern within the scenario. If this occurs frequently, and cannot be solved by generating alternative design options, this may indicate a need to review the system requirements, or to review assumptions about the number and role of human operators in the system. 
	Suggestion is immediately available	Available on competitor systems	Low risk, low cost R& D	High risk or high cost R&D	
Large improvement in primary concern, no deterioration in secondary concerns	1	2	3	4	
Improvement in primary concern, no deterioration in secondary concerns	5	6	7	8	
Improvement in primary concern, minor deterioration in secondary concerns	9	10	11	12	
Improvement in primary concern, large deterioration in secondary concerns	13	14	15	16	
No significant improvement in primary concern	17	18	19	20	
Large deterioration in primary concern	21	22	23	24	
Table 3: Matrix for partially automated solutions
3.6.	Emergent Activities
When a design option for each function has been selected, the scenario is re-analysed with a new baseline based on the proposed allocations. The purpose of this re-analysis is to identify any new activities that may be an emergent consequence of the new design. If new activities are identified, then designers must consider whether the impact of these activities upon performance, workload and situation awareness is acceptable. If the emergent activities do create an unacceptable situation, then the selection matrix is re-visited, including any options that might improve outcomes for the emergent tasks. This may result in the selection of automated aids to support the emergent tasks, or may result in changed selections for the original functions. 
3.7.	Dynamic allocation
As design options for functions within individual scenarios are selected, the recommendations are reviewed to identify functions where different allocations are recommended in different scenarios. Contradictions of this kind may be resolved by recourse to a dynamic allocation solution (either an adaptive or an adaptable allocation method). If dynamic allocation mechanisms are proposed then a safety analysis of the scenarios should examine:
a)	what consequences might occur if the scenario was encountered but the system was in the wrong configuration (i.e. configured for the wrong allocation of functions)
b)	the likelihood that such an incorrect configuration could occur.
Alternatively, contradictions may be resolved by either:
a)	prioritising one scenario over another, thus requiring the designers to re-consider other allocations within scenario that has been over-ruled.
b)	searching for a compromise solution that is not ideal for either of the two conflicting scenarios, but allows both scenarios to be successfully accommodated. Again this would require some re-examination of the scenarios to see whether other changes to the allocations are required.
4.	An example of scenario based allocation
In this section, we illustrate the proposed process by considering the analysis of a single scenario. We have used the method to examine a number of real scenarios involving allocation for future military aircraft, however, we cannot present these for security reasons. Instead, we have chosen a fictitious scenario in a civil aircraft domain. The scenario shares some similarities with the scenario examined by Fields et al (1997) discussed in section 2.5.
4.1.	Initial analysis of the scenario
The scenario is part of a (hypothetical) exploration of the use of a single pilot design for a large body (300+ passengers) civil transport aircraft. Whilst the use of a single pilot in such aircraft is not permitted at the current time, improved reliability of aircraft in the future, and developments such as DataLink might lead to economic pressures for such systems in the future. Would such an aircraft be technically feasible, and what would the automation requirements be? The scenario deals with the problems that might occur following ingestion of a foreign object into an engine during take-off. In figure 2, we present the scenario using the structure given in section 3.3.
FIGURE 2 ABOUT HERE
4.1.1.	Role selected
As a ‘role’ model we have adapted the philosophy outlined by Higgins (1996). Our statement of role is:
The captain is the final authority for the operation of the airplane. The captain is responsible for the safe conduct of the flight and its co-ordination with other air  traffic. The captain must prioritise tasks according to the following hierarchy: safety, passenger comfort, and flight efficiency.
4.1.2.	Functions identified
Within the scenario we identified the following functions:
1)	detect engine problem
2)	diagnose engine problem
3)	set thrust on affected engine (to balance need to climb and desire to prevent further engine damage)
4)	maximise thrust on unaffected engine
5)	retract landing gear (to reduce drag)
6)	configure flaps and slats to balance requirements for lift and drag
7)	set appropriate angle of attack (to continue climb and avoid stall)
8)	trim aircraft for uneven thrust
9)	idle affected engine
10)	extinguish engine fires
11)	start auxiliary power unit (to provide backup electrical power)
12)	communicate nature of problem to air-traffic control
13)	request emergency return to departure airport
14)	revise flightplan to return to departure airport
15)	manoeuvre aircraft to follow revised flightplan maintaining separations
16)	inform cabin crew/passengers
4.2.	Identifying candidates for total automation
Taking the list of functions above, and considering the role of the pilot as described, we entered the (numbered) functions into the matrix as shown in table 4. 
State of Automation Research 
Versus
Relation to pilot role.	Existing with immediate access	Existing in competitor systems	High confidence/ low cost research	Low confidence or high cost research	Infeasible	
Separable	8		11, 16			
Role related information or control	1, 2		12			
Role critical information or control	4, 5, 9, 10, 15					
Central to role	7		3, 6	13, 14		
Table 4: Evaluating proposals for total automation in the scenario
Although we shall not justify all of our classifications, some of them deserve comment. For example, we concluded that, although it is perfectly feasible to set angle of attack to maximise the climb rate for the current thrust automatically, whilst avoiding a stall, this function is central to the pilots role as the final authority with responsibility for flight safety, so the pilot needs some level of control. Similarly, although low cost research and development might provide automated systems that can handle the thrust for the affected engine and configure flaps and slats for climbing, we decided that these highly contingent trade-off decisions are central to the pilot’s role as the final authority for safe conduct of the flight, and therefore should not be totally automated. We spent some time discussing whether the decision about retracting the landing gear was ‘critical’ to the pilots’ role, or merely ‘relevant’. Based on the reporting of two accidents in which pilots decided (correctly in hindsight) to abort take-offs and crash land in similar situations, we decided that this decision was in fact critical to safe operation, and therefore should not be fully automated.
Another feature of our analysis is the relationship between the pilot and air-traffic control that it suggests. On the one hand, we felt that requesting an emergency return to the departure airport (13) was central to the pilot’s role in co-ordinating with other air traffic, however, informing controllers of the precise nature of the problem (12) was not central to the role, rather it involves information that is related to co-ordination. This suggests that DataLink facilities might be worth considering for this function. Similarly, informing the passengers of the problem might be offloaded from the pilot, perhaps by automatically informing the cabin crew of the problem. 
Based on the entries in table 4, we decided that functions of trimming the aircraft (8), starting the auxiliary power unit (11) and informing the cabin crew/passengers (16)  should be totally automated, as should functions (1) detecting the engine problem, (2) diagnosing the problem, and (11) notifying air-traffic control of the incident. Because these latter functions (1,2 & 12) involve information that is relevant to the pilots’ role, it is important that designs for these functions provide adequate feedback to the pilot about what has occurred. 
Functions 3, 4, 5, 6, 7, 9, 10, 12, 13 and 14 were carried forwards to the next stage of analysis.
4.3.	Selecting options for partial automation
In the baseline scenario, the pilot can take direct control over the three main flight parameters of engine settings, wing configuration and angle of attack (functions 3, 4, 6 &  7). Alternatively, these parameters can be left under the control of autopilot and autothrottle systems, which themselves are controlled by the flight management system. The autopilot can be set to a mode such that it will not accept any climb commands that exceed its capabilities when operating on a single engine. Raising the landing gear (function 5) is achieved by a single command. The engine idling and fire extinguishing (functions 9 & 10) are achieved by following a checklist given by a centralised aircraft monitoring and warning system. Obtaining clearance (function 13) is achieved by the pilot talking to air-traffic control, revising the route and then flying the route (functions 14 and 15) can be achieved either: by programming the new route into the flight management system which then uses the autopilot to fly the route; by the pilot navigating the route and commanding the autopilot for each leg; or by the pilot flying manually. An emergent task within the scenario is the co-ordination of the various recovery actions (e.g. ordering the fire recovery and APU start procedures, actions to reduce drag such as raising the landing gear and selecting an appropriate configuration of flaps and slats).
Various changes from this baseline were suggested, we list some below.
Option 1)	The engine idling (9), fire extinguishing (10) and starting the APU procedure could be either:
a)	wrapped up into a single command, with an automated system reporting whether the fire has been successfully extinguished, and whether restarting the engine is possible; or
b)	conducted fully automatically where the pilot does not intervene at all.
Option 2)	a new route to return to the departure airport (13 & 14) could be either:
a)	suggested by an automatic route planning system, taking into account other traffic, with the pilot able to modify the proposed route, this system would also inform air-traffic control of the proposed plan using DataLink; or
b)	generated by air-traffic controllers and uploaded across a DataLink to the flight management system.
These solutions would allow the route to be flown automatically (15).
Option 3)	An automated system could calculate the optimum configuration of wing flaps and slats for maximum climb rate given the thrust of one engine, taking into account current airspeed, weight, altitude etc. (functions 6 & 7) and either:
a)	offer this to the pilot to accept or reject; or 
b)	automatically configure the wing in this way. 
Option 4)	The emergent task of co-ordination of the functions of raising the landing gear, configuring the wing and managing the fire could be supported by allowing the pilot to command all of these actions from the same point in the interface, e.g. by interacting with a centralised problem management system which would offer all three functions on a single screen, the pilot selecting each one in turn by a single keystroke.
Option 5)	The autothrottle could have a facility to manage the thrust from the aircraft as a single parameter (rather than independent control of the two engines) delivering the maximum safe thrust given the engine state (functions 3 & 4).
Taking workload as the primary concern in the scenario, the design options above were assessed and placed in the matrix as shown in table 5.

	Suggestion is immediately available	Available on competitor systems	Low risk, low cost R& D
	High risk or high cost R&D	
Large improvement in primary concern, no deterioration in secondaries			Option 3a (system suggests wing config)		
Some improvement in primary concern, no deterioration in secondaries	Option 4 (centralised presentation of gear, wing & fire actions)			Option 2a 
(on-board auto route planner)	
Improvement in primary concern, minor deterioration in secondaries		Option 1a (single fire drill command)
	Option 2b (ATC provide new route via DataLink)
		
Improvement in primary concern, large deterioration in secondaries 		Option 1b 
(full auto fire extinguish)	Option 5 
(full auto thrust management) 
Option 3b (full auto wing config)		
No significant improvement in primary concern	Baseline design for functions 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, & 14			
	
Large deterioration in primary concern					
Table 5: Evaluating options for partially automated support in the scenario
Our analysis of the options suggests that the developers should examine:
1)	option 1a - automating the execution of the whole fire extinguishing & APU start procedure – option 1b is rejected because the workload improvement (saving a single command) is not sufficient to justify the deterioration in situation awareness that might follow;
2)	option 3a - automatically calculating an ‘optimal configuration’ for climbing given the reduced thrust, and offering this configuration to the pilot – option 3b is rejected on the grounds of reduced situation awareness;
3)	option 4 – a centralised problem management system that allowed the pilot to command various actions directly from the same location in the interface;
4)	option 2a – an on-board auto-route planner that could take into account the current system condition – this option offers a greater reduction in workload than using the ATC to plan the new route, although it may require more research than an ATC based solution. This is because an on-board system would have more information available about the specific capabilities of the aircraft after the engine fire.
Option 5, the autothrust management is rejected on the grounds that it would significantly reduce the pilots situation awareness since the single thrust control might mask the existence of the engine problem.
4.4.	Summary
In this section, we have illustrated some parts of the proposed method using a hypothetical case study. The case study serves to illustrate the method, rather than to evaluate it, and the single scenario does not illustrate all features of the method. For instance, we have not examined the way in which the role statement was constructed, neither, since we have only considered one scenario, have we examined the handling of possibilities for dynamic allocation.
5.	Discussion
In this section, we discuss the relationship between our method and the review of existing allocation techniques developed in section 2.
5.1.	Characterising different allocation alternatives
The proposed method allows the allocation team to specify the way that a function is shared between human and machine at any level of granularity that the team regard as appropriate. In the case study in section 4, the design alternatives were represented by short textual descriptions. In Dearden et al (1998) we propose a generic framework that allows designers to explore the different ways in which a function might be shared. In our work with British Aerospace we are using this framework to specify the design options that are selected. Users of the method might also wish to use Billings’ (1996) concept of 7 levels of automation, or Sheridan & Verplanck’s (1978) 10 level model. Thus our model is at least as flexible as existing practices.
5.2.	Selecting and combining criteria in allocation decisions
The proposed method contrasts with some recent work in function allocation (e.g. Meister, 1985; Clegg et al., 1996; Papantonopoulos & Salvendy, 1993) in that we have selected a small number of criteria, and do not seek to use sophisticated mathematical tools to combine these criteria into a global assessment. This decision is based on a desire to avoid making strong assumptions about the nature of the preference criteria, e.g. that ratio scales are appropriate (which is unlikely given the high degree of uncertainty surrounding the design at this early stage), or that preferential independence holds (see Vincke, 1992, for a discussion of the role of this assumption in decision theory). 
In considering whether total automation is appropriate, we apply only two criteria – an aggregated assessment of the cost or difficulty of researching & developing a solution, and the relation between the function and the operator’s assigned role. 
In selecting options for partial automation, we again aggregate the costs and difficulty of research and development, and we seek to avoid aggregating the ‘potential benefits’ with respect to workload, situation awareness and performance. Our choice of these three criteria is guided by the following observations:
1)	for any given design option, the abstraction ‘performance’ can be regarded as an attribute of the functions that the design option addresses directly;
2)	the abstraction ‘workload’ captures factors that will have an immediate impact on the ability of the human operator to deal with other functions that are happening concurrently;
3)	the ‘situation awareness’ abstraction captures factors that could have an impact on the operators capabilities with respect to other functions that occur at other times outside of the scope of the scenario being analysed.
5.3.	The search procedure
The search procedure in the new method examines sets of functions together, based on their co-occurrence within scenarios. This approach seeks to avoid the pitfalls of treating functions in isolation, and the complexities of dealing with all the functions in the system in one session. Our aim is to decompose the design problem into manageable units, without hiding the complex interactions between functions.
5.4.	The need for iteration
Our method seeks to reduce the number of iterations required in allocation by dealing with the trade-offs between functions within the scenarios. The procedure does not avoid iteration altogether. Two possible causes of iteration are:
1)	different scenarios may call for conflicting design solutions, in which case some re-evaluation of one of the scenarios may be required, or dynamic allocation may be considered;
2)	the total amount of expensive and / or risky research and development called for by teams working on different scenarios may be unacceptable, in which case some proposals may need to be modified and associated scenarios re-examined.
However, we contend that the number and complexity of the iterations should be limited, and that the iterations should converge towards a feasible solution (or will expose the infeasibility of the stated system requirements).
Our approach can be related to Fuld’s (1993) recommendation that human-factors should be relegated to reviewing designs for misfit. However, we place this human-factors review activity firmly within the initial process of selecting conceptual designs. For example, we note that:
•	the starting point of the method is the identification of scenarios that might exhibit misfits between the human and the automation – scenarios are chosen because they involve high workload or safety and performance challenges;
•	the method seeks to identify misfits between conceptual design proposals and the operator’s role, workload and situation awareness.
As such the method offers a principled way in which human factors concerns can be brought to bear upon the earliest stages of conceptual design, where the costs associated with design modifications are minimised.
5.5.	Regarding work context
Our critique highlighted the need to take the operator’s role, and the work context into account. Our use of scenarios aims to make an understanding of work context explicit within the decision making process. 
The treatment of context within the method requires some qualification. Studies of work in context reveal the complexity of the relationships between details of the work context and human performance (see Wright et al., this volume). It is inevitable that a method that seeks to envisage a work context that will not be created for another 10 years must abstract away from some of the rich detail that will affect the work and the human’s performance within the work. It might be possible to use a more detailed scenario description including more assumptions about the social context. However, extending the detail of the scenario would have some drawbacks, e.g. it would increase the complexity when presenting the analysis to a group of designers working on allocation; also, it may require additional assumptions about the nature of the working environment which may (or may not) be reflected in the eventual working practices that surround the system. Our proposed scenario syntax represents one possible trade-off between the conflicting goals of minimising complexity and maximising the completeness of analysis.
5.6.	Revealing organisational trade-offs
The method centres around the use of the two grids to support trade-off decision making by the systems designers, human-factors experts and domain experts engaged in the allocation process. The grid maps the organisational trade-offs to physical dimensions, with the top left corner of the grid representing the most desirable design options and the bottom right representing the least desirable outcomes. The decisions made within each scenario must take into account the other allocations that have already been made, thus making such organisational trade-offs explicit.
6.	Conclusion and further work
In this paper, we have presented an alternative approach to the allocation of function problem that reframes the allocation question. Rather than asking the question:
‘To which agent, human or machine, should we allocate this function ?’, 
we pose the question:
‘To which functions, required of the system in this context, should we allocate the limited resources of this agent, human or development organisation?’.
Our reasoning for restating the focus of allocation is based on two observations. 
•	Firstly, studies of the impact of technology on work practice demonstrate that designers need to take the context of use into account.
•	Secondly, basic results in economic utility theory demonstrate that optimal decisions regarding the use of resources must consider the relationship between: the marginal benefit and marginal cost implied by a design proposal; and the benefits and costs associated with proposals that relate to other functions. 
The method we have presented aims to support design teams in applying these observations. As such, we have been concerned with developing a method that will be relatively easy to learn, easy to use and easy to understand in the hands of designers. Use of the method generates both an allocation decision for each function, and a rationale for that decision, in terms of the relation to existing systems (the baseline scenarios), and to rejected design options. Additionally, the method provides a mechanism which might alert systems designers to potential (project) risks in their design.
Our future research will be concerned with evaluating the use of the method in practice and with extending the method to multi-operator environments. We hope to examine:
•	the ease (or difficulty) encountered by systems designers in learning to use the method;
•	the effect the method has on the time and effort required to perform allocation for whole systems;
•	the degree of iteration that is required when global trade-offs are considered;
•	the situations in which recommendations generated using the method differ from those using existing allocation techniques;
•	an appropriate method for making decisions about dynamic allocation, and particularly about the question of whether the human or machine should make the dynamic allocation decision;
•	extensions of the method to multi-operator environments;
•	ways of adapting the method to later allocation decisions, for instance for determining the division of work when new subsystems are proposed for integration into existing systems (e.g. mid-life updates). 
7.	Acknowledgements
This work was conducted within the Centre of Excellence for Cockpit Automation Research at the University of York. The authors gratefully acknowledge the financial support and active collaboration of British Aerospace Military Aircraft & Aerostructures Ltd. In particular, we would like to thank Peter Gosling of the Cockpit R&D Group at BAe MA&A for his participation in the development of this work. Also we wish to thank our anonymous reviewers for their helpful suggestions.
8.	References
Beyer, H. and Holtzblatt, K., 1997. Contextual Design. San Francisco, CA.: Morgan Kaufmann.
Billings, C.E. 1996. Aviation Automation: the search for a human-centred approach. Hillsdale, NJ.: Lawrence Erlbaum.
Beevis, D., Essens, P. and Schuffel, H. (Eds.), 1996. Improving Function Allocation for Integrated Systems Design. Technical report  CSERIAC SOAR 96-01. Crew Systems Ergonomics Information Analysis Centre, Wright-Patterson Airforce Base, Ohio.
Beevis, D., Essens, P., 1997. The NATO Defence Research Group Workshop on Function Allocation. In Fallon, E., Bannon, L. and McCarthy, J. (Eds.) ALLFN’97 Revisiting the Allocation of Functions Issue: New Perspectives. pp. 1 - 15. Louisville, KY.:IEA Press. 
Booher, H. R. (Ed.), 1990. MANPRINT: An approach to systems integration. New York: Van Nostrad, Reinhold.
Carroll, J.M. (Ed.), 1995. Scenario-based design: envisioning work and technology in system development. New York: Wiley.
Clegg, C., Coleman, P., Hornby, P. , MacLaren, R., Robson, J., Carey, N. and Symon, G., 1996. Tools to incorporate some psychological and organisational issues during the development of computer based systems. Ergonomics, 39, pp 482 – 511. 
Cook, C.A. and Corbridge, C., 1997. Tasks or Functions: What are we allocating ? In Fallon, E., Bannon, L. and McCarthy, J. (Eds.) ALLFN’97 Revisiting the Allocation of Functions Issue: New Perspectives.  pp. 115 - 124. Louisville, KY.: IEA Press.
Corbridge, C. and Cook, C.A., 1997. The Role of Function Allocation in the Design of Future Naval Systems. In Fallon, E., Bannon, L. and McCarthy, J. (Eds.) ALLFN’97 Revisiting the Allocation of Functions Issue: New Perspectives. pp. 73 – 88. Louisville, KY.: IEA Press.
Dearden, A. M., Harrison, M. D., Wright, P. C. and Gosling, P., 1998. IDA-S: A new framework for understanding function allocations.. University of York / BAe Centre of Excellence for Cockpit Automation Research. Technical report, COE/TR/98/03.
Dekker, S.W. and Wright, P.C., 1997. Function Allocation: A Question of Task Transformation not Allocation. In Fallon, E., Bannon, L. and McCarthy, J. (Eds.) ALLFN’97 Revisiting the Allocation of Functions Issue: New Perspectives. pp. 31 – 40. Louisville, KY.: IEA Press.
Denley, I., 1989. A method for function allocation in advanced manufacturing technology. In Megaw, E.D. (ed.), Contemporary Ergonomics. pp 55 – 61. London: Taylor & Francis.
Fields, B., Harrison, M. and Wright, P., 1997. THEA Human Error Analysis for Requirements Definition. Technical Report YCS 294, Department of Computer Science, University of York.
Fitts, P.M., 1951. Human engineering for an effective air navigation and traffic control system. (National Research Council, Washington D.C.) Reprinted as Appendix 1 in Beevis, D., Essens, P. and Schuffel, H. (Eds.), 1996. Improving Function Allocation for Integrated Systems Design. Technical report  CSERIAC SOAR 96-01. Crew Systems Ergonomics Information Analysis Centre, Wright-Patterson Airforce Base, Ohio.
Fuld, R.B., 1993. The fiction of function allocation. Ergonomics in Design, 1, January, pp. 20 – 24.
Fuld, R.B., 1997. The fiction of function allocation, revisited. In Fallon, E., Bannon, L. and McCarthy, J. (Eds.) ALLFN’97 Revisiting the Allocation of Functions Issue: New Perspectives. pp. 17 – 30. Louisville, KY.: IEA Press.
Gilb, T., 1997. Towards the Engineering of Requirements. Requirements Engineering Journal (1997) 2(3):165-169.
Goom, M.K., 1996. Function Allocation and MANPRINT. In, Beevis et al. (1996) pp. 45 – 61.
Gramopadhye, A. K., Drury, C.G., Sharit, J. and Sudit, M., 1992. A framework for function allocation in inspection, in Brodner, P. and Karwowski, W. (eds.), Ergonomics of Hybrid Automated Systems III. Elsevier, Amsterdam, pp. 249 – 257.
J.K. Higgins, 1996. Boeing’s Design Approach for Crew-Centred Flight Decks. In, The Future Flight Deck – Safe and User Friendly. London: Royal Aeronautical Society. ISBN 1-85768-037-5. 
Hutchins, E.L., 1995. Cognition in the Wild. Cambridge, MA.: MIT Press
Ip, W.K, Damordaran, L., Olphert, C.W. and Maguire, M. C., 1990. The Use of Task Allocation Charts in System Design: A Critical Appraisal. In D. Diaper, D. Gilmore, G. Cockton and B. Shackel (Eds.) Proceedings of Interact '90. pp289 – 294. Amsterdam: North Holland. 
Kuutti, K., 1995. Work Processes: Scenarios as a preliminary vocabulary. Ch. 1 in Carroll, 1995, pp. 109 – 134.
McCarthy, J., Wright, P.C., Healey, P.G.T. and Harrison, M. (1997) Human error in context: Accountability and work activity in high-consequence work systems. Int. J. Human Computer Systems. 47(6), pp. 735-766.
Meister, D., 1985. Behavioural Analysis and Measurement Methods. New York, NY.: John Wiley & Sons.
MOD, 1989. Defence Standard 00-25. Human Factors for Designers of Equipment. Part 12: Systems.UK Ministry of Defence
Nardi, B. (ed.), 1995. Context and Consciousness, activity theory and human-computer interaction. Cambridge, MA.: MIT Press.
Older, M.T., Waterson, P.E. and Clegg, C.W., 1997. A critical assessment of task allocation methods and their applicability. Ergonomics, 40(2), pp 151 – 171.
Palmer, E.A., Hutchins, E.L., Ritter, R.D, and van Cleemput, I., 1993. Altitude Deviations: Breakdowns in an Error Tolerant System. NASA Technical Memorandum DOT/FAA/RD-92/7.
Palmer, M.T.,  Rogers, W.H, Press, H.N., Latorella, K.A. and Abbott, T.S., 1995. A Crew-Centred Flight Deck Design Philosophy for High Speed Civil Transport (HSCT) Aircraft. NASA Technical memorandum, 109171. NASA Langley Research Centre.
Papantonopoulos, S. A. and Salvendy, G., 1993. A decision analytic model for cognitive task analysis. Proceedings of the Human Factors and Ergonomics Society, 37th Annual meeting, pp. 302 – 396.
Parkin, M. and King, D., 1995. Economics (2nd Edition). Wokingham, England: Addison Wesley.
Pulliam, R., and Price, H.E., 1984. Allocating Functions to Man or Machine in Nuclear Power Plant Control. The Nuclear Engineer, 25 (3), pp. 79 – 85. 
Rencken, W.D. and Durrant-Whyte, H.F, 1993. A quantitative model for adaptive task allocation in human-computer interfaces. IEEE Transactions on Systems Man & Cybernetics, 23, pp. 1072 – 1090.
Sheridan, T.B. and Verplanck, W.L. 1978. Human and computer control of undersea teleoperators. Technical report. Man-machine systems lab, Dept of Mechanical Engineering, MIT, Cambridge, MA.
Shoval, S., Koren, Y. and Borenstein, J., 1993. Optimal task allocation in task agent control state space. In, Proceedings of the IEEE Conference on Systems, Man and Cybernetics. pp. 27 – 32. Piscataway, NJ.: IEEE Press. 
Suchman, L.A., 1993. Plans and situated actions. Cambridge, UK.: Cambridge University Press. 
Vincke, P., 1992. Multicriteria Decision-Aid. Chichester, UK. John Wiley & Sons.
C.P. Warren, 1997. Allocation of Function: A Brief Review. Report number JS13076, British Aerospace, Sowerby Research Centre.
Woods, D.D., Johannesen, L.J., Cook, R.I. and Sarter, N.B., 1994. Behind Human Error: Cognitive Systems, Computers, and Hindsight. Technical report: CSERIAC SOAR 94-01. Crew Systems Ergonomics Information Analysis Centre, Wright-Patterson Airforce Base, Ohio
Wright, P. C. , Dearden, A. M., Fields, B., 1999. Function Allocation: A perspective from studies of work practice. This volume.
Wright, P., Pocock, S., and Fields, B., 1998. The prescription and practice of work on the flight deck. In, Green, T.R.G., Bannon, L., Warren, C.P. and Buckley, J. (Eds.) ECCE 9:  Proceedings of the ninth European Conference on Cognitive Ergonomics. pp. 37 – 42. Le Chesnay, France: EACE. ISBN 1-874653-48-8.

8.1.1.	

8.1.2.	
8.1.3.	
8.1.4.	

Equation 1

Equation 2

Equation 3

Equation 4

Equation 5

Equation 6

Figure 1: Overview of the proposed allocation method

1) Humans in the system
Single pilot plus air-traffic control personnel.
2) Rationale for the scenario
Take-off is recognised as a high-workload phase of current civil flights
Various types of foreign object ingestion are possible, e.g. bird-strikes, ice, litter. If such ingestion occurs it may cause engine fires or engine failures. 
On 27 December, 1991, Scandinavian Airlines flight SK751, from Stockholm to Copenhagen, suffered wing ice ingestion shortly after departure. Both engines failed and the an emergency landing had to be made in a field. The aircraft broke in three pieces on impact. Fortunately, no lives were lost.
3) Situation and environment
The scenario assumes that the aircraft is departing from a busy modern civil airport (Manchester, UK) during good weather conditions. The aircraft is part of the fleet of  a large international carrier based with its operations centre based in the USA. 5 seconds into the climb, when the aircraft is at 250ft, a flock of geese is ingested into one engine.
4) Scenario baseline
The aircraft has two wing mounted engines, one on each wing. The aircraft is capable of climbing after take-off using a single engine. The aircraft and air traffic control are equipped with DataLink facilities for communicating flight plans and clearances. The aircraft has a flight management systems and autopilots with at least as much functionality as those in use in the current generation of glass-cockpit aircraft (e.g. Airbus A340, McDonnell Douglas MD-80, Boeing 777). The aircraft has centralised systems monitoring, warning and diagnostic software. 
5) Exceptional circumstances
In this paper we do not consider any exceptional circumstances. Possible circumstances that could be considered are: mountainous terrain in the area around the airport, failures of either DataLink or the automated diagnostics systems, or failures associated with ground maintenance.

Figure 2: An example scenario

1) Humans in the system
Single pilot plus air-traffic control personnel.
2) Rationale for the scenario
Take-off is recognised as a high-workload phase of current civil flights
Various types of foreign object ingestion are possible, e.g. bird-strikes, ice, litter. If such ingestion occurs it may cause engine fires or engine failures. 
On 27 December, 1991, Scandinavian Airlines flight SK751, from Stockholm to Copenhagen, suffered wing ice ingestion shortly after departure. Both engines failed and the an emergency landing had to be made in a field. The aircraft broke in three pieces on impact. Fortunately, no lives were lost.
3) Situation and environment
The scenario assumes that the aircraft is departing from a busy modern civil airport (Manchester, UK) during good weather conditions. The aircraft is part of the fleet of  a large international carrier based with its operations centre based in the USA. 5 seconds into the climb, when the aircraft is at 250ft, a flock of geese is ingested into one engine.
4) Scenario baseline
The aircraft has two wing mounted engines, one on each wing. The aircraft is capable of climbing after take-off using a single engine. The aircraft and air traffic control are equipped with DataLink facilities for communicating flight plans and clearances. The aircraft has a flight management systems and autopilots with at least as much functionality as those in use in the current generation of glass-cockpit aircraft (e.g. Airbus A340, McDonnell Douglas MD-80, Boeing 777). The aircraft has centralised systems monitoring, warning and diagnostic software. 
5) Exceptional circumstances
In this paper we do not consider any exceptional circumstances. Possible circumstances that could be considered are: mountainous terrain in the area around the airport, failures of either DataLink or the automated diagnostics systems, or failures associated with ground maintenance.