1.
An introduction to experience management
Human problem
solving in many fields is based on extensive experience. Experience
management is a special kind of knowledge management, focussing on
the dissemination of specific knowledge situated in a particular
problem-solving context (Bergmann 2002). For experience management,
information and communication technologies play an important role,
providing the connectivity that is required to share experience.
Pellucid is a
project tackling the management of experience in public organisations,
particularly those aspects of experience management related to
organisational mobility, the movement or circulation of staff from one
unit to another within an organisation (Lambert et al. 2003).
Organisational mobility is increasingly commonplace in public
organisations, and presents both opportunities and challenges:
opportunities for improving working practices through the introduction
of new perspectives, and challenges arising from the constant loss of
experience and the learning curve of the newly arrived staff.
The Pellucid
project is developing an adaptable, customisable software platform for
creating and integrating experience management systems in organisations.
The basic metaphor for experience management is of an intelligent
assistant that looks over one’s shoulder and answers questions one might
have at a particular point of work (Staab and Schnurr 2002). The
assistant detects that an employee is working in a particular context,
offering knowledge resources that facilitate their work according to
their expertise. To this end, the Pellucid platform integrates
technologies such as autonomous cooperating agents, organisational
memory, workflow and process modelling, and metadata for accessing
document repositories.
This paper
presents the experience management framework developed in Pellucid.
Section 2 introduces the three knowledge-intensive applications where
the framework will be installed. Section 3 describes our experience
management framework, it shows the main concepts that constitute our
experience management model and the challenges we tackled in its
development. Section 4 discuses an important component of the framework,
the modelling of the worker context. Section 5 examines integration
issues such as obtaining the context from external system as well as
document searching and similarity matching. Finally, section 6 concludes
the paper by highlighting main finding and outlining future work.
2.
Three applications for experience
management
Within the
Pellucid project, three pilot applications have been chosen in order to
test and to show the functionalities of the platform. The three end
users involved in the project are:
§
Mancomunidad de
Municipios del Bajo Guadalquivir (MMBG), an association of local
governments in the south of Spain.
§
Consejería de la
Presidencia (CPRE-JA), a body of the regional government of Andalusia
(Junta de Andalucía) in Spain, together with the company SADESI that
operates its call centre for telephony problems.
§
Municipality of
Genoa, Mobility and Transport Directorate (CdG), Italy.
These
organisations selected the following processes with experience
management needs:
§
Management of
projects and services (MMBG).
§
Management and
resolution of fixed telephony breakdowns in the call centre (CPRE-JA/SADESI).
§
The installation
of new traffic lights (CdG).
These
applications cover a wide range of processes existing within the public
organisations in several sectors. In this way the Pellucid platform will
be tested by different varieties of end users and will be addressed to
several employees. Moreover they cover different ranges of
organisational mobility. In the cases of MMBG and CdG, there are many
different mobility scenarios, in that there are many ways in which staff
can move from one position to another, whether vertically through
promotion or horizontally from another unit. For SADESI, the mobility
arises from the high level of staff turnover—a common situation in call
centres generally.
All the processes
have much experience to manage and also to capture and to return to new
employees: sometimes the experience is technical, and other times it is
more related to administrative issues and procedures. For example, it
might relate to who to contact with enquiries about a particular case,
or how long to expect a certain process to take before it needs
checking, or particular factors to take into account in special
circumstances.
In the case of
MMBG, the management of public funded projects includes all the tasks to
be performed from the very early stages of a project (definition of the
idea, preparation of the proposal and submission to the funding
authority) to the justification of the project’s costs and activities,
and the preservation of all the documents generated during the project
execution. The main problems to be addressed are due to the lack of
integration among the departments involved in the different project
phases, and to the non-existence of a solid information structure to
provide support for external and internal issues, such as contacting
potential providers or preparing a proposal. This is however not simply
an IT issue; it reflects the varied and ill-structured nature of the
work done. Thus staff find themselves without support—not only new
employees but also experienced staff who can be involved in very
different proposals and projects.
Employees in the
Call Centre of SADESI have to adapt in a very short time to a very
specific working environment, getting a slight ‘on-the-job’ training
support. These employees suffer from a very high mobility degree, and it
has been proven that, due to their working conditions, they typically
stay in their position only 6 months as an average. This very unstable
environment makes the need for systems for registering, keeping and
exploiting efficiently the experience and knowledge of the employees
before they leave the Call Centre or leave their jobs.
In the case of
CdG, the application related to the installation of a new traffic light
exemplifies the important role of experience in the effective execution
of the task. This procedure involves many actors and is made up of
several phases: a preliminary analysis following a request for a new
traffic light; verification of information; design in detail the plant;
definitive evaluation of all aspects of the project; definition of
financial and administrative aspects; installation of the traffic light;
and finally, testing and maintenance. A major source of knowledge
management problems is the large number of actors involved, and the need
for co-ordination among the several departments in order to get the
optimal final result. Less experienced employees have less awareness of
who they need to contact at what stage among the external actors, how
this contact should be made (by email, by letter, whether there is a
standard form for a letter,), and how long the actor normally takes to
reply. These are all examples of knowledge that an experienced employee
will have acquired, and they should be shared effectively.
The main benefits
expected from the Pellucid project are the improvement in efficiency and
effectiveness due to the reduction of time spent, and leveraging of
experience due to the movement of staff among different roles.
3.
Dissemination of experience in practice
3.1
The Pellucid experience management model
The experience
management model in Pellucid exploits the experience sharing concepts
expressed in (Bergmann 2002). It is based on two main ideas: every
employee in an organisation is both a provider and a user of experience;
and employees communicate their experience to a common medium, then
retrieving experience in a future from this common medium. Pellucid aims
to be such a common medium.
The Pellucid
experience management model comprises three phases: Capture and Store,
Analysis and Presentation, and Experience Evolution.
The Capture
and Store phase is concerned with observing and storing experience
in a particular context. There are three ways of capturing experience:
analysing employees’ actions and workflow events, analysing documents
entered into the system, and by direct input from workers. Capturing
experience from working actions and events is particularly beneficial in
repetitive tasks; they are used to create common patterns that can be
retrieved in the future in order to assist other employees. Documents
constitute an important asset in an organisation. Metadata is added to
documents, enabling the system to retrieve in an automatic way documents
useful in a particular working context. The direct capture of experience
from employees is carried out through free-text notes written by the
employees themselves. This constitutes a good source of knowledge,
particularly in the transmission of experience from experienced
employees to novices.
The purpose of
Pellucid is to support and enhance employees’ performance by providing
them with the knowledge required by the activity they are performing at
the time they are actually performing the activity. The Analysis and
Presentation phase is concerned with providing such knowledge. To do
so, the concept of an active hint is introduced, a representation
of experience within the organisation. An active hint is triggered in a
context and includes an action, a knowledge resource and a justification
for the hint. The context is determined by the particular activity that
is carried out by the employee at that time in a workflow system. An
action corresponds to an atomic act on a knowledge resource, for example
use a document template, read a document or a note, or consider a
contact list. The justification gives to the employee a reason for the
hint. The idea of active hints was borrowed from the DECOR project (Abecker
et al. 2002) and worked out in a somewhat different way in Pellucid. Let
us now describe the stages included in this phase:
§
An employee is
performing a particular activity. The system tries to match the
current working context with other context stored in the organisational
memory. In general, every past context is stored with possible solutions
(hints) to assist employees in that particular situation.
§
Rarely (if not
ever) a perfect match occurs. So the system should be able to adapt the
previous context to the new one as well as the past solution.
§
The system
suggests the past solution (maybe adapted) in the form of hint.
§
The user is free
to follow or not the hint given by the system. The user should be able
to communicate this decision to the system which:
°
If the hint is followed,
it also contains a judgment of the proposed solution (it worked
well/bad)
°
If the hint is not
followed, it also contains a motivation for it (I did not follow it
because ...)
§
The system should
store this information, classifying them maybe with keywords or other
techniques.
The aim of
Experience Evolution is updating the available experience. Due to
the rapidly changing environment, experience may have only a limited
lifetime. Invalid experience must be identified and removed or updated.
To this end, the final Pellucid platform will include a set of methods
and semi-automatic tools to allow knowledge engineers and expert users
to update the experience stored in the organisational memory.
3.2
Active hints
The main conveyor
of experience in Pellucid is an active hint. Active hints present
suggestions to the user to assist with the current activity. Table 1
depicts a schematic example of an active hint in the context of a
proposal evaluation when managing a project for the MMBG site
application, while Table 2 shows an example from the CdG application.
Table 1:
An example of an active hint for MMBG
|
Context: |
Proposal Evaluation |
|
Action: |
Consider |
|
Resource: |
List of People and
Documents |
|
Justification: |
People in the list have
evaluated similar proposals in the past, and they have used the
associated documents for such activity. |
Table 2:
An example of an active hint for CdG
|
Context: |
New employee at Technical
Staff level is designing planimetry
Installation is located near school or hospital or railway station |
|
Action: |
Examine |
|
Resource: |
Topographic relief map for
the zone |
|
Justification: |
Such locations have high
pedestrian flows and need special consideration to accommodate it,
particularly size of the sidewalk and availability of pedestrian
crossing. |
Pellucid offers a
variety of active hints, which are returned by analysing the current
context of an employee and using techniques such as document similarity.
Thus, active hints management allow the experience dissemination within
the organisation.
The engineering
process followed in realising Pellucid experience management model is
further explained in (Arenas et al. 2004).
3.3
Some challenges in experience management
In this part we
explore some challenges that have arisen in the project concerning
experience management.
§
Definition of
Experience. The
first challenge concerns with the definition of the word experience.
During the project, we take a practical view of the word experience,
avoiding to be attached to a particular definition. Our view is of a
common medium –Pellucid- in which users can put and share their
experience, like a digital collaborative library into which every one is
author and reader. The active hint approach is in agreement with this
view: the system is able to store and fire hints. A hint
is not experience, it is only a representation of a fragment
of experience formulated in a way which is practical for a computer to
store and manage.
§
The Start up
Problem. Pellucid
uses an Organisational Memory containing all relevant information
of the organisation. Every Organisational Memory must, at first, store
some content which have been collected in the customisation phase,
perhaps with the help of end users. The content should be valid and form
a critical mass after that the system is able to grow
independently.
This is not necessarily a problem, but only an additional
cost which cannot be anticipated in the framework, but must be part of
the initial set up of the system. In a certain sense the initial set up
is the system! The framework, without set up, is fairly useless, like an
operating system. It is the union “framework + setup” that creates a
usable system.
§
Creation of
Hints. A crucial
part of this set up is the creation of hints. To be successful, it is
extremely important to have a throughout knowledge of the end users'
environment and domain. Writing good domain ontology is necessary, but
not sufficient. The creation of hints can be viewed as a knowledge
acquisition problem, and the methods and techniques available for
knowledge acquisition may be used effectively. In general, it is
necessary to understand clearly the needs of the end-users, meeting them
regularly and giving them the opportunity to assess intermediate
versions of the developing system.
§
Context and
Content. As we
mentioned before, a hint is composed of: context, action, resource
and justification.
Our goal is to have smart hints (that is, hints
which say intelligent things) presented at the right time.
Unfortunately, the smartest hints usually do not have a context which is
easily expressible in machine format; on the contrary, hints that can be
simple characterised in the context, are usually rather poor in content
(they correspond to the class: “look this”, “do that”...)
The best we can do in our system is to mix these two
classes of hints. That is, we try to have simple hints for the
newcomers, which are clearly specified in the context, even if their
content is not so smart; and intelligent hints for the experts which,
even if they have no a clearly defined context, are nevertheless useful
because the expert will give them one. The aim is that either the
content or context of each hint (or ideally both) should be rich.
§
Developing for
the Future (Scalability, Maintenance).
As the start-up, the future of the
system is also a problem. First, the system should be able to forget;
otherwise, the database will grow too much and performance will degrade.
Second, periodically a person should adapt the system to the changes in
the organisations. Hints may become old; some facts could be not true
any more; people want to add their personal contribution to the database
and so on.
All these aspects
are analysed and developed in the Experience Evolution Phase.
3.4
Related experience management models
One of the
influential works in the definition of the Pellucid experience
management model is (Bergmann 2002). Bergmann’s experience model
consists of a Knowledge Kernel, a Problem Solving Cycle,
and a Development and Maintenance Methodology. The Knowledge
Kernel contains the experience base and the reuse-related knowledge
as well as the vocabulary on which both are based. The Problem
Solving Cycle describes problem solving that is supported by
experience reuse; it includes steps such as problem acquisition,
experience evaluation and retrieval, experience adaptation, and
experience presentation. The Development and Maintenance Methodology
addresses the acquisition and maintenance of the knowledge in the
knowledge kernel as well as the technical, organisational, and also
managerial aspects of the problem solving cycle and its implementation.
Bergmann’s Knowledge Kernel corresponds to Pellucid’s organisational
memory, the repository storing the experience, data and metadata needed
by the systems. Some actions of the Capture and Store phase are also
linked to the kernel. The Problem Solving cycle is associated with
Pellucid’s Analysis and Presentation phase, although Bergmann’s cycle is
more general but strongly linked to case-based reasoning. The active
hint approach has the advantage that it can be attached to several
reasoning techniques. Finally, the Development and Maintenance
Methodology is related to Pellucid’s Experience Evolution phase.
The Experience
Factory approach defines a framework for experience management based on
organisational learning (Basili et al. 2001). An experience factory is a
logical and/or physical organisation that supports project development
by analysing and synthesizing all kinds of experience, acting as a
repository for such experience, and supplying that experience to various
projects on demand. The experience consists of informal or formal models
and measures of various processes, products, and other forms of
knowledge. It is tailored for the software business. The steps in the
experience factory are organised into a cycle called the Quality
Development Paradigm, including six steps: a characterisation of the
project and its environment based on the available information; a
definition of the goals of the project; selecting appropriate processes
for implementing the project; execution of the project; analysing data
collected at the end of the project for identifying reusable experience;
and packaging, where the models stored in the experience base are
refined on the basis of the new experience. The Experience Factory
provides a mechanism for continuous improvement through the
experimentation, packaging, and reuse of experience based on the needs
of a business. Although specific to a particular domain, it offers
important lessons; in particular, the identification of reusable
experience and packaging, and thus will influence the final form of the
Pellucid platform.
4.
Modelling working context
4.1
The Pellucid approach to context modelling
The Pellucid
platform enables modelling and handling the working context in order to
manage active hints for fulfilling the EM purpose: supporting the
employees in their working situation and make their task easier. The
Pellucid approach is that of context-based retrieval information (Bauer
et. al. 2002). Thereby, active hints are triggered on the working
context characterising the environment of the user. In the Pellucid
framework, the relevant contextual information is described by the
position in the work process (starting or completing a task, opening a
document, …) and domain-specific characteristics.
(Klemke 2000)
describes a context typology organised around four aspects:
organisational, domain, personal and physical aspects. In the Pellucid
approach, the context mainly is the combination of the two first aspects
from this context typology. Hence, the organisational aspect is covered
by the description of the position in the workflow process and the
domain aspect by the domain specific characteristics. The personal
aspect is also taken into account, but in a rather simple way,
distinguishing different kinds of employees according to their level of
experience.
Each aspect of
the working context is modelled by means of several concepts extracted
from both ontologies: the workflow ontology and the domain ontology.
Consequently, the development of ontologies is one of the central
threads of modelling work. The workflow ontology deals with the process
of an organisation, which is essentially composed of activities, tasks
and instances. The domain ontology encompasses the relevant conceptual
description about the actions (what the employee is able to do), the
resources (document, person) and the relevant descriptive concepts
closely related to the domain application (e.g. type of document, topic
of document, role of person, etc).
For the modelling
of working context, the current position in the workflow is acquired,
together with the current user action as stimuli, plus events and the
current descriptive concepts as relevant similarity information. So
thanks to this modelling, the events trigger appropriate active hints
(action, resource, justification) which contain suitable resources
according to the similarity information.
4.2
Examples of context for the pilot applications
In the Pellucid
project, three pilot applications have been studied. Each of them
implements a workflow and is related to a specific domain application,
i.e., call centre, traffic light installation and project management.
The three applications in the project differ from the domain, the
workflow and the so-called WfInstance, i.e. the key concept throughout a
process cycle: an incidence resolution, a dossier and a project.
The context
modelling is based on several concepts from both workflow and domain
ontologies. Table 3 illustrates part of the working context modelling in
the framework of each pilot application.
