Presentation of the dissertation project and the research results to
date:
Motivation
There
is a remarkable success of robots in automation industries. In
structured environments the robots’ speed, accuracy and reliability
by far exceed human capabilities. But in natural human environments
robots are often unable to perform simple tasks.
An
advanced memory and knowledge processing system is essential for
mobile service robots with sophisticated cognitive capabilities
[Tenorth and Beetz, 2008].
Goals
The
initial questions of the dissertation project are:
-
How
can different modalities be represented in such a way as to enable a
mobile service robot to integrate the available sensing and acting
modalities spontaneously into reasonable cross-modal skills?
-
Can
a reasonably executable model that mimics the cross-modal
integration, representation and reasoning techniques of natural
cognitive systems be constructed and applied to a service robot
platform?
-
How
can a memory system that is based on the advanced reasoning
techniques of an existing knowledge representation system be
constructed in a way that enables a mobile service robot to improve
its cognitive capabilities?
-
How
can such a memory system be integrated with an AI-based general
purpose planner?
The
dissertation project with the working title “Cross-Modal Enhanced
Memory for Mobile Service Robots” tries to develop a biologically
inspired cross-modal memory system for mobile service robots. The
memory system will be based on the current neuroscientific
understanding of cross-modal integration and knowledge
representation. From general ontological knowledge down to low-level
information, knowledge will be represented at different abstraction
levels. Abstract concepts will be grounded to physical sensing and
acting modalities.
The
memory system will further be based on a knowledge representation
system with advanced reasoning capabilities. It will therefore be
able to deduce new knowledge. The final goal is to enable the robot
to understand high-level tasks, reason about high-level concepts and
decompose abstract tasks into executable robot skills.
The
memory system will be integrated with the state-of-the-art service
robot platform TASER [Weser, 2009]. TASER works in terms of
plan-based robot control and is equipped with a general purpose
deliberative planner. The planner requires knowledge about it its
environment in order to find a reasonable plan for a task. This
knowledge about the environment should be provided by the memory
system. If necessary information is not represented explicitly, the
system will try to acquire the information via perception, deduction
or human-robot interaction. Recently the planner received its own
rather rudimental knowledge representation technique that states
information as “raw” facts. Relationships between abstract
concepts cannot be modelled and general inference rules are not
supported. Therefore, the planner should leave the knowledge
representation to the memory system. It is further impossible to
integrate external knowledge sources (e.g. perception, human-robot
interaction or deduction) into the deliberative planning approaches.
An interface between the planner and the memory system will be
defined. Furthermore, the planner will be adapted in way that enables
it to use the memory system, take advantage of the advanced reasoning
capabilities and integrate external knowledge sources into the
planning process.
The
different sensing and acting modalities will be represented
explicitly. This approach makes it easier for the planner to compose
different modalities dynamically into complex robots skills. Inspired
by natural multi-modal cognitive systems and the neuroscientific
dissertation projects of CINACS, the planner will in so doing be
qualified to find plans even in the presence of malfunctions.
Unavailable modalities will be compensated in several situations via
an appropriate composition of the remaining modalities.
In
contrast to other existing service robots, TASER is endowed with a
general purpose planner from the AI community that is not explicitly
developed for a specific domain. This approach has the following
advantages:
-
Maintenance
and further development are provided by a large open source
community.
-
State-of-the-art
methods from theoretical AI are implemented; provably good
performance.
-
The
well-defined interface to reactive parts of the architecture permits
future replacement of the planning component.
But
applicable general purpose planners presume the closed world
assumption and are therefore unable to deal with incomplete
information. Since mobile service robots act in a real dynamic
environment and construct or adapt their world model autonomously
based on sensory data, they are inevitably confronted with uncertain
and incomplete information about the world. The memory system will
therefore be able to represent uncertain and incomplete information
about the world and additionally allow for reasoning about this kind
of knowledge.
In
a nutshell, the dissertation project is expected to lead to concepts
and implementations that can improve the cognitive capabilities of a
service robot in the following ways:
-
Available
modalities can be composed into new cross-modal robot skills. In
this way, the malfunction of modalities can in several situations be
compensated by the remaining robot skills.
-
The
exploitation of a multi-modal memory system will improve the
planning process in the following ways:
-
-
The
efficient generation of reasonable plans will be enabled in more
complex scenarios.
-
Unnecessary,
often time-intensive perception processes are avoided when the
memory system can provide the needed information.
-
Necessary
unknown information can be acquired from external knowledge
sources.
-
The
presence of grounded high-level concepts permits instructors to
formulate tasks in terms of abstract high-level concepts.
-
High-level
knowledge and reasoning can be exploited to deduce new explicit
information.
Current
State of Work
Currently,
an important part of the memory system has already been developed and
implemented. Robot skills are modelled in a way that enables the
planner to autonomously compensate the malfunction of modalities by
composing the remaining modalities dynamically into reasonable robot
skills that are able to fulfil a given task. The memory system is
able to represent factual, conceptual and axiomatic knowledge. From
robot platform specific knowledge to common sense knowledge all kinds
of concepts and information can be expressed with this approach. The
applied knowledge representation techniques have been significantly
inspired and influenced by the knowledge representation techniques of
natural cognitive systems. An existing state-of-the-art planning
approach from the artificial intelligence community has been
enhanced. The new planning approach is able to take advantage of
natural cognitive systems’ methods like problem solving strategies,
representing and reasoning with an incomplete world model and
autonomously acquiring necessary information from external knowledge
sources. The memory and planning systems are integrated into a
coherent artificial cognitive system.
How
I plan to proceed
In
the next weeks the developed artificial cognitive system will be
applied to the service robot platform TASER. Further experiments with
the robot platform will be carried out in order to demonstrate the
scientific innovations of the current approach.
The
further course of action is:
The
work will conclude with the evaluation phase of the developed and
implemented architecture. Experiments will demonstrate the cognitive
capabilities of the improved robot platform TASER. Finally, the
results will be interpreted and the robot’s cognitive capabilities
will be compared with other state-of-the-art service robots.
Links
to other projects of the IRTG
The
dissertation project is strongly connected to the work of Martin
Weser who contributed important innovations by composing different
modalities into coherent cross-modal robot skills. His dissertation
project further demonstrates how the integration of cross-modal robot
skills with a forefront deliberative planning system can improve the
cognitive capabilities of a service robot platform. My dissertation
project goes into a similar direction and will benefit substantially
from Martin Weser’s results.
Furthermore
there is a strong connection to Jianhua Zhang's project. We are
developing a coherent artificial cognitive system together. Our
cooperation includes the grounding of high-level concepts to acting
and sensing modalities and the composition of these modalities to
executable robot skills. Moreover we are using the same experimental
robot platform (TASER) and the same knowledge representation system.
Several joint experiments and evaluations are scheduled.
It
is further linked to Christopher Baumgärtner’s and
Nils Beuck’s project, because we are going to integrate the
multi-modal natural language understanding systems of their
dissertation projects into the artificial cognitive system of my
project. Several profitable discussions about adequate knowledge
representation techniques and interfaces helped me to make my own
approach more flexible and adaptable. The integrated cognitive system
will finally be implemented on the service robot platform TASER.
Moreover
my project is linked to the projects of Dong Wang, Xiaobing Liu and
Cailiang Liu. They are all working on visual concept representation
and image/video analysis. We planned to share data and software
systems. My work benefits a lot from their broad knowledge in this
research area. I will try to integrate their results into my
artificial cognitive system. All our dissertation projects will
additionally profit from joint experiments on our service robot
platform (TASER).
There
is also an important connection to Mario Maiworm’s dissertation
project. His project focuses on the integrative processing of
different sources of information in humans. Maximum likelihood
estimation models are used to describe cross-modal integration
processes. In several discussions we gathered ideas of how to
integrate these models into a coherent artificial cognitive system.
I
further had several inspiring discussion about conceptual knowledge
representation with Kris Lohmann, because that issue also plays an
important role in his project.
Integration
of visits abroad into the dissertation project
During
my stay at the Tsinghua University in September 2009 I had several
profitable discussions that influenced my dissertation projects. I
had the opportunity to present my dissertation project to scientists
from the robotic laboratory. They gave me several helpful hints about
connected projects and literature. In this way they had an important
impact on the further direction of my dissertation project. I further
discussed several ideas with neuroscientists. These discussions
helped me to get a better understanding of cognitive systems in
general and influenced some parts of my artificial cognitive system.
References
M.
Weser. Hierarchical
Memory Organization of Multimodal Robot Skills for Plan-based Robot
Control. Doctoral
Thesis, Hamburg,
Germany, 2009.
M.
Tenorth and
M. Beetz. Towards
practical and grounded knowledge representation systems for
autonomous household robots. Proc. IWCTS
2008 - the 1st International Workshop of
Cognition for Technical Systems, München, Germany,
Oct 06-07, 2008
|