Deterministic Planning in the Fifth International Planning Competition: PDDL3 and Experimental Evaluation of the Planners

The international planning competition (IPC) is an important driver for planning research.
The general goals of the IPC include pushing the state of the art in planning technology by
posing new scientific challenges, encouraging direct comparison of planning systems and
techniques, developing and improving a common planning domain definition language, and
designing new planning domains and problems for the research community. This paper
focuses on the deterministic part of the fifth international planning competition (IPC5),
presenting the language and benchmark domains that we developed for the competition,
as well as a detailed experimental evaluation of the deterministic planners that entered
IPC5, which helps to understand the state of the art in the field.
We present an extension of pddl, called pddl3, allowing the user to express strong and soft
constraints about the structure of the desired plans, as well as strong and soft problem
goals. We discuss the expressive power of the new language focusing on the restricted
version that was used in IPC5, for which we give some basic results about its compilability
into pddl2. Moreover, we study the relative performance of the IPC5 planners in terms of
solved problems, CPU time, and plan quality; we analyse their behaviour with respect to
the winners of the previous competition; and we evaluate them in terms of their capability
of dealing with soft goals and constraints, and of finding good quality plans in general.
Overall, the results indicate significant progress in the field, but they also reveal that
some important issues remain open and require further research, such as dealing with
strong constraints and computing high quality plans in metric-time domains and domains
involving soft goals or constraints.