Introduction
We propose a novel introspective planning scheme that prompts language-enabled agents to proactively
assess their own confidence regarding task compliance and safety for multiple candidate plans, with a guaranteed probability
that the agent will either execute the actions desired by the user or ask an appropriate follow-up question to disambiguate the user's intent.
We introduce a new, weakly supervised offline knowledge base construction method that guides the LLM to generate
human-aligned introspective reasoning examples as post-hoc rationalizations of human-selected safe-and-compliant plans.
We create a new Safe Mobile Manipulation benchmark, which augments previous mobile manipulation datasets with safety-critical scenarios and introduces new metrics to evaluate a planner's specification compliance, safety, and degree of conservativeness.
Introspective Planning
Knowledge base construction: the LLM is prompted to generate knowledge entries based on human-provided instructions and the correct options.
Deployment: Upon receiving an instruction, the LLM formulates possible next steps and consults the knowledge base to retrieve the most relevant examples, which are later used as the prompt for prediction.
Introspective conformal planning
After generating multiple options, we query the LLM for the explanation by introspective planning and then ask the model to predict the most correct option.
Based on the likelihood scores of true intents from a calibration dataset, conformal prediction finds the quantile value (0.85), and includes any options scoring above 0.15 in the prediction set for each test scenario.
This method guarantees the correct answer is included among the options, at a confidence level specified by the user.
Results
We compared our approach with KnowNo, both using conformal prediction with an 85% target success rate. Our method generates explanations via introspective planning before applying conformal prediction, whereas KnowNo directly predicts valid options using conformal prediction. We observed that KnowNo over-step in the left case and over-ask in the right case while IntroPlan generates more precise prediction sets.
Introspective planning guides the LLM to generate more precise prediction sets, achieving the highest exact set rate and lowest non-compliant contamination rate. It avoids over-asking, rarely oversteps, exhibiting the lowest unsafe rate. This demonstrates effective reasoning about both uncertainty and safety on our new benchmark, safe mobile manipulation.
BibTeX
@article{liang2024introspective,
title={Introspective Planning: Guiding Language-Enabled Agents to Refine Their Own Uncertainty},
author={Liang, Kaiqu and Zhang, Zixu and Fisac, Jaime Fern{\'a}ndez},
journal={arXiv preprint arXiv:2402.06529},
year={2024}
}