%0 Journal Article %J Frontiers in Robotics and AI %D 2021 %T Compositional RL Agents That Follow Language Commands in Temporal Logic %A Kuo, Yen-Ling %A Katz, Boris %A Andrei Barbu %X

We demonstrate how a reinforcement learning agent can use compositional recurrent neural networks to learn to carry out commands specified in linear temporal logic (LTL). Our approach takes as input an LTL formula, structures a deep network according to the parse of the formula, and determines satisfying actions. This compositional structure of the network enables zero-shot generalization to significantly more complex unseen formulas. We demonstrate this ability in multiple problem domains with both discrete and continuous state-action spaces. In a symbolic domain, the agent finds a sequence of letters that satisfy a specification. In a Minecraft-like environment, the agent finds a sequence of actions that conform to a formula. In the Fetch environment, the robot finds a sequence of arm configurations that move blocks on a table to fulfill the commands. While most prior work can learn to execute one formula reliably, we develop a novel form of multi-task learning for RL agents that allows them to learn from a diverse set of tasks and generalize to a new set of diverse tasks without any additional training. The compositional structures presented here are not specific to LTL, thus opening the path to RL agents that perform zero-shot generalization in other compositional domains.

%B Frontiers in Robotics and AI %V 8 %8 07/2022 %G eng %U https://www.frontiersin.org/articles/10.3389/frobt.2021.689550/full %! Front. Robot. AI %R 10.3389/frobt.2021.689550