\section{Introduction}

The ability to reason about objects’ properties and behaviours in physical environments lies at the core of human cognitive development \cite{CLEVRER,Davis2006}. Few days after birth, infants understand object solidity \cite{Valenza2006} and within the first year of birth, they understand notions such as object permanence \cite{permanence}, spatiotemporal continuity \cite{continuity}, stability \cite{stability}, support \cite{support}, causality \cite{causality}, and shape constancy \cite{shapeconstancy}. Generalization performance on novel physical puzzles is commonly used as a measure of physical reasoning abilities for children \cite{Carmel2000,Cheke2012}, animals \cite{Chimpanzees1,Crows1,Crows2,crows3}, and AI agents \cite{Phyre,OGRE,Allen2020,ahmed2020causalworld}.

Chollet's study \cite{Chollet19} on the measure of intelligence proposes a qualitative spectrum of different forms of generalization that includes \textit{local generalization} and \textit{broad generalization}. Current evidence \cite{lg1,lg2,lg3,lg4} suggests that contemporary deep learning models are local-generalization systems, i.e., systems that adapt to known unknowns within a single task. \textit{Broad generalization}, on the other hand, can be characterized as `adaptation to unknown unknowns across a broad category of related tasks' and is being increasingly emphasized among the AI research community \cite{Chollet19,geirhos2020shortcut, firestone2020performance}. Moreover, when solving physics puzzles, it is common that a player must use strategy to work out a plan and must use dexterity to accurately execute the strategic plan \cite{strategyAndDexterity}. For instance, in a snooker game, a player needs to plan the path of the white cue ball, e.g., where it should go and where to stop, and then execute the strike that precisely produces to the planned path. As humans are known to have inaccurate forward physics prediction models \cite{mccloskey_naive_1983,smith_sources_2013,siegler_three_1976,keil_folkscience:_2003,gilden_heuristic_1994,levillain_dissociation_2011,lawson_mirrors_2012,proffitt_understanding_1990} and hence require practice to improve dexterity, high dexterity requirements of physics tasks make it unfair to compare AI agents' physical reasoning ability with average humans'. For example, when a human player fails a physics puzzle, it is hard to tell if it is due to incorrect physical reasoning or due to inability of making precises actions. Despite the recent advancement in physical reasoning benchmarks and testbeds \cite{Phyre,OGRE,Allen2020,IntPhys2019,CLEVRER,Cophy,ahmed2020causalworld,Physion,animalAI}, there is a lack of a benchmark or a testbed with human comparable strategic physics puzzles and a one that explicitly evaluates learning agents’ local and broad generalization.

To close these gaps, we propose a new testbed Phy-Q and the associated Phy-Q score that measures physical reasoning intelligence. Inspired by the physical knowledge acquired in infancy and the abilities required by the robots to operate in the real world, we have created a wide variety of tasks with low dexterity requirements in the video game Angry Birds. Also we included a module to aid trajectory planning.
We believe the contributions from this paper pave the way to develop agents with human level strategic physical reasoning capabilities.
Our main contributions can be summarized as follows:

\begin{itemize}[leftmargin=*]
    \item \textbf{Phy-Q: A testbed for physical reasoning: } We designed a variety of task templates in Angry Birds with 15 physical scenarios, where all the task templates of a scenario can be solved by following a common strategic physical rule. Then we generated task instances from the templates using a task variation generator. This design allows us to evaluate both the local and the broad generalization ability of an agent.
    We also define the Phy-Q score, a quantitative measure that reflects physical reasoning intelligence.

    \item \textbf{An agent-friendly framework: } We propose a framework, which allows training multi-agent instances simultaneously with accelerated game-play speed up to 50 times.

    \item \textbf{Establishing results for baseline agents: } The evaluation consists of nine baseline agents: four of our best performing learning agents, four heuristic-based agents, and a random agent.

    For each of the baseline agents, we present the Phy-Q score, the \textit{broad generalization} performance, and the \textit{local generalization} performance.
     We have collected human player data so that agent performance can be directly compared to human performance.

    \item \textbf{A guidance for agents in AIBIRDS competition: }
    Angry Birds is a popular physical reasoning domain among AI researchers with the long running AIBIRDS competition since 2012 \cite{Renzet2019}. In 2016, Angry Birds was considered to be the next milestone in AI where AI will surpass humans \cite{Grace2018}. A time horizon of four years was predicted and so far such a breakthrough seems very unlikely.
    In the AIBIRDS competition, heuristic methods generally perform better than their deep learning counterparts, but it remains unclear what has contributed to the gap in the performances. It has also not yet been analysed why current AI agents fall short when comparing to humans. By the systematic analysis of agents from the AIBIRDS competition, we show how they need to be improved to achieve human-level performance.
\end{itemize}

\section{Background and Related Work} \label{Background and Related Work}

In this section, we conduct a comparison between ten related physical reasoning benchmarks and two physics-based AI game competitions to show how Phy-Q testbed advances upon existing work. The comparison is done with respect to six criteria.
1) Measuring \textit{broad generalization} in individual physical scenario/s, i.e, testing the ability of an agent in generalizing to tasks that require the same physical rule to solve.
2) Categorization of tasks of the test environment into different physical scenarios, i.e, agents can be evaluated for individual scenarios to recognize the scenarios that they can perform well.
3) Procedural generation of tasks or variations of the tasks, i.e., the tasks/variants of the tasks in the test environment are created algorithmically facilitating the users to generate any amount of data.
4) Destructibility of objects in the environment, i.e., if the environment contains objects that can be destroyed upon the application of forces. Having destructible objects makes the environment more realistic than an environment that only has indestructible objects since the agents need to consider the magnitude of the force that is applied to the objects. For example, when a robot moves a cup, it needs to reason that the force to exert should be large enough to grab the cup but not too large to break it.
5) Observing the outcome of a desired physical action, i.e., if an agent can physically interact and observe the outcome of the action the agent takes.
6) Inclusion of human player data, i.e., if the evaluation has results of human players.

We consider PHYRE \cite{Phyre}, Virtual Tools game \cite{Allen2020}, and OGRE \cite{OGRE}, which are game based benchmarks, IntPhys \cite{IntPhys2019}, CLEVERER \cite{CLEVRER}, CATER \cite{CATER}, and Physion \cite{Physion} which are video based benchmarks, COPHY \cite{Cophy} which is an image based benchmark, CausalWorld \cite{ahmed2020causalworld} and RLBench \cite{james2020rlbench} which are robotic benchmarks. The AI game competitions we consider are Computational Pool \cite{Computational_Pool}, and Geometry Friends \cite{GeometryFriends}. We also included the AIBIRDS \cite{AIBirds} competition for the comparison to show what properties in Phy-Q facilitate systematic evaluation of AIBIRDS competition agents. Table \ref{benchmarks-table} summarises the comparison.

\begin{table}
  \scriptsize
  \caption{Comparison of Phy-Q with related physics benchmarks and competitions}
  \label{benchmarks-table}
  \centering
  \begin{tabular}{lllllll}
    \toprule
     Test & generalization & categorization & procedurally & destructible & observe outcome & human \\
     Environment & to individual & of tasks to & generated & objects & of a desired & player\\
     & physical scenario/s & physical scenarios & tasks/variations &  & physical action & data\\
    \midrule
    PHYRE \cite{Phyre} & \xmark & \xmark & \checkmark & \xmark & \checkmark & \xmark\\
    Virtual Tools \cite{Allen2020} & \xmark & \checkmark & \xmark & \xmark & \checkmark & \checkmark \\
    OGRE \cite{OGRE} & \xmark & \xmark & \checkmark & \xmark & \checkmark & \xmark\\
    \midrule
    IntPhys 2019 \cite{IntPhys2019} & \checkmark & \checkmark & \checkmark & \xmark & \xmark & \checkmark\\
    CLEVERER \cite{CLEVRER} & \checkmark & \xmark & \checkmark & \xmark & \xmark & \xmark \\
    CATER \cite{CATER} & \checkmark & \checkmark & \checkmark & \xmark & \xmark & \xmark\\
    Physion \cite{Physion} & \checkmark & \checkmark & \checkmark & \xmark & \xmark & \checkmark\\
    COPHY \cite{Cophy} & \checkmark & \checkmark & \checkmark & \xmark & \xmark & \checkmark\\
    \midrule
    CausalWorld \cite{ahmed2020causalworld} & \checkmark & \checkmark & \checkmark & \xmark & \checkmark & \xmark\\
    RLBench \cite{james2020rlbench} & \xmark & \xmark & \checkmark & \xmark & \checkmark & \xmark\\
    \midrule
    Computational Pool \cite{Computational_Pool}& \xmark & \xmark & \xmark & \xmark & \checkmark & \xmark\\
    Geometry Friends \cite{GeometryFriends} & \xmark & \xmark & \checkmark & \xmark & \checkmark & \xmark\\
    \midrule
    AIBIRDS \cite{AIBirds}& \xmark & \xmark & \checkmark & \checkmark & \checkmark & \checkmark\\
    Phy-Q (ours) & \checkmark & \checkmark & \checkmark & \checkmark & \checkmark & \checkmark\\
    \bottomrule
  \end{tabular}
\end{table}

The most closely related physical reasoning test environment to ours is PHYRE \cite{Phyre}, which also consists of tasks to measure two levels of generalization of agents. PHYRE benchmark tests if agents can generalize to solve tasks within a task template (within template) and if agents can generalize between different task templates (cross template).
The cross template evaluation in PHYRE does not guarantee that the physical rules required to solve the testing tasks exist in the training tasks. This leads to uncertainties in understanding agents' performance: inferior performance may not be an indicator of inferior physical reasoning but of a difficult training and testing split. Whereas, the \textit{broad generalization} evaluation in our testbed always ensures that physical rules required in testing tasks are covered in the training tasks, hence we guarantee a more systematic evaluation of the physical reasoning capabilities of AI agents.
According to the task design in PHYRE, tasks need to be solved by trial and error. That is, even when the physical rule is known, multiple attempts are still needed to solve the tasks. Therefore, PHYRE promotes developing agents with physical dexterity. In contrast, we focus on strategy based physical reasoning tasks that can be solved by a single attempt when the physical rule is understood. We promote developing agents that can understand a physical rule rather than taking a precise action in a physical environment (i.e., agents with strategic physical reasoning capabilities).
Furthermore, a limited number of object shapes, motion, and material properties on scene dynamics hinders the ability of a comprehensive evaluation, as performing well on these tests might not indicate greater physical reasoning ability in more general and realistic contexts \cite{Physion}. Therefore, compared to PHYRE,  in Phy-Q we have 1) 3 more object shapes (rectangles, squares, and triangles) to allow more diverse physical dynamics, 2) destructible objects to make our environment more realistic, and 3) objects with three different materials that has different densities, bounciness, and friction to allow physical reasoning in a more realistic context.

As a recent visual and physical prediction benchmark, Physion \cite{Physion} evaluates algorithms' physical prediction capability using videos of eight different physical scenarios. Compared to Physion, Phy-Q testbed has a more comprehensive set of 15 physical scenarios enabling evaluation of agents in a wider range of physical scenarios. Phy-Q testbed requires agents to interact with the environment and select the desired action to accomplish physical tasks. Therefore, on top of predicting physical events outcome, agents need to apply the acquired physical knowledge to solve new situations, which is considered to be a more advanced type of tasks in bloom’s taxonomy \cite{bt2001}. In addition, a study on forward prediction for physical reasoning \cite{Rohit2020} confirms that predicting the outcome in physical events does not necessarily increase performance in domains that require selecting an action. Therefore, the research problem focused in Physion is different to the problem in the Phy-Q challenge.

Despite Angry Birds being a simplified and controlled physics environment as compared to the much messier real physical world, no AI system has been developed that comes close to human performance. As to encourage the development of AI agents that can reason with physics like humans do, the AIBIRDS competition has been organized annually since 2012, mostly collocated with the International Joint Conference on Artificial Intelligence \cite{AIBirds}. Since then, many different AI approaches have been proposed, ranging from modern deep reinforcement learning methods to more old-school heuristic methods, e.g. qualitative physical reasoning methods.
However, none of these approaches have reached the milestone of achieving human level performance. One major reason is that the agent performance in the competition does not enable an agent developer to identify the physical scenarios that agent falls short. This is because the tasks in the competition are generally complex with multiple physical scenarios within the same task. In this work, we show how the Phy-Q testbed can be used towards guiding the competition agents through a systematic evaluation of agents performance.

\section{Phy-Q Testbed} \label{Section Hi-Phy Physical Reasoning Benchmark}
In this section, we introduce our testbed, discuss the physical scenarios we have identified, and explain the evaluation settings we have used in the testbed.

\subsection{Introduction to the Phy-Q Testbed}

Based on the 15 identified physical scenarios (discussed in detail in Section \ref{Section Physical scenarios in Phy-Q}), we develop a physical reasoning testbed using Angry Birds. In Angry Birds, the player interacts with the game by shooting birds at the pigs from a slingshot. The goal of the player is to destroy all the pigs using the provided set of birds. As the original game by Rovio Entertainment \cite{AngryRovio.com} is not open-sourced, we use a research clone of the game developed in Unity \cite{Ferreira2014}. The game environment is a deterministic 2D world with Newtonian physics. The game objects are of four types: birds, pigs, blocks, and platforms. There are five types of birds in which four of them have powers that can be activated once tapped in their flight. There are three types of pigs varying in size, the health points of the pigs increase with the increase in size. Blocks in the game are made of three materials (wood, ice, and stone) and each of them has 12 variations in shape. Platforms are static objects that remain at a fixed position and they are not affected by forces and are indestructible. All other objects are dynamic, i.e., they can be moved by applying forces. Dynamic objects have health points that get reduced upon collisions with other objects and they get destroyed and disappear when health points reach zero. The initial state of a game level is physically stable (i.e., none of the objects is in motion) and the goal is not achieved. The action of an agent is to shoot the bird on the slingshot by providing the release coordinates relative to the slingshot and the tap time of the bird to activate powers (if available). This means the action space is essentially continuous. When playing, an agent takes a sequence of actions, i.e., shoots the birds in a predefined order. The agent passes a game level when it destroys all pigs with the provided set of birds, and fails otherwise. We do not provide the full world state that includes the exact location of objects in the simulator or their physical properties such as mass and friction to the agents as these properties are not directly observable in the real world. Instead, an agent can request screenshots and/or a symbolic representation of the game level at any time while playing. A game screenshot is a 480 x 640 coloured image and the symbolic representation is in JSON format containing all objects in the screenshot represented as a polygon of its vertices (provided in order) and its respective colour map. The colour map provides the list of 8-bit quantized colours that appear in the game object with their respective percentages.

\subsection{Physical Scenarios in Phy-Q Testbed} \label{Section Physical scenarios in Phy-Q}

In this section, we explain the 15 physical scenarios we consider in our testbed.
Firstly, we consider the basic physical scenarios associated with applying forces directly on the target objects, i.e., the effect of a single force and the effect of multiple forces \cite{force}. On top of the application of single force, we also include scenarios associated with more complex motion including rolling, falling, sliding, and bouncing, which are inspired by the physical reasoning capabilities developed in human infancy \cite{rolling_falling,falling}. Furthermore, we define the objects' relative weight \cite{wang_williamson_meltzoff_2018}, the relative height \cite{baillargeon_devos_1991,hespos_baillargeon_2001}, the relative width \cite{wang_2004}, the shape differences \cite{newcombe_huttenlocher_learmonth_1999, baillargeon_li_ng_yuan_2008}, and the stability \cite{wilcox_chapa_2004} scenarios, which require physical reasoning abilities infants acquire typically in a later stage.
On the other hand, we also incorporate clearing path, adequate timing, and manoeuvring \cite{Robotic_Challenges}, and taking non-greedy actions \cite{non_greedy}, which are required to overcome challenges for robots to work safely and efficiently in physical environments. To sum up, the physical scenarios we consider and the corresponding high-level strategic physical rules that can use to achieve the goal of the associated tasks are the following.

\begin{enumerate}
\item \textbf{Single force: } Target objects have to be destroyed with a single force.
\item \textbf{Multiple forces: } Target objects need multiple forces to destroy.
\item \textbf{Rolling: }Circular objects have to be rolled along a surface to a target.
\item \textbf{Falling: }Objects have to be fallen on to a target.
\item \textbf{Sliding: }Non-circular objects have to be slid along a surface to a target.
\item \textbf{Bouncing: }Objects have to be bounced off a surface to reach a target.
\item \textbf{Relative weight: }Objects with correct weight have to be moved to reach a target.
\item \textbf{Relative height: }Objects with correct height have to be moved to reach a target.
\item \textbf{Relative width: }Objects with correct width or the opening with correct width have to be selected to reach a target.
\item \textbf{Shape difference: }Objects with correct shape have to be moved/destroyed to reach a target.
\item \textbf{Non-greedy actions: }Actions have to be selected in the correct order based on physical consequences. The immediate action may be less effective in the short term but advantageous in long term. i.e., reach less targets in the short term to reach more targets later.
\item \textbf{Structural analysis: }The correct target have to be chosen to break the stability of a structure.
\item \textbf{Clearing paths: }A path have to be created before the target can be reached.
\item \textbf{Adequate timing: }Correct actions have to be performed within time constraints.
\item \textbf{Manoeuvring: }Objects have to be carefully guided to reach a target.
\end{enumerate}

\subsection{Task Templates and Task Generation}

We design task templates in Angry Birds for each of the 15 physical scenarios mentioned above. A task template can be solved by a specific strategic physical rule and all the templates belonging to the same scenario can be solved by the high-level strategic physical rules discussed above. The task templates are handcrafted and we ensure that if an agent understood the strategic physical rule to solve the template, they can solve the template without requiring high accurate shooting, e.g., the template can be solved by shooting at a specific object rather than shooting a specific coordinate. This design criteria is followed to reduce the dexterity requirement when solving the tasks in our testbed. We have developed 2-8 task templates for each scenario in the hierarchy totalling 75 task templates. Figure \ref{fig_six_example_tasks_in_hiphy} shows example task templates for the 15 scenarios.

We generate 100 game levels from each template and we refer to these game levels as tasks of the task template. All tasks of the same template share the same strategic physical rule to solve. Similar to \cite{Phyre}, the tasks are generated by varying the location of the game objects in the task template within a suitable range. Furthermore, various game objects are added at random positions in the task as distractions, ensuring that they do not alter the solution of the task. Although we provide 100 tasks for each task template, we also provide a task variation generation module to generate more tasks if needed. The 75 task templates and example task variations can be found in the Supplement.

\begin{figure}[t]
  \centering

  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/1.1.1.png}
    \caption{Single force}
    \label{single_force}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/1.2.2.png}
    \caption{Multiple forces}
    \label{multiple_forces}
  \end{subfigure}
    \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/2.1.4.png}
    \caption{Rolling}
    \label{rolling}
  \end{subfigure}

  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/2.2.1.png}
    \caption{Falling}
    \label{falling}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/2.3.1.png}
    \caption{Sliding}
    \label{sliding}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/2.4.1.png}
    \caption{Bouncing}
    \label{bouncing}
  \end{subfigure}

  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.1.3.png}
    \caption{Relative weight}
    \label{relative_weight}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.2.3.png}
    \caption{Relative height}
    \label{relative_height}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.3.3.png}
    \caption{Relative width}
    \label{relative_width}
  \end{subfigure}

  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.4.3.png}
    \caption{Shape difference}
    \label{shape}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.5.5.png}
    \caption{Non-greedy actions}
    \label{non_greedy}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.6.5.png}
    \caption{Structural analysis}
    \label{structural}
  \end{subfigure}

  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.7.5.png}
    \caption{Clearing paths}
    \label{clearing_paths}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.8.1.png}
    \caption{Adequate timing}
    \label{adequate_timimg}
  \end{subfigure}
  \begin{subfigure}[b]{0.31\columnwidth}
    \includegraphics[width=\linewidth]{task_mainpaper_screenshots/3.9.4.png}
    \caption{Manoeuvring}
    \label{manoeuvring}
  \end{subfigure}

\caption{15 example tasks in Phy-Q representing the 15 physical scenarios. The slingshot with birds is situated on the left of the task. The goal of the agent is to kill all the green-coloured pigs by shooting birds from the slingshot. The dark-brown objects are static platforms. The objects with other colours are dynamic and subject to the physics in the environments.}
\label{fig_six_example_tasks_in_hiphy}
\end{figure}

\subsection{Proposed Evaluation Settings} \label{section Evaluation Settings}

The spectrum of generalization proposed by Chollet \cite{Chollet19} can be used to measure intelligence as laid out by theories of the structure of intelligence in cognitive psychology. There are three different levels in the spectrum: \textit{local generalization}, \textit{broad generalization}, and \textit{extreme generalization}. Having 15 physical scenarios, a variety of task templates for each scenario, and task variations for each task template, our testbed is capable of evaluating all of the three different generalization levels. However, in this work,  we focus on measuring the \textit{local generalization} and the \textit{broad generalization} of agents, as \textit{local generalization} is the form of generalization that has been concerned from 1950s up to this day and there is an increasing research interest in achieving \textit{broad generalization} \cite{Chollet19}.

More formally, consider each scenario $scenario_i$ in the set of all scenarios $SCENARIO$, where $|SCENARIO| = 15$, we define template $template_j \in scenario_i$, where $|scenario_i| = NT_i$ and $NT_i$ is the number of templates we included for $scenario_i$. As we have $100$ tasks for each templates, we define $task_k \in template_j$, where $|template_j| = 100$ for all templates.

To evaluate \textit{local generalization}, we train an agent on a subset of the tasks from
a template $j$, $TrainTasks^j_{local} \subset template_j$ and test on the rest of the tasks within the same template as $TestTasks^j_{local} = template_j \setminus TrainTasks^j_{local}$. We evaluate the \textit{local generalization} for all of the $75$ templates.

For evaluating the \textit{broad generalization}, we train an agent on the training tasks on a subset of templates, $TrainTasks^i_{broad} = \cup_{\{j | template_j \in scenario^{train}_i\}} TrainTasks^j_{local}$,  and evaluate on the testing tasks of the rest of the templates within the same scenario, $TestTasks^i_{broad} = \cup_{\{j | template_j \in scenario^{test}_i\}} TestTasks^j_{local}$, where $scenario^{train}_i$ is the set of training templates and $scenario^{test}_i$ is the testing templates for $scenario_i$. We also make sure that $\forall i, scenario^{train}_i \cap scenario^{test}_i = \emptyset $. We evaluate the \textit{broad generalization} performance for all 15 scenarios. We assume that if an agent learns the required strategic physical rule to solve a set of task templates, it should be able to apply the same strategic physical rule to solve unseen tasks from other templates within the same scenario. Opposed to this, the performance on \textit{local generalization} evaluation may not represent an agent's physical rule generalizing capability but memorizing a special-purpose heuristic. Figure \ref{fig_evaluation_setup} is a diagrammatic representation of the two evaluation settings.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.98\linewidth]{Fig_evaluation_setup.png}
  \caption{The \textit{local generalization} and the \textit{broad generalization} evaluation settings}
  \label{fig_evaluation_setup}
\end{figure}

Our physical reasoning quotient (Phy-Q) is inspired by deviation IQ \cite{wechslerIQ} for humans.
We calculate Phy-Q of an agent using the results of our \textit{broad generalization} evaluation since we consider that this evaluation measures the agent's ability in generalizing strategic physical rules.
When calculating the Phy-Q we exclude the first two scenarios: single force and multiple forces, as the solution for the two scenarios is directly shooting the bird to the exact location of the pig. Given that we have provided a trajectory planner for both humans and agents, solving the tasks of the two scenarios are straightforward. This is also evident from the exceptionally high results (in Section \ref{loc_and_bro_results}) of the Pig Shooter agent that directly shoots at the pigs without doing any physical reasoning. We define the Phy-Q score as follows:

\begin{equation}
\resizebox{0.9\linewidth}{!}{
\textit{$Z_{agent}$} = $\frac{1}{|SCENARIO - \{scenario_1,scenario_2\}|} \sum_{m=3}^{|SCENARIO|} \frac{P_{agent,m} - {P_{human,m}}}{\sigma_{human,m}} $}
\end{equation}

\begin{equation}
\resizebox{.42\linewidth}{!}{
\textit{Phy-Q score$_{agent}$} = $100 + Z_{agent} \frac{100}{|Z_{random}|}$
}
\end{equation}

where $P_{n,m}$ is the average pass rate of subject \textit{n} in the $m^{th}$ scenario. $\sigma_{human,m}$ is the standard deviation of human pass rate in scenario $m$ and $random$ represents the random agent which selects a random action (See Section \ref{baseline agent}). A Phy-Q score of 100 represents the agent has an average human level performance and if the score is less than 100, the agent's performance is less than the average humans' performance and vice versa. As the random agent does not have any physical reasoning capabilities, we bring the random agent's Phy-Q score to zero. Therefore, the interval for each standard deviation of Phy-Q is set to be 13.58 as compared to 15 of IQ. Therefore, a Phy-Q score more than zero indicates a performance better than an agent that selects a random action.

\section{Experiments}

We conduct experiments on baseline learning agents to measure how good they can generalise in two different settings: \textit{local generalization} and \textit{broad generalization}.
We also conduct experiments using heuristic baseline agents in the two generalization settings. In addition, we establish human performance on the 15 scenarios.
Further, we conduct an additional experiment using heuristic agents in AIBIRDS competition game levels to examine if the performance of agents in the testbed resembles the performance in the competition.

\subsection{Baseline Agents} \label{baseline agent}

We present experimental results of nine baseline agents: two DQN agents - one uses screenshot as input and the other uses symbolic representation, two relational agents - one uses screenshot as input and the other uses symbolic representation, four heuristic agents from the AIBIRDS competition, and a random agent.

\paragraph{Learning Agents}
For learning agents, we evaluate value-based and policy-based reinforcement learning algorithms. We evaluate Double Dueling Deep Q-network (D-DDQN) \cite{DBLP:journals/corr/WangFL15, DBLP:journals/corr/HasseltGS15} for value-based and Proximal Policy Optimization (PPO) \cite{Schulman2017ProximalPO} and A2C \cite{Mnih2016AsynchronousMF} for policy gradient agents.

\begin{itemize}[leftmargin=*]
    \item \textbf{Deep Q-network (DQN): } The DQN \cite{mnih2015humanlevel} agent collects state-action-reward-next state quadruplets at the training time following decaying epsilon greedy. The quadruplets are then stored in a \textit{replay buffer} $E=\{e_1,...,e_n\}$, where each $e_i = (s_i, a_i, r_{i+1}, s_{i+1})$ is an \textit{experience}. We define the reward function as task pass status, meaning the agent receives $1$ if the task is passed and $0$ otherwise. The agent uses a discretized action space of 180 actions, where each corresponds to a release degree from the slingshot with a maximum stretch. At the end of each update step, the agent trains a Deep Q-network on the sampled experiences to predict the Q-value of actions for a given state. In our experiments, we use Double Dueling Deep Q-network \cite{DBLP:journals/corr/HasseltGS15, DBLP:journals/corr/WangFL15}, which comprises of a state encoder that transforms an input state into a hidden representation through convolutional filters, which is then separated into a state-value and advantage streams. We tested four improvements upon a vanilla DQN: Double DQN \cite{DBLP:journals/corr/HasseltGS15}, Dueling DQN \cite{DBLP:journals/corr/WangFL15}, Prioritized Experience Replay \cite{schaul2015prioritized} and Double Dueling DQN, each with two different input types: symbolic representation and screenshot. For symbolic representation, we map each game state to a $h \times w \times o_t$ tensor, where $o_t$ is the number of object types. In our experiments, we set $h=120$, $w=160$, and $o_t = 12$. For the state encoder, we use a $1\times1$ convolutional filter to squeeze the channels to $1$ and then use $h \times w$ filters to extract features from the representation that preserves crucial spatial information. For agents with screenshot state representation, we re-scale and normalise the image to $h=120$, $w=160$, and use ResNet-18 \cite{DBLP:journals/corr/HeZRS15} to extract features and estimate Q-values. We trained both of the networks using an Adam optimizer \cite{journals/corr/KingmaB14} with a learning rate of $0.0003$. We refer the Dueling Double DQNs we experimented with the two different input types as D-DDQN-Image and D-DDQN-Symbolic.

    \item \textbf{Policy Learners:} To support the development of the learning agents and to allow using existing RL libraries, our framework follows the OpenAI Gym \cite{Brockman2016OpenAIG} requirements. This allowed us to evaluate the two multi-processing agents from the Stable-Baselines3 \cite{stable-baselines3}: A2C \cite{Mnih2016AsynchronousMF} and PPO \cite{Schulman2017ProximalPO}. We trained the two agents with both discrete and continuous action spaces, different training settings and various hyper-parameters, but similar to \cite{Phyre}, policy-gradient methods did not show good results compared to DQN and were unable to converge to the good policy in a reasonable time. Therefore, we exclude these two agents in the results section.

    \item \textbf{Relational Deep Q-network}: The relational agent consists of the relational module \cite{Zambaldi2018RelationalDR} that was build on top of the deep Q-network. The aim of this agent is to generalize over the presented templates/events by using the structured perception and relational reasoning. In our experiments, we wanted to test if the relational agent would be able to learn the important relations between the objects that could be generalized to other templates or events. We have tested the agent with symbolic and image input types and refer to them as Relational-symbolic and Relational-image agents respectively. For the image agent we use ResNet-18 as the feature extractor and feed its output to the Multi-head dot product attention module (MHDPA) \cite{Zambaldi2018RelationalDR}. The output of the MHDPA is then used to compute the policy. For the symbolic agent, the setting is similar to the symbolic DQN agent except that the output of the convolutional layer is passed to the relational module.

\end{itemize}

\paragraph{Heuristic Agents:}

The heuristic agents are based on hard-coded strategic physical rules designed by the developers. We included four heuristic agents from AIBIRDS competition.
We compare heuristic agents' performance on our testbed with the generalization performance of the baseline learning agents.

\begin{itemize}[leftmargin=*]
    \item \textbf{Bambirds:} Bambirds is the winner of 2016 and 2019 AIBIRDS competition. The agent chooses one of nine different strategies. The strategies include creating a domino effect, targeting blocks that support heavy objects, maximum structure penetration, prioritizing protective blocks, targeting pigs, and utilizing certain bird’s powers \cite{Bambirds}.
    \item \textbf{Eagle's Wing:} Eagle’s Wing is the winner of 2017 and 2018 AIBIRDS competitions. This agent selects action based on strategies including shoot at pigs, destroy most blocks, shoot high round objects, and destroy structures \cite{EagleWings}.
    \item \textbf{Datalab:} Datalab is the winner of 2014 and 2015 AIBIRDS competitions. The agent use strategies: destroy pigs, destroy physical structures, and shoot at round blocks. The agent selects the strategy based on the game states, possible trajectories, bird types, and the remaining birds \cite{Datalab}.
    \item \textbf{Pig Shooter:} The strategy of the Pig Shooter is to directly shoot at the pigs. The agent shoots the bird on the slingshot by randomly selecting a pig and a trajectory to shoot the pig \cite{Stephenson2018TheCompetition}.
\end{itemize}

\paragraph{Random agent:} For each shot, the agent selects a random release point $(x, y)$, where $x$ is sampled from $[-100,-10]$ and $y$ from $[-100,100]$ relative to the slingshot. It also provides a tapping time that is when the bird is in between $50\%$ and $80\%$ of the trajectory length where applicable.

\subsection{Experimental Setups} \label{section experimental setup}
\paragraph{Human experiment setup:} Experiments were approved by the Australian National University committee on human ethics under protocol 2021/293. We recruited 20 volunteers for the experiment. For each of them, we provided two tasks from each physical scenario for the 15 scenarios in Phy-Q (except the manoeuvring scenario which used four tasks representing the four types of birds with powers). We provide a trajectory visualizer of the bird to the participants to remove the need of precise shooting. If the participants solved a task or failed to solve a task in five attempts, they move to the next task.

\paragraph{D-DDQN and Relational DQN experimental setup:} \label{DQN_exp_setup}
We conduct separate experiments on the D-DDQN and Relational agents in the two settings: \textit{local generalization} and \textit{broad generalization} setting.
For the \textit{local generalization} evaluation, to speed up the training, we run 10 sampling agents that use the same DQN model to collect experiences. Each sampling agent runs on the randomly selected task for 10 episodes. After the set of experiences is collected, the DQN model is trained for 10 epochs with randomly sampled batches of size 32. We train DQN until it either converges or reaches $N$ update steps, where $N$ is the number of training tasks per template divided by 5. Similar to \cite{Phyre}, for each batch we sample 16 experiences that solve a task and 16 that do not. We train our agent on 80$\%$ of the tasks of the task template and evaluate on the rest of the 20$\%$ tasks of the same template. As to discourage hyper-parameters tuning, we used the same training setting for all of the task templates. At the testing time, the agent runs on each of the testing tasks only once and selects the action that has the highest Q-value for a given state.
For {\it broad generalization} evaluation, we use the same training and testing setting as in the \textit{local generalization} evaluation, except we train our agents on the tasks in the training templates in each scenario and test on the tasks from testing templates.

\paragraph{Heuristic agents experimental setup:} We conduct two experiments using the AIBIRDS heuristic agents.The first experiment is to evaluate the \textit{local and broad generalization} capabilities and the second to evaluate the performance in the AIBIRDS competition game levels.

\begin{itemize}[leftmargin=*]
    \item \textbf{\textit{Local and broad generalization setup:}}
    Due to the randomness in the heuristic agents, we allow them to have five attempts per task and calculate the task pass rate by averaging the result over these five attempts. For the \textit{local generalization} setting, the agents were tested on the same $20\%$ of the test tasks from each task template (1500 tasks in total) used for the D-DDQN evaluation. We report the \textit{local generalization} performance by averaging pass rates of all templates. For the \textit{broad generalization} setting, the same testing templates used for the D-DDQN evaluation were used and the within scenario pass rate is calculated by averaging over all the tested templates within the scenario.

    \item \textbf{AIBIRDS competition setup:} We evaluate the AIBIRDS heuristic agents on 2021 AIBIRDS competition game levels to compare their performance in the competition game levels and the Phy-Q testbed tasks. We exclude the competition game levels with unrealistic effects as our focus in the testbed is scenarios with realistic physics. The game levels used for this evaluation are shown in Appendix F. In the AIBIRDS competition, the agent with the highest score wins the competition. Therefore in this experiment, we record the score and pass rate of the agents. The agents are allowed to have five attempts per game level to account for their randomness. Altogether an agent had 40 plays.

\end{itemize}

\paragraph{Random agent experimental setup:}
The random agent was tested on the same set of testing tasks from each task template with 1500 tasks. As to reduce the bias in pass rate, we run the random agent 50 times per task and report the pass rate using the average of these 50 attempts.
Same as how we evaluate the heuristic agents, we further average the task performance within the same task template and average the pass rate of all the templates to present the \textit{local generalization} performance. For the \textit{broad generalization} setting, within scenario pass rate is calculated by averaging over all the tested templates within the scenario.

\section{Results and Analysis} \label{result and analysis}

In this section, we first present and analyze the results obtained from our experiment with human players. Next, we present the results obtained from our experiments in measuring the local and broad generalization ability of agents and Phy-Q score. We further analyze the results and discuss what we can derive from the experiments. We also discuss the results obtained from the heuristic agents in the 2021 AIBIRDS competition levels and Phy-Q testbed tasks to show how the testbed can be used as a guidance for the competition.

\subsection{Human Performance}

Figure \ref{fig_human_scaled} presents the average pass rate, the pass rate the human players achieved within five attempts, the maximum number of attempts made, and the total thinking time of human players for the 15 capabilities.
The average pass rate is calculated as 100\% if the player passes in the first attempt and if the player passes in the fifth attempt, the pass rate is 20\%.
We record the thinking time of an attempt as the time between the task loads and the player makes the first action. The total thinking time of a player is the sum of the thinking time of all his/her attempts.
The number of attempts made and the total thinking time are scaled to 0-1 using min-max scaling in Figure \ref{fig_human_scaled}. Charts with the real values are available in the Appendix E.

Overall, human players have passed almost all the tasks in each scenario within the five attempts. On average, they used 1.86 attempts per task and took 23.73 seconds to think per task. On average, the low number of attempts to pass the tasks shows that the dexterity required to solve the tasks when the strategy is determined is low. The average thinking time per task shows that humans have to carefully think about the strategic physical rule required to solve the task.

\begin{figure}[t]
  \centering
  \includegraphics[width=1.0\linewidth]{Fig_human_scaled.PNG}
  \caption{Performance of human players. The x-axis is the scenario and y-axis is the scaled value.}
  \label{fig_human_scaled}
\end{figure}

Humans have the longest thinking time for the tasks in the adequate timing scenario but the average pass rate for these tasks is the second lowest. Similarly, the tasks from the non-greedy actions scenario has the lowest average pass rate with highest number of attempts while the thinking time is the second longest. This shows that figuring out the correct strategies for the tasks of these scenarios has been difficult for the humans.
In the relative weight scenario, the pass rate achieved within five attempts is the lowest. But the thinking time is average for this scenario. This suggests that some humans take the action without carefully thinking the strategy, and the strategy realized at a glance is not the correct strategy to solve the task. This also agrees with our observation that is humans are over confident in their wrong actions.

\subsection{Local and Broad Generalization Performance and Phy-Q Score}\label{loc_and_bro_results}

\begin{table}
    \scriptsize
    \centering
    \caption{Comparison of agents for \textit{local generalization} and \textit{broad generalization} (Phy-Q score). The results are shown for the nine agents: Bambirds (BB), Eagle's Wing (EW), Datalab (DL), Pig Shooter (PS), D-DDQN-Symbolic (D-DDQN-Sy), D-DDQN-Image (D-DDQN-Im), Relational-Symbolic (Rel-Sy), Relational-Image (Rel-Im), and Random. Learning agents have higher \textit{local generalization} values and lower values in \textit{broad generalization} than performance of heuristic agents. Human performance is way beyond agents.}
    \label{table generalization and phy-q}
    \begin{tabular}{cc|ccccccccc}
    \toprule
        & Human & BB & EW & DL & PS & D-DDQN-Sy & D-DDQN-Im & Rel-Sy & Rel-Im & Random \\
         \midrule
        Local gen. & - & 0.1536 & 0.1367 & 0.1481 & 0.1050 & \textbf{0.3350} & 0.2487 & 0.3220 & 0.2393 & 0.0338 \\
        \midrule
        Broad gen. & 0.8288 & 0.1776 & \textbf{0.1999} & 0.1880 & 0.1282 & 0.1209 & 0.0997 & 0.1411 & 0.0897 & 0.0427 \\

        \bottomrule
    \end{tabular}
\end{table}

\paragraph{\textit{Local generalization} performance}

Table \ref{table generalization and phy-q} first row presents the average \textit{local generalization} evaluation pass rate for all of our baseline agents, we have also included the full results of pass rate per agent per template in Appendix C. The table shows that the four learning agents perform significantly better than their heuristic counterparts. While both the symbolic learning agents and both the image learning agents on average pass approximately $33\%$ and $24\%$ of the test levels respectively, the previous champions in AIBIRDS competition-Bambirds, Eagle's Wing, and Datalab-pass around only half of the levels as compared with the learning agents, averaging $15\%$, $14\%$, and $15\%$ respectively. This agrees with what is generally accepted that deep learning systems can perform a single narrow task much better than heuristic methods when enough densely sampled training data is available.

\paragraph{\textit{Broad generalization} performance}

\begin{figure}[t]
  \centering
  \includegraphics[width=1.0\linewidth]{Fig_within_event_based_on_avg_pass_rate_submission_2.PNG}
  \caption{\textit{Broad generalization} of the baseline agents. The x-axis is the scenario and y-axis is the passing rate.}
  \label{fig_broad_generalization}
\end{figure}

Figure \ref{fig_broad_generalization} presents the average pass rate of test templates of the \textit{broad generalization} evaluation of all the baseline agents and human players. It is clear that the humans substantially outperform all the other agents, while all the agents have above-chance performance compared to the random agent. Heuristic agents achieved a better pass rate in the single force scenario (scenario 1) and multiple force scenario (scenario 2) as these two scenarios are corresponding to the essential ability needed to play Angry Birds - shooting directly at pigs.
It can be seen that heuristic agents generally perform better if the physical scenario is covered in their built-in rules. For example, Datalab and Eagle's Wing have a built-in strategic physical rule to roll round objects and they have the highest pass rate in scenario 3 (rolling) among all agents. For scenario 4 (falling) and scenario 13 (clearing paths), Bambirds dominates the leader board of pass rate because it explicitly analyses spatial relationships between blocks and pigs and is the only heuristic agent with the `prioritizing protective blocks' rule.

Second row in Table \ref{table generalization and phy-q} shows the overall average pass rate for the \textit{broad generalization} evaluation of the agents and humans. The heuristic agents' results were obtained in a similar way as the \textit{local generalization} evaluation, except we only consider the tasks from the testing task templates given to the learning agents. In contrast to the \textit{local generalization} results, in this evaluation setup, the learning agents have worse results than all the heuristic agents. The D-DDQN-Symbolic and the D-DDQN-Image agents have an average pass rate of 12\% and 10\% respectively while Relational-Symbolic and Relational-Image have 14\% and 9\% respectively. The champions in the AIBIRDS competition have almost double pass rate compared to the learning agents. Our result further advocates the claim that deep learning agents often exploit spurious statistical patterns instead of learning in a meaningful and generalizable way as humans do \cite{Chollet19,EBL,bengio2020,Yixin2020,marcus2020}.

\paragraph{Phy-Q score}
As discussed in Section \ref{section Evaluation Settings}, the Phy-Q score of humans is set to 100 while the random agent is set to 0. A Phy-Q score above 100 indicates superhuman performance.
Figure \ref{fig_phy_q_distribution} shows the positions of agents and humans in the Phy-Q score distribution. Even though Eagle's Wing was the first in the \textit{broad generalization} leader board (even after removing the results of the first two scenarios Eagle's Wing: 0.1142, Bambirds: 0.1022), Bambirds has taken the lead in the Phy-Q score dragging Eagle's Wing into the second place. This is because the Phy-Q score positions the agent with respect to the human performance.
Interestingly D-DDQN-Image agent and Relational-Symbolic agents have higher Phy-Q values compared to Datalab.
Similar to the above reason, this is due to the positioning of the agents with respect to the human performance.
Moreover, the Phy-Q of the Pig Shooter is negative. This result is expected as the Pig Shooter only shoots at the pigs, thus having a below chance performance compared to the random agent. Overall, it can be seen that all the agents are far below the humans' Phy-Q score.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.7\linewidth]{Fig_PhyQ_distribution.png}
  \caption{Phy-Q Score of humans and the agents}
  \label{fig_phy_q_distribution}
\end{figure}

\subsection{AIBIRDS Competition Performance}

\begin{table}
    \scriptsize
    \centering
    \caption{Comparison of AIBIRDS competition performance with Phy-Q score of heuristic agents}
    \label{table AIBIRDS competition results}
    \begin{tabular}{c|cccc}
    \toprule
        Rank from Phy-Q score & 1 & 2 & 3 & 4 \\
         \midrule
        Agent Name & Bambirds & Eagle's Wing & Datalab & Pig Shooter\\
        \midrule
        Competition performance & 0.2727 & 0.2000 & 0.2000 & 0.0000 \\
        \midrule
        Phy-Q score & 14 & 12 & 7 & -2\\
        \bottomrule
    \end{tabular}
\end{table}

Table \ref{table AIBIRDS competition results} presents the results of the AIBIRDS heuristic agents in the AIBIRDS competition game levels.
As it can be seen from the results, the pass rate of the agents in competition game levels agree to the rank they achieved using the Phy-Q score. Eagle's Wing and Datalab have achieved the same pass rate of 0.2. However, considering the total score, Eagle's Wing has obtained 667,394 while Datalab has obtained 634,435 taking Eagle's Wing to the second position.
Overall, these results portray that the tasks in the Phy-Q tested are representative to the tasks in the AIBIRDS competition.

From this result, we infer that the physical scenarios available in the Phy-Q tasks are also the scenarios that are commonly encountered in the AIBIRDS competition game levels. The within scenario (\textit{broad generalization}) evaluation we have conducted can be used to identify an agent's ability in performing in those individual physical scenarios. Therefore one can use the Phy-Q testbed to thoroughly analyze the physical reasoning capabilities of their AIBIRDS agent and figure out where the agent falls short and improve on those capabilities. Additionally, the human performance results we have established on the 15 scenarios facilitate to compare the agents' performance with humans' performance, allowing to set targets for agents to achieve human level performance in those scenarios. Thus, the Phy-Q testbed and the evaluation settings we proposed in the testbed can be used to better evaluate AIBIRDS agents and guide them towards achieving human level performance in the competition.

\section{Conclusion and Future Work}

The goal of Phy-Q is to facilitate the development of physical reasoning AI methods with broad generalizing abilities similar to that of humans.
As humans do not have accurate forward physics prediction models, we focus on tasks that agents can solve by understanding a strategic physical rule instead of solving tasks that require accurate forward prediction.
Therefore, towards the goal of developing AI methods similar to humans, we designed 75 task templates considering 15 different physical scenarios in our testbed.
The tasks that belong to the same physical scenario can be solved by a specific strategic physical rule. Therefore, it enables us to measure the \textit{broad generalization} of agents by allowing the agent to learn a strategic physical rule in the learning phase that can be used in the testing phase. Apart from the \textit{broad generalization} performance evaluation, the Phy-Q testbed also enables evaluating agents' \textit{local generalization} performance.
We have established baseline results from the testbed and have shown that even though current learning agents can generalize locally, the \textit{broad generalization} ability of these agents is below heuristic agents and is far below human performance.
Further, we have defined Phy-Q score, a score to reflect the physical reasoning ability of agents.
In addition, we have shown how the testbed can be used towards the advancement of the AIBIRDS competition agents.

Although we discourage developing heuristic agents with hard-coded rules that apply only to Angry Birds, we believe the superior performance of these rule-based systems, given that none of the agent developers has seen the Phy-Q tasks previously, indicates that the human extracted strategic physical rules are highly generalizable. Therefore,  we foresee several areas of improvement: 1) agents should learn and store generalizable abstract causal knowledge \cite{marcus2020}, e.g. strategic physical rules. For example, humans understand not only shooting a bird at a pig can destroy the pig, but also the pig is destroyed because collision between two objects creates an impact \cite{baillargeon_li_ng_yuan_2008} and if the impact is large enough, an object will be destroyed. One possible way to learn this abstract causal knowledge is through Explanation-Based Learning (EBL) \cite{EBL}, where an agent constructs an explanation for initial exemplars and then construct a candidate rule that depends only on the explanation; if the rule is proven true for a small number of additional exemplars, the rule is adapted. As the representation of abstract and causal knowledge allows for symbolic manipulation \cite{marcus2020}, 2) it is also worthwhile to explore the possibility of combining deep learning techniques with reasoning over knowledge systems in physical domains, where Neural-Symbolic methods such as NS-DR \cite{CLEVRER} and NS-CL \cite{DBLP:journals/corr/abs-1904-12584} have shown promising results on physical reasoning.

Phy-Q can be advanced in different directions. Characteristics such as deforming can be introduced to the objects in the tasks. Further, complex scenarios can be added to the testbed by combining the existing scenarios. This will also enable measuring of the combinatorial generalization of the agents. Moreover, additional physical scenarios that are not covered in the testbed can be introduced such as shape constancy, object permanence, spatiotemporal continuity, and causality. We hope that Phy-Q can provide a foundation for future research on developing AI agents with human-level physical reasoning capabilities, thereby coordinating research efforts towards ever new goals.

\bibliographystyle{unsrt}
{\small