Limited Sensors: For this variation, we restrict the obser-
vations to only provide positional information (including
joint angles), excluding velocities. An agent now has to
learn to infer velocity information in order to recover the
full state. Similar tasks have been explored in Gomez &
Miikkulainen (1998); Sch¨afer & Udluft (2005); Heess et al.
(2015a); Wierstra et al. (2007).
Noisy Observations and Delayed Actions: In this case,
sensor noise is simulated through the addition of Gaussian
noise to the observations. We also introduce a time de-
lay between taking an action and the action being in effect,
accounting for physical latencies (Hester & Stone, 2013).
Agents now need to learn to integrate both past observa-
tions and past actions to infer the current state. Similar
tasks have been proposed in Bakker (2001).
System Identification: For this category, the underly-
ing physical model parameters are varied across different
episodes (Szita et al., 2003). The agents must learn to gen-
eralize across different models, as well as to infer the model
parameters from its observation and action history.

Partially Observable Tasks（部分可观测任务）

有限感知（Limited Sensors）

在该变体任务中，我们对观测进行限制，仅提供位置信息（包括关节角度），剔除速度信息。此时智能体必须学会推断速度，才能恢复出完整系统状态。Gomez & Miikkulainen (1998)、Schäfer & Udluft (2005)、Heess et al. (2015a) 以及 Wierstra et al. (2007) 均对类似任务进行过研究。

带噪声观测与动作延迟（Noisy Observations and Delayed Actions）

在该设置下，我们通过对观测值叠加高斯噪声来模拟传感器噪声。同时，我们在智能体执行动作与动作实际生效之间引入时间延迟，以模拟真实物理系统中的固有延迟（Hester & Stone, 2013）。智能体需要学习融合历史观测与历史动作序列，从而推断当前系统状态。Bakker (2001) 中也曾提出过类似任务设定。

系统辨识（System Identification）

在这一类任务中，底层物理模型参数在不同回合间随机变化。智能体不仅需要在不同动力学参数下具备泛化能力，还必须从历史观测与动作序列中推断出当前的模型参数。

Partially Observable Tasks（部分可观测任务）：

• 部分可观测马尔可夫决策过程（POMDP）类

• 含扰动与时延的鲁棒控制类（Perturbation & Delay）

• 动态系统辨识 + 元强化学习类（Meta-RL / System ID）