A small glossary for a small robot

Policy, observation and action

Policy

A controller that maps what it observes to an action. A learned policy acquires that mapping through training.

Observation

The information supplied to the policy at a step. Its ordering and normalization are part of the interface, not incidental formatting.

Action

The output used by the control system. Its meaning depends on the environment and runtime.

Training vocabulary

Reinforcement learning

Learning from interaction with an environment and a reward signal.

PPO

Proximal Policy Optimization, a method for updating policies during training.

Reward

A numerical training signal. A high reward only means the objective was satisfied as defined; it does not establish every behavior you might care about.

Domain randomization

Varying simulated conditions during training to reduce dependence on one exact setup.

Sim-to-real

The process of transferring and evaluating behavior learned in simulation on physical hardware.

Software vocabulary

MuJoCo

A physics simulator used in robotics research and development.

ONNX

A format for representing machine-learning models for use across tools.

Manifest

Metadata accompanying an artifact. The Microduck reference defines what a policy publisher must provide.

Daemon

A background service responsible for part of the robot’s software.

JSON-RPC

A protocol for representing method calls and their responses using JSON.

Read a term in its context

When you encounter an unfamiliar term, first identify whether the document is describing training, runtime operation or a proposed design. The same broad concept can have a very specific interface in a project.

Build your own glossary as you read: write the term, the source file and one concrete example. Use the resource directory to return to the original reference whenever an implementation detail matters.