Gym.jl is a package that implements a subset of the Gymnasium RL environments in Julia.

Installation

Clone the repository and instantiate the project environment:

git clone https://github.com/scascino4/Gym.jl
cd Gym.jl
julia --project=. -e 'using Pkg; Pkg.resolve(); Pkg.instantiate()'

To inspect the currently registered environments:

julia --project=. -e 'using Gym; Gym.pprint_registry()'

Quick Start

From the repository root, start Julia with julia --project=. and run:

using Gym

env = Gym.make("CartPole-v1"; render_mode=:human)
observation, info = Gym.reset!(env)

while true
    action = rand(Gym.action_space(env))
    observation, reward, terminated, truncated, info = Gym.step!(env, action)
    if terminated || truncated
        break
    end
end

Gym.close(env)

render_mode=:human uses an OpenGL renderer backend, so it requires a working OpenGL 3.3 context at runtime. Some rendered environments also load packaged PNG assets through project dependencies, so run Pkg.instantiate() after pulling dependency changes.

Vector environments are available through Gym.make_vec(...), like:

envs = Gym.make_vec("CartPole-v1"; num_envs=2, vectorization_mode=:async, render_mode=:human)
observations, infos = Gym.reset!(envs)

vectorization_mode=:async runs one worker task per environment. For real parallel speedups, start Julia with multiple threads, for example julia --project=. -t auto.

Repository Guide

• src/: package source, spaces, vector environments, and environment implementations
• examples/: scripts that use and validate environment implementations
• benchmark/: scalar step-throughput benchmarks against Gymnasium
• test/: Julia tests and the Gymnasium compatibility harness

Examples

The examples/ directory contains example usages of some of the defined environments. From a repository checkout, run scripts like:

julia --project=. examples/ppo.jl CartPole-v1
julia --project=. examples/ppo.jl CartPole-v1 --render-final
julia --project=. examples/dqn.jl Acrobot-v1 --render-final

Tests

The test/ directory contains the Julia package tests and the optional Python compatibility harness used to compare environments against Gymnasium.

From a repository checkout, run the Julia test suite with:

julia --project=. test/runtests.jl

The compatibility checks require:

• Python 3.10+
• uv

On first use, uv will create test/.venv and install compatible dependencies for the local harness. If uv is missing, the compatibility checks are skipped automatically with an info message.

To force skipping the Gymnasium compatibility harness and run only the Julia tests:

julia --project=. test/runtests.jl --no-compatibility

Documentation

Full package documentation lives under docs/ and can be built locally with:

julia --project=docs docs/make.jl

Environment Checklist

This checklist is copied from the core Gymnasium environments. You can inspect the currently available environments at any time with Gym.registered_envs() or Gym.pprint_registry().

• Classic Control (5/5)

  • Acrobot
  • CartPole
  • Mountain Car
  • Continuous Mountain Car
  • Pendulum

• Toy Text (4/4)

  • Blackjack
  • Taxi
  • Cliff Walking
  • Frozen Lake

Box2D, MuJoCo, and Atari environments use external engines, so the benefits of a Julia re-implementation are quite limited. For this reason, they're (currently) out of scope.

Benchmarks

The benchmark/ directory contains a scalar environment throughput benchmark that compares Gym.jl against Gymnasium. From a repository checkout, regenerate the plot with:

julia --project=. benchmark/benchmark.jl
gnuplot benchmark/plot_benchmark.gp