Machine Learning Notes

Commands

conda create -y -n sb3 python=3.11 # no mujoco2.3.7 version for python 3.12
conda activate sb3
conda install pip
pip install mujoco==2.3.7
pip install gymnasium[mujoco]
pip install stable-baselines3[extra]
# pip install gymnasium
# pip install git+https://github.com/Farama-Foundation/Gymnasium.git # addresses issue between mujoco3 and gymnasium .29
conda install -y jupyter
conda install -y -c conda-forge libstdcxx-ng #fix for anaconda linux, found here: https://stackoverflow.com/questions/71010343/cannot-load-swrast-and-iris-drivers-in-fedora-35/72200748#72200748
# conda env config vars set MUJOCO_GL=glfw PYOPENGL_PLATFORM=glfw
conda env config vars set MUJOCO_GL=osmesa PYOPENGL_PLATFORM=osmesa
conda deactivate && conda activate sb3

plain python environment

sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt update
sudo apt install python3.11-venv
python3.11 -m venv ~/envs/sb3
. envs/sb3/bin/activate
pip install --upgrade pip
pip install mujoco==2.3.7
pip install 'stable-baselines3[extra]'
pip install gymnasium[mujoco]
pip install jupyter
pip install mediapy

some things get installed in .local/bin, so we need to do

echo "export PATH=\$PATH:$HOME/.local/bin" >> .bashrc

conda activate sb3
git clone rl zoo
cd rl zoo
pip install -e .

Pytorch workflow

https://machinelearningmastery.com/pytorch-tutorial-develop-deep-learning-models/ https://keras.io/guides/writing_a_custom_training_loop_in_torch/

SB3

https://stable-baselines3.readthedocs.io/ https://github.com/DLR-RM/stable-baselines3?tab=readme-ov-file https://araffin.github.io/post/sb3/

Colab Notebooks

https://github.com/araffin/rl-tutorial-jnrr19 (made local) https://github.com/Stable-Baselines-Team/rl-colab-notebooks/tree/sb3 (made local)

tensorboard: https://stable-baselines3.readthedocs.io/en/master/guide/tensorboard.html

tensorboard --logdir leg1-tb/

RL baselines zoo

https://github.com/DLR-RM/rl-baselines3-zoo

https://huggingface.co/MattStammers/appo-mujoco_ant-sota

Mujoco

https://pypi.org/project/mujoco/#history https://mujoco.readthedocs.io/en/latest/XMLreference.html#actuator-position

IsaacSim

running in cpu mode

https://forums.developer.nvidia.com/t/run-isaac-gym-in-cpu-mode-without-a-cuda-capable-gpu-on-the-device/227954/2

MuJoCo / gymnasium

https://duckduckgo.com/?t=ffab&q=gymnasium+mujoco&ia=web https://gymnasium.farama.org/environments/mujoco/

Gymnasium

https://pypi.org/project/gymnasium/ https://gymnasium.farama.org/environments/mujoco/half_cheetah/ https://github.com/Farama-Foundation/Gymnasium/ https://github.com/Farama-Foundation/gym-examples

RL within gym https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/#sphx-glr-tutorials-training-agents-reinforce-invpend-gym-v26-py

render mode: gym jupyter stable baselines You tried to call render() but no render_mode was passed to the env constructor. https://github.com/openai/gym/issues/2780 https://github.com/openai/gym/issues/762