Posts |

Tracking Experiments, Now Easier!

Fri, 23 Jul 2021 00:00:00 +0000

One aspect of doing reinforcement learning research that has been more annoying than I would like is keeping track of experiments and experimental results. While this may sound a bit like an infomercial, I have to say that using Weights and Bias’s integration with Stable Baselines 3 has made experiment tracking way easier than it was before!

Using it is pretty simple, update to the latest version of WandB and then use¹:

from wandb.integration.sb3 import WandbCallback

model.learn(..., callback=WandbCallback())

The basic example from WandB uses Tensorboard’s output to log metrics:

import time
import gym
from stable_baselines3 import PPO
from stable_baselines3.common.monitor import Monitor
from stable_baselines3.common.vec_env import DummyVecEnv, VecVideoRecorder
from wandb.integration.sb3 import WandbCallback
import wandb

config = {"policy_type": "MlpPolicy", "total_timesteps": 25000}
experiment_name = f"PPO_{int(time.time())}"

# Initialise a W&B run
wandb.init(
    name=experiment_name,
    project="sb3",
    config=config,
    sync_tensorboard=True,  # auto-upload sb3's tensorboard metrics
    monitor_gym=True,  # auto-upload the videos of agents playing the game
    save_code=True,  # optional
)

def make_env():
    env = gym.make("CartPole-v1")
    env = Monitor(env)  # record stats such as returns
    return env

env = DummyVecEnv([make_env])

env = VecVideoRecorder(env, "videos",
    record_video_trigger=lambda x: x % 2000 == 0, video_length=200)

model = PPO(config["policy_type"], env, verbose=1,
    tensorboard_log=f"runs/{experiment_name}")

# Add the WandbCallback 
model.learn(
    total_timesteps=config["total_timesteps"],
    callback=WandbCallback(
        gradient_save_freq=100,
        model_save_freq=1000,
        model_save_path=f"models/{experiment_name}",
    ),
)

Stable Baselines 3 ↩︎

Prepping the Haydn's, Week 0

Sun, 18 Jul 2021 00:00:00 +0000

Every year, Purdue Fort Wayne’s School of Music holds a student concerto competition. The student who wins this competition will perform their chosen concerto with the Community Orchestra during the Spring semester. My goal for this upcoming year will be to enter with Haydn’s Trumpet Concerto in Eb Major. In order to prepare for this entry, I have been developing a training schedule as follows:

Monday: Movement I
Tuesday: Movement II
Wednesday: Movement III
Thursday: Movement I
Friday: Movement II
Saturday: Movement III
Sunday: Endurance

The goal is to practice for at least 30 minutes per day, and slowly expand the amount of time per to day to at least 60 by the end of August.

Move to Hugo

Mon, 14 Dec 2020 00:00:00 +0000

You may have noticed that my site looks very different than it used to. I decided to move both my site and blog over to Hugo, partly due to my dissatisfaction with my old blogging software and my desire to make a more professional looking site as I get ready to start graduate school in Fall 2021.

Part of my dissatisfaction with the old setup, which used the fastai template, is it felt very fragile; I constantly felt like I was going to break something with no hope of being able to get it back. I also felt like it wasn’t a great way to demonstrate projects I was working on.

As you can see from the site as it sits right now, I have numerous small updates that need to be made, such as updating all of the cards that discuss what my research and career interests are.