brain of mat kelcey...

crazy large batch sizes

February 14, 2021 at 10:30 PM | categories: quick_hack, tpu, jax

a quick hack to see how fast we can get a v3-32 pod slice cranking with a global batch size of 170,000; tl-dr pretty fast!

solving y=mx+b... with jax on a tpu pod slice

February 07, 2021 at 01:00 PM | categories: tpu, ensemble_nets, jax, projects, haiku

a 4 (and a bit) part tutorial / colab / screencast series starting with jax fundamentals working up a data parallel approach to running on a cloud tpu pod slice... all focused on solving the toughest problem in machine learning; 1d y=mx+b

popular posts...

ensemble nets : training ensembles as a single model using jax on a tpu pod slice(sept 2020)

bnn : counting bees with a rasp pi (may 2018)

drivebot : learning to do laps with reinforcement learning and neural nets (feb 2016)

wikipedia philosophy : do all first links on wikipedia lead to philosophy? (aug 2011)

cartpole++ : deep RL hacking with a complex 3d cart pole environment (aug 2016)

malmomo : deep RL hacking on minecraft with malmo (jan 2017)

some papers from my time at google research / brain...

my honours thesis

the co-evolution of cooperative behaviour (1997) evolving neural nets with genetic algorithms for communication problems.

old projects...