brain of mat kelcey...

rather than use all 13 channels in a multi spectral image for classification can we train a model that is robust to all combos, at all resolutions, and use a genetic algorithm to choose which are the most valuable? (spoiler; yes)

crazy large batch sizes

February 14, 2021 at 10:30 PM | categories: quick_hack, tpu, jax

a quick hack to see how fast we can get a v3-32 pod slice cranking with a global batch size of 170,000; tl-dr pretty fast!

solving y=mx+b... with jax on a tpu pod slice

February 07, 2021 at 01:00 PM | categories: tpu, ensemble_nets, jax, projects, haiku

a 4 (and a bit) part tutorial / colab / screencast series starting with jax fundamentals working up a data parallel approach to running on a cloud tpu pod slice... all focused on solving the toughest problem in machine learning; 1d y=mx+b

develomentor.com podcast interview

December 07, 2020 at 05:00 PM | categories: talk

was a guest on the develomentor podcast talking about random parts of my career

out of distribution detection using focal loss

December 02, 2020 at 01:00 PM | categories: objax, jax, projects

a series of small experiments on using focal loss to do out of distribution detection

my updated list of cool machine learning books

November 01, 2020 at 09:40 PM | categories: Uncategorized

it's been ten years so it's probably time to update my list of cool machine learning books.

dithernet very slow movie player

October 21, 2020 at 10:30 PM | categories: gan, jax, projects, objax

a GAN experiment to generate dithers for an eink screen minimising pixel change between frames for a very slow movie player.

ensemble networks

September 17, 2020 at 06:30 AM | categories: objax, projects, ensemble_nets, jax

ensemble nets; using jax vmap to batch over not just the inputs of a model but also sets of multiple models parameters.

metric learning for image similarity search in objax

September 02, 2020 at 12:00 PM | categories: objax, metric_learning, jax

an objax tutorial on using contrastive learning for image similarity.

objax on honeysuckle farm

August 30, 2020 at 02:45 PM | categories: talk

i think high level short explainer videos on jax frameworks while doing farm chores is going to be a growing genre.

the map interpretation of attention

August 19, 2020 at 10:30 AM | categories: talk, three_strikes_rule

a talk i did at melbourne ml/ai on how attention mechanism can be interpretated as a form of differentiable map. check out the recording!

self supervised learning and making use of unlabelled data

July 02, 2020 at 05:00 PM | categories: talk

a recording of a talk i did on self supervised learning at yow data.

a jax random embedding ensemble network

June 15, 2020 at 06:30 AM | categories: ensemble_nets, jax

random embedding networks can be used to generate weakly labelled data for contrastive learning and can be run in single model ensembles as a single forward pass in jax.