Axon: Deep Learning in Elixir

ax·on/ˈakˌsän/nounthe long threadlike part of a nerve cell along which impulses are conducted from the cell body to other cells. Today I am excited to publicly announce Axon, a library for creating neural networks in Elixir. Axon is still pre-release; however, I believe it's reached a point where it's ready for experimentation and input from … Continue reading Axon: Deep Learning in Elixir →

Nx Tip of the Week #6 – Compiler or Backend?

I've recently seen some confusion with respect to compilers and backends. This post is intended to clear up some of that confusion. TLDR: If performance matters, benchmark and decide. If you need flexibility or want to prototype quickly and not sacrifice speed, backends are a good choice. If you need AOT compilation or your programs … Continue reading Nx Tip of the Week #6 – Compiler or Backend? →

Nx Tip of the Week #5 – Named Tensors

Note: The original named tensors article, Tensor Considered Harmful, goes through these details in much more detail and explains much better than I can. I recommend reading that as well. One of my biggest frustrations when working with NumPy and TensorFlow comes when working with axes. Take for example, this TensorFlow implementation of the Mean … Continue reading Nx Tip of the Week #5 – Named Tensors →

Nx Tip of the Week #4 – Using Keywords

Numerical definitions can only accept tensors or numbers as positional arguments; however, you can get around this inflexibility using keyword lists. You can pass and use optional keyword arguments in your numerical definitions with the keyword! method. Let's take a look at some ways this might be useful. Parameter Initializers In many ML applications, you … Continue reading Nx Tip of the Week #4 – Using Keywords →

Nx Tip of the Week #3 – Many Ways to Create Arrays*

*tensors In Nx, the fundamental type is the Tensor. You can think of a tensor as a multi-dimensional array, like the numpy.ndarray. For Elixir programmers, it's easy to think of Nx.Tensor as a list, or a list-of-lists, or a list-of-lists-of-lists, ... and so on. This thought process is fine, but it might lead you to … Continue reading Nx Tip of the Week #3 – Many Ways to Create Arrays* →

Nx Tip of the Week #2 – Tensor Operations for Elixir Programmers

In Elixir, it's common to manipulate data using the Enum module. Enum provides a set of library functions for working with types that implement the Enumerable protocol. The Enum module is a productive interface for manipulating lists, maps, sets, etc. However, learning how to think about tensor manipulation using Nx can be a bit difficult … Continue reading Nx Tip of the Week #2 – Tensor Operations for Elixir Programmers →

Nx Tip of the Week #1 – Using transforms

Note: This is an idea I had after learning a significant amount from the abseil C++ Tips of the Week during my work on EXLA. I'll keep writing them as long as there's interest. If there's anything in particular you'd like to read about, feel free to let me know! Nx is an exciting new … Continue reading Nx Tip of the Week #1 – Using transforms →

Continual Learning with Elastic Weight Consolidation in TensorFlow 2

Based on Overcoming catastrophic forgetting in neural networks. You can view the accompanying Jupyter Notebook here. The mammalian brain allows for the learning of tasks in a sequential order. That is, we are capable of learning new tasks without forgetting how to perform old ones. Research suggests that retention of task-specific skills relies primarily on … Continue reading Continual Learning with Elastic Weight Consolidation in TensorFlow 2 →