In partnership with

Stay up-to-date with AI

The Rundown is the most trusted AI newsletter in the world, with 1,000,000+ readers and exclusive interviews with AI leaders like Mark Zuckerberg, Demis Hassibis, Mustafa Suleyman, and more.

Their expert research team spends all day learning what’s new in AI and talking with industry experts, then distills the most important developments into one free email every morning.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

Sign up to start learning.

Mathematical Legos and the Foundation of Deep Learning

This past week, I had the opportunity to speak at a meetup in New York City. Shout out to the NYC BIM Group!

During the Q&A session, someone asked a fantastic question: What does someone need to know to start learning about Machine Learning and AI? It’s a great question—and one with many correct answers.

This week, I wanted to dive a little deeper into that question, lean right on in to the math part of “Not Magic, Just Math,” and outline the mathematical concepts that are essential to understand if you want to approach ML and AI with a solid foundation. Whether you aim to go beyond the level of a hobbyist or avoid being an ill-informed commentator, having a grasp of these ideas is key.

Mathematical Underpinning

Broadly speaking, the important concepts one needs to get comfortable with in Machine Learning (ML) and Artificial Intelligence (AI) come from the following branches of mathematics—listed in somewhat of an order of importance:

1. Linear Algebra – The clay to the pottery of AI

2. Statistics – Your tool belt to understand your data and problem space

3. Calculus – The engine of improvement in state-of-the-art learning systems

Linear Algebra

Linear algebra is the medium through which modern AI, particularly deep learning systems, flows. In every deep learning system, data is represented using key mathematical structures from linear algebra: scalars (0D), vectors (1D), matrices (2D), and tensors (3D and higher). These structures enable efficient processing, manipulation, and transformation of data, making them essential to the functionality of AI systems.

Why Is Data Transformation Important?

At its core, AI systems aim to learn the best manipulations and transformations to apply to raw data, enabling decision-making. Transforming data isn't just a mathematical exercise—it’s the essence of how AI derives meaning and generates useful outputs.

A Simplified Example: Coffee Cups vs. School Buses

Imagine you've designed an AI system to differentiate between images of coffee cups and school buses. By learning from many examples, the system identifies specific transformations it can apply to the input images to distinguish these two categories.

To see this in action, check out the CNN Explainer. This excellent web demo illustrates how each layer of a convolutional neural network processes input data. On the site, you’ll notice that every layer (represented as a column) applies a series of 10 different manipulations to the input information.

How Do These Transformations Work?

Each manipulation is created by multiplying two tensors:

1. The Input Tensor: The output from the previous layer.

2. The Layer's Internal Tensor of "Weights": These weights describe the learned transformation to apply to the incoming data.

Through this iterative process (Calculus!), the network gradually learns the most effective transformations to solve the task at hand.

A Concrete Example: Edge Detection

For a hands-on look at how linear algebra powers these transformations, consider the math behind edge detection in image processing. Edge detection involves applying a convolution operation—a form of matrix multiplication—to identify changes in intensity or color in an image.

Here is a great article on it: https://aryamansharda.medium.com/how-image-edge-detection-works-b759baac01e2

Once you grasp the math behind edge detection, you can start to internalize a key idea: The essence of AI is enabling models to learn the most useful mathematical transformations for completing the tasks you give them.

Building with Mathematical Legos

I heard AI described once as mathematical legos - I think that rings pretty true once you get a grasp of linear algebra. Eventually, each tensor becomes a Lego brick and the task at hand is to build the best learning system out of those bricks. Now the meaning of “best learning system” and how you look at your tensor bricks and determine how to put them together is the real art - but at the end of the day, that is what we are doing.

I’l admit it - I got a little over-excited and thought I’d cover statistics and calculus as well this week - but that will have to wait until another time - but just in case you want to jump ahead - check out the chain rule!

Thank you again for reading Not Magic, Just Math - if you haven’t already, and you made it this far - do consider subscribing - it means a lot to me every time I see a new person join the subscriber list!!

Have a good week!