Artificial Neural Networks (ANNs)

These notes record of my evolving understanding of neural networks..

1. Overview of ANNs
2. Neural Net Math
   1. Multi-Dimensional Hilbert Space
   2. Linear Algebra
3. Class Notes from an ANN class (Graphical)
4. Glossary of ANN terms

Overview

Neural Net Math

It's entirely possible to set up a bunch of "neuron" objects in code, and allow them to send each other messages, the way real neurons do. While this is probably cute, it's also wasteful of computer resources, because there's a much more elegant, mathematical system for representing neural nets.

This is a number line:

<---- -3 -- -2 -- -1 -- 0 -- 1 -- 2 -- 3 -->

You probably learned about the number line in kindergarten. It's pretty simple, a line with zero in the center, positive numbers on the right, and negative numbers on the left. You can add and subtract with the number line, and tell the distance between two points by taking the absolute value (square root of the square) of one coordinate minus the other:

d=sqrt((X₁ - X₂)^{^2})

If you take two perpendicular number lines that cross at their zeroes, you get a cartesian coordinate system:

You could imagine another

These are some class notes I made for my "Artificial Neural Networks" class in college.... I'll interpret them soon (they're quite graphical, though, so if you're visually minded and know a bit about math, they might make sense...

overview (47K gif)
vectors & matrices (46K gif)
hebbian learning & eigenvectors (53K gif)