Neural networks: Each individual neuron has a bunch of weights + a bias.  In total, this simple network:

• Each neuron is a polynomial with a hunch of multiplications of weights x the output of each of the prior layers' neurons

• Neural networks exploit nonlinearity, which allows them to express a much wider set of possible functions, whereas linear regression is relatively limited.
• Neural networks are learned using gradient descent, meaning their power isn't upper-bounded by the algorithms which programmers or mathematicians can feasibly design by hand.
• Per Claudio: pre-training, mid-training, etc - each of them have a different loss function that you optimize for
  • Learning the internet, answering in certain ways; reducing harm, etc
  • Doesn't have to be self-supervised. Can develop for example policies.

Activation functions: A neuron computes a weighted sum of its inputs, something like: z = w₁x₁ + w₂x₂ + w₃x₃ + b

• That's just a linear combination. Then the activation function f takes z and produces the neuron's actual output: a = f(z)
• The activation function bends the output in a non-linear way, and that's what lets networks learn complex patterns like curves, edges, shapes, and eventually abstract concepts.
  • ReLU (Rectified Linear Unit) is the most popular. It's dead simple: if z is positive, output z; if z is negative, output 0. Just f(z) = max(0, z). This creates a "kink" at zero, which is enough non-linearity to work, and it's extremely fast to compute.
  • Sigmoid squashes any input into a value between 0 and 1, creating an S-shaped curve. Useful when you want to interpret output as a probability, but it's fallen out of favor for hidden layers because it causes training problems in deep networks.
    • ReLU more effectively avoids the vanishing gradient problem, which is common in sigmoids.
  • Tanh is similar to sigmoid but squashes to the range -1 to 1.
  • Softmax, which we discussed earlier, is a special one used at the output layer for classification. It takes a whole vector of numbers and converts them into probabilities that sum to 1.

Backprop: efficient way of changing weights to get to more accurate results / reduce loss

• "Starting from the loss at the end, you work backwards through the network and figure out, for every single weight, how much it contributed to the error. This uses the chain rule from calculus: the error at the output depends on the last layer, which depends on the layer before it, and so on, so you can propagate the "blame" layer by layer in reverse."
• Vanishing gradient problem:
  • "gradients become extremely small as they're propagated backward through layers, so weights in earlier layers barely update and the network struggles to learn."
  • "By the time the gradient reaches the early layers, it's effectively zero."
  • "ReLU helps because its derivative is exactly 1 for any positive input (and 0 for negative). For active neurons, the gradient passes through the layer unchanged rather than being attenuated. So gradients can flow much further back through a deep network without collapsing toward zero."

Vectors vs Points

• A point is a location in space. It answers "where?" It has no size or direction, just position relative to some origin.
• A **vector is a displacement** — a direction with a magnitude. It answers "how far and which way?"
  • A vector doesn't have a fixed location; the vector (3, 4) means "move 3 right and 4 up" whether you start from the origin, from (10, 10), or anywhere else.
  • A 2-dimensional vector like (3, 4) is just the most common example because it's easy to draw on paper. But vectors come in all sizes
  • Two embedding vectors pointing in similar directions (small angle between them) represent semantically similar words — that's literally how similarity search works under the hood.
• First number tells you how far to walk along x axis; second along y axis. Distinguished from points thru vertical stacking.
• Basis vectors: i-hat - vector pointing one to the right; j-hat; unit pointing one straight up
• Scalars - numbers that scale vectors (e.g. multiply them)
• Length: vectors are tuples (ordered list of values) of numbers of any length, and the 2D arrows you draw in geometry class are just the easiest case to visualize.
  • A word embedding might be a 768-dimensional vector representing the "meaning" of a word in some learned space.