Neural Network Basics

Deep Learning is inspired by the human brain. Just as our brain is made of billions of connected neurons, a Neural Network is made of layers of artificial neurons.

1. The Artificial Neuron 🧠

An artificial neuron (or Perceptron) is a simple mathematical function.

Output = Activation(\sum (Inputs \times Weights) + Bias)

1. Inputs ($x$): The data coming in (e.g., pixel values).
2. Weights ($w$): How important each input is.
3. Bias ($b$): Allows the neuron to fire even when inputs are zero.
4. Activation Function: Decides if the neuron should "fire" (output a signal).

2. Layers of a Network 🍰

• Input Layer: Receives the raw data.
• Hidden Layers: The "magic" happens here. They extract features. Deep Learning = Many hidden layers.
• Output Layer: Gives the final prediction (e.g., "Cat" or "Dog").

3. Activation Functions ⚡

Without activation functions, a neural network is just a big linear regression model. We need non-linearity to learn complex patterns.

• ReLU (Rectified Linear Unit): max(0, x). The most popular choice. Fast and effective.
• Sigmoid: Squashes output between 0 and 1. Used for binary classification.
• Softmax: Converts outputs into probabilities (sum = 1). Used for multi-class classification.

4. How It Learns: Forward & Backward 🔄

1. Forward Pass: Data flows through the network -> Prediction.
2. Calculate Loss: Compare Prediction vs Actual.
3. Backward Pass (Backpropagation): Go back and adjust weights to reduce error.
4. Optimizer: The algorithm that updates weights (e.g., SGD, Adam).

Interactive Challenge: Build a Neuron

Calculate the output of a single neuron manually.

Quiz

Key Takeaways

✅ Neurons combine inputs, weights, and bias.
✅ Activation Functions (ReLU, Sigmoid) add non-linearity.
✅ Backpropagation is how the network learns from mistakes.

What's Next?

Building neurons from scratch is hard. Let's use a framework to do the heavy lifting.

Next Chapter: Introduction to TensorFlow / PyTorch.