Machine Learning
Unit 6: Advanced Topics & MLOps (Bonus) / Deep Learning Fundamentals
1. Deep Learning Fundamentals
Deep Learning is a subset of ML based on Artificial Neural Networks (ANNs). It is designed to simulate the way the human brain processes information.
The Perceptron
The fundamental building block. It takes multiple inputs, multiplies them by weights, adds a bias, and passes the sum through an activation function.
y = f( Σ (wᵢxᵢ) + b )
Multi-Layer Perceptron (MLP)
A stack of perceptrons forming layers: an Input Layer, one or more hidden layers, and an Output Layer.
Deep Dive: Activation Functions
Activation functions introduce non-linearity, allowing the network to learn complex patterns.
| Function | Formula | Usage |
|---|---|---|
| Sigmoid | 1 / (1 + e⁻ˣ) | Binary classification output. Maps to (0, 1). |
| ReLU | max(0, x) | Hidden layers. Solves vanishing gradient. |
| Softmax | eˣⁱ / Σ eˣʲ | Multi-class classification output (probabilities sum to 1). |
Training: Forward & Backpropagation
- Forward Pass: Data flows from input to output, generating a prediction.
- Loss Calculation: Compare prediction to actual target (e.g., Cross-Entropy, MSE).
- Backward Pass (Backprop): Uses the Calculus Chain Rule to calculate the gradient of the loss with respect to every weight in the network.
- Gradient Descent: Update weights by stepping in the opposite direction of the gradient.