Life Cycle of Training

Here is the standard "Life Cycle" of a single training step:

1. Forward Pass (The Guess)

• Linear: $z = w \cdot x + b$
• Activation: $a = f(z)$ (ReLU, Sigmoid, etc.)
• This happens layer by layer until you get a final prediction.

2. Loss Calculation (The Error)

• You compare the prediction to the Actual Target.
• Cross-Entropy: Used for classification (e.g., "Is this a cat or a dog?").
• MSE (Mean Squared Error): Used for regression (e.g., "What is the price of this house?").

3. Backpropagation (The Blame)

• You calculate the Gradients.
• This tells you exactly how much each weight ($w$) and bias ($b$) contributed to the error.

4. Optimization (The Fix)

• You use Gradient Descent to slightly nudge the weights in the direction that reduces the Loss.
• $w = w - (\text{learning rate} \cdot \text{gradient})$

Layer 1 (Weighted SUM -> Activation Function) ->Layer 2 (Weighted SUM -> Activation Function)

get output passing input via the layers for Neural networks

🏁 Summary

Forward (Guess) --> Loss (Check) --> Backward (Assign Blame) --> Optimizer (Update).
To keep going, do you want to:

• See how Cross-Entropy math actually works?
• Understand the difference between Weights and Biases in the update step?