What Is a Neural Network? (From Math to Modern AI)

Source: Dev.to

Editorial Introduction

Artificial Intelligence didn’t become powerful overnight.
Behind today’s breakthroughs—image recognition, speech understanding, self‑driving cars, and tools like ChatGPT—lies a deceptively simple idea inspired by the human brain: neural networks.

These mathematical structures transformed how machines learn, shifting software from rigid rules to systems that learn patterns from data. If you want to understand modern AI, neural networks are not optional knowledge—they are the foundation.

Let’s break them down step by step.

What Is a Neural Network?

A neural network is a mathematical model inspired by how biological neurons process information. Instead of explicit instructions, it learns by observing examples and adjusting internal parameters.

Think of it as a system that answers questions like:

“Given this input, what is the most likely output?”

A Concrete Example: Handwritten Digit Recognition

Imagine drawing the number 3 on paper and converting that drawing into a 20 × 20 grayscale image (400 pixels). Each pixel holds a brightness value (0–100 %). These 400 numbers become the input neurons of the network.

Input Layer

400 neurons → one per pixel

Output Layer

The output layer contains 10 neurons, one for each digit (0–9). Each neuron outputs a probability.

Example output:

• Digit 3 → 0.90
• Digit 8 → 0.84

Neural networks think in probabilities, not absolute certainty—just like humans.

Hidden Layers: Where Intelligence Lives

Between input and output lie the hidden layers—the true brain of the system.

Example architecture:

• Input layer: 400 neurons
• Hidden layer 1: 15 neurons
• Hidden layer 2: 15 neurons
• Hidden layer 3: 15 neurons
• Output layer: 10 neurons

What Do Hidden Layers Learn?

They automatically extract features:

• Straight lines
• Curves
• Shape combinations

For example:

• 9 → circle + line
• 7 → angled lines

⚠️ More layers ≠ more intelligence. Overly complex networks can waste resources and even learn worse.

How Do Neural Networks Learn?

Learning happens during training, which follows this loop:

1. Initialization – Random weights are assigned.
2. Data Feeding – Thousands of labeled examples are shown.
3. Forward Propagation – Data flows through the network → prediction.
4. Error Calculation – Prediction is compared to the correct answer.
5. Backpropagation – Weights are adjusted using calculus and activation functions (ReLU, sigmoid).

🔁 One full pass through the dataset = epoch. Neural networks usually require many epochs to learn well.

What Changes During Training?

• Early layers detect simple patterns (lines).
• Middle layers detect shapes.
• Later layers detect complex structures.

By the end, the neuron for digit 3 activates strongly when shown a 3, while others remain quiet.

From Digits to Language Models

Digit recognition is simple. Language is not. Instead of pixels, language models use tokens (words, subwords, characters). English ≈ 50,000 tokens.

A language model like ChatGPT requires:

• Input layer: 50,000 neurons
• Massive hidden layers
• Output layer: 50,000 neurons

⚠️ Classic neural networks are not enough. Modern AI uses transformers and attention mechanisms to understand context.

Tools Behind Neural Networks

Neural networks rely on:

• Linear algebra (matrices & vectors)
• Calculus (gradients & derivatives)
• Probability
• Tensors

Popular frameworks:

• TensorFlow
• PyTorch

Minimal TensorFlow Example

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(15, activation='relu', input_shape=(400,)),
    tf.keras.layers.Dense(15, activation='relu'),
    tf.keras.layers.Dense(15, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

Why This Knowledge Matters

Understanding neural networks puts you in a small, high‑impact group of developers. AI is reshaping:

• Software engineering
• Medicine
• Finance
• Science
• Art

And neural networks are the engine behind it all.

Quick Recap

• Neural networks learn patterns, not rules.
• Built from layers of neurons and weights.
• Learn using backpropagation.
• Powered by math, not magic.
• Different architectures solve different problems (CNNs, RNNs, Transformers).

Final Question

Have you ever implemented a neural network—even a small one?
What confused you the most when learning AI?

Let’s discuss below.