Neural Network Architecture

Question 1

What is a "Hidden Layer"?

Accepted Answer

Any layer between input and output — Hidden layers are located between the input and output of the algorithm, where the function applies weights to the inputs and directs them through an activation function.

Question 2

A neural network is organized into…

Accepted Answer

Layers of interconnected neurons — Data flows through an input layer, one or more hidden layers, and an output layer, each made of neurons applying weights and activations.

Question 3

Each neuron computes…

Accepted Answer

A weighted sum plus a bias, activated — output = activation(Σ wᵢxᵢ + b). The weights and bias are learned; the activation adds non-linearity.

Question 4

What makes a network 'DEEP'?

Accepted Answer

Having many hidden layers stacked — Depth refers to the number of layers; deep networks learn hierarchical features, each layer building on the previous.

Question 5

The Universal Approximation Theorem states that a neural network can…

Accepted Answer

Approximate any continuous function — Sufficiently wide networks are universal function approximators in theory; depth makes this far more parameter-efficient in practice.

Question 6

In a fully connected (dense) layer, each neuron is connected to…

Accepted Answer

Every neuron of the previous layer — Dense layers connect all inputs to all outputs via a weight matrix, computed as a matrix–vector multiply plus bias.

Question 7

The forward pass of a layer is essentially…

Accepted Answer

A matrix multiplication plus bias addition — Each layer transforms its input vector by W·x + b followed by a non-linearity — efficient on GPUs as batched matrix ops.

Question 8

Weights and biases in a neural network are…

Accepted Answer

Learned parameters, adjusted during training — Training uses backprop + an optimizer to update these parameters so the network's predictions improve.

Question 9

Why do hidden layers enable learning complex patterns?

Accepted Answer

Non-linear layers compose abstract features — Each layer transforms features non-linearly; stacking builds a hierarchy (edges→shapes→objects) — representation learning.

Question 10

Network DEPTH vs WIDTH: increasing depth tends to…

Accepted Answer

Build more abstract, hierarchical features — Deeper networks reuse and compose features across layers, often achieving more with fewer parameters than a single very wide layer.

Neural Network Visualizer

Neural Network: 2 → 4 → 3 → 1 Architecture

Live Forward Pass Computation

Name the layers

Test your knowledge

What is a "Hidden Layer"?

Learn these first

Interview Questions

References & Further Reading

Love this visualization?