Relu is an activation function defined as:$$Relu(x)=\begin{cases}x, & x > 0 \\0, & otherwise\end{cases}$$. This function has an interesting property that it can act like an open/close switch. To see why, suppose $\vec{x}$ is a […]
Month: April 2026
Understand Why A Sole Linear Layer Does Not Classify Well
In deep neural network, the most frequently used component may be the linear layer. However, the linear layer itself does not work well as a classifier. In this article, I intend to explain why from […]