Let $\varepsilon > 0$. If $x, x_0 \in \mathbb{R}^{N \times 1}$ such that $x \neq x_0$, then we have \begin{equation} \begin{split} \frac{|f(x) - f(x_0) - L(x - x_0)|}{|x - x_0|} & = \frac{|A x + b - (A x_0 + b) - A (x - x_0)|}{|x - x_0|} \\ & = \frac{|A x + b - A x_0 - b - A x + A x_0)|}{|x - x_0|} \\ & = \frac{|0|}{|x - x_0|} \\ & = 0 \\ & < \varepsilon \end{split} \end{equation} Since $\varepsilon > 0$ was arbitrary, the claim follows. $\square$