Theorem 1
Let $A \in \mathbb C^{n \times n}$. $A$ is injective if and only if $\ker (A) = \{ 0 \}$.
Proof.
$\boxed \Rightarrow$ : Let $v \in \ker (A)$. Then
$$\begin{align} Av = 0 &\Rightarrow A(-v) = - Av = 0 \\ &\hspace{-0.8em} \stackrel{\text{injective}}{\Rightarrow} v = -v \\ &\Rightarrow v = 0 \end{align}$$
$\boxed \Leftarrow$ : Let $A v_1 = A v_2$. Then
$$\begin{align} A(v_1 - v_2) = A v_1 - A v_2 = 0 &\Rightarrow v_1 - v_2 \in \ker (A) \\&\hspace{-1.4em} \stackrel{\ker (A) = \{ 0\}}{\Rightarrow} v_1 - v_2 = 0 \\ &\Rightarrow v_1 = v_2 \\ & && & \square \end{align}$$
Remark
$\| A \|_2$ is also equal to the largest singular value of $A$. In addition, since $A^HA$ (and $A A ^H$) is a Hermitian positive semi-definite matrix, the spectral radius of $A ^H A$ (and $A A ^H$) is simply the largest eigenvalue of $A^HA$, and we can write
$$\| A \|_2 = \sqrt{\lambda_{max} ( A ^H A )} = \sqrt{\lambda_{max} ( A A ^H)}$$
Definition (Norm equivalence)
The norms $\| \cdot \|_p$ and $\| \cdot \|_q$ over a space $V$ are equivalent if there exists positive constants $\alpha, \beta > 0$ such that for all $A \in V$
$$ \alpha \| A \|_p \leq \| A \|_q \leq \beta \| A \|_p$$
Definition (Similarity and diagonalisability)
$A \in \mathbb C^{n \times n}$ is similar to $B \in \mathbb C^{n \times n}$ if there exists an invertible $S \in \mathbb C^{n \times n}$ such that $S^{-1} A S = B$.
If $S$ is unitary, i.e. $S^{-1} = S^H$, then $A$ and $B$ are unitarily similar.
$A \in \mathbb C^{n \times n}$ is diagonalisable if it is similar to a diagonal matrix.
Remark (Eigenvalues of similar matrices)
Similar matrices have equal eigenvalues of equal multiplicities. If $(\lambda, v)$ is an eigenpair of $A$ and $B = S^{-1} A S$, then $(\lambda, S^{-1} v)$ is an eigenpair of $B$. Indeed, since
$$ B S^{-1} v = S^{-1} A S S^{-1} v = S^{-1} A v = \lambda S^{-1} v.$$
Thus, the eigenvalues of $A$ and $B$ are equal and their multiplicities the same.
Remark
If $A$ and $B$ are unitarily similar and $A$ is Hermitian, so is $B$, since
$$ B = U^{-1} A U = U^H A U.$$
Remark
$A \in \mathbb C^{n \times n}$ is diagonalizable if and only is there exists $n$ linearly independent eigenvectors $v_i, i = 1, \dots n$. To see this, we collect all eigenvectors as columns
$$ S = \begin{bmatrix} v_1 & v_2 & \dots & v_n \\ \downarrow & \downarrow & & \downarrow \end{bmatrix}$$
implying $AS = S \Lambda$ with $\Lambda = \text{diag} (\lambda_1, \dots, \lambda_n)$. Since $v_i$ are linearly independent, $S$ is invertible, thus $S^{-1} A S = \Lambda$.
Remark (Invariance of trace)
The trace is invariant under a similarity transform. Indeed, noticing $\text{tr} (ABC) = \text{tr} (BCA)$, we obtain
$$ \text{tr} ( S ^{-1} A S ) = \text{tr} (A S S^{-1}) = \text{tr} (A)$$
Definitions (Trace, kernel, image)
Let $A \in \mathbb C^{n \times n}$. We further define the following quantities
$$\begin{align} &\text{Trace} & \text{tr} (A) &\coloneqq \sum_i A_{ii} \\ &\text{Kernel} & \ker (A) &\coloneqq \{ v \in \mathbb C^n \mid Av = 0\} \\ &\text{Image} & \operatorname{im} (A) &\coloneqq \{ w \in \mathbb C^n \mid \exists v \in \mathbb C^n \text{ s.t. } Av = w \} \end{align}$$
Another important matrix norm is the Frobenius norm
$$\| A \|_F = \sqrt{\mathrm{tr} ( A A ^H)} = \sqrt{\sum_{i,j = 1}^n | A _{ij}^2|}$$
Its relevance will become clear once we consider the equivalence of matrix norms, which we treat in the next section.
Show outline
Proposition 2 (Equivalence of Frobenius norm and 2-norm)
We consider the vector space of matrices $A \in \mathbb C^{n \times n}$. The Frobenius norm $\| \cdot \|_F$ and the spectral norm $\| \cdot \|_2$ are equivalent. Moreover,
$$ \| A \|_2 \stackrel{(a)}{\leq} \| A \|_F \stackrel{(b)}{\leq} \sqrt{n} \| A \|_2$$
This relationship is relevant as $\| A \|_2$ proves useful in theory but is hard to compute, unlike $\| A \|_F$ for which computation is quite straightforward.
Proof. (a) is left as an exercise. For (b), we have
$$\begin{align} \| A \|_F^2 = \text{tr} ( A ^H A ) = \sum_{i=1}^n \lambda_i( A ^H A) \leq \sum_{i=1}^n \lambda_{max} ( A ^H A ) = n \| A \|_2^2 && \square \end{align}$$
Some matrix concepts that may be useful to have handy for the course. For more details, see Youssef Saad. Numerical Methods for Large Eigenvalue Problems, SIAM (2011).
Remark
$\ker(A)$ and $\operatorname{im} (A)$ are subspaces of $\mathbb C^n$, and
$$\dim ( \ker(A)) + \dim (\operatorname{im}(A)) = n.$$
Recall the conditions a norm should satisfy, for all $A \in \mathbb C^{n\times n}$ :
$$\begin{align} \| A \| &\geq 0 && \| A \| = 0 \iff A = 0 \\ \| \alpha A \| &= | \alpha| \| A \| && \forall \alpha \in \mathbb C \\ \| A + B \| &\leq \| A \| + \| B \| && \forall B \in \mathbb C^{n \times n} \tag{$\triangle$} \end{align}$$
The third condition is known as the triangle inequality and sometimes denoted as $\triangle$.
The $p$-norms of $\mathbb C^n$ ($\| v \|_p = (\sum_{i=1}^n |v_i|^p)^{1/p} \quad \forall v \in \mathbb C^n$) inherit a matrix norm for $p \geq 1$ or $p = \infty$ as
$$\| A \|_p = \max_{0 \neq v \in \mathbb C^n} \frac{\| A v \|_p}{\|v\|_p},$$
the so-called induced matrix norm.
They are always consistent with the associated vector norm, i.e.
$$\| A v \|_p = \| v \|_p \frac{\| A v \|_p }{\|v\|_p} \leq \| v\|_p \max_{0 \neq v \in \mathbb C^n} \frac{\| A v \|_p}{\| v \|_p} = \| A \|_p \| v \|_p.$$
Further, we obtain
$$ \| A B \|_p = \max_{x \neq v \in \mathbb C^n} \frac{\| A B v \|_p}{\| v \|_p} \leq \| A \|_p \max_{0 \neq v \in \mathbb C^n } \frac{\| B v \|_p}{\| v\|_p} = \| A \|_p \| B \|_p$$
Thus,
$$\begin{align} \| A ^2 \|_p &\leq \| A \|_p^2. \end{align}$$
More generally,
$$\begin{align} \| A ^k \|_p &\leq \| A \|_p^k. \end{align}$$
Clearly, $A \in \mathbb C^{n \times n}$ is a vector space. What are good norms for it ?
Here are some explicit definitions for induced matrix norms
$$\begin{align} \| A \|_1 &= \max_{j = 1, \dots, n} \sum_{i=1}^n | A_{ij} | \tag{Column-sum norm} \\ \| A \|_\infty &= \max_{i = 1, \dots, n} \sum_{j=1}^n |A_{ij}| \tag{Row-sum norm} \\ \| A\|_2 &= \sqrt{ϱ (A A^H)} = \sqrt{ϱ(A^H A)} \end{align}$$
Here, $A^H$ denotes the Hermitian conjugate, i.e. $A^H = \overline{ A ^T} = (\bar A ) ^T$ and $ϱ$ is the spectral radius of $A$. For those more familiar with linear algebra over the reals, Hermitian matrices ($A^H = A$) generalize many properties of real symmetric matrices, which are themselves all Hermitian.