【注】对于相同的特殊运算不同书籍可能使用的符号不同，本文采用与 Wikipedia 相同的符号表示。

1. 直和

1.1 定义

m×mm \times m 矩阵 A\boldsymbol{A} 与 n×nn \times n 矩阵 B\boldsymbol{B} 的直和记作 A⊕B \boldsymbol{A} \oplus \boldsymbol{B} ，它是一个 (m+n)×(m+n)(m+n) \times (m+n) 的矩阵，定义为

A⊕B=[A0m×n0n×mB]\begin{array}{c} \boldsymbol{A} \oplus \boldsymbol{B} = \left[ \begin{matrix} \boldsymbol{A} & \boldsymbol{0}_{m \times n} \\ \boldsymbol{0}_{n \times m} & \boldsymbol{B} \end{matrix} \right] \end{array}

1.2 性质

  • 若 cc 为常数，则 c(A⊕B)=cA⊕cB c(\boldsymbol{A} \oplus \boldsymbol{B}) = c\boldsymbol{A} \oplus c\boldsymbol{B} 。

  • 直和通常不满足交换性质，A⊕B=B⊕A \boldsymbol{A} \oplus \boldsymbol{B} = \boldsymbol{B} \oplus \boldsymbol{A} 当且仅当 A=B \boldsymbol{A} = \boldsymbol{B} 。

  • 若 A,B\boldsymbol{A},\boldsymbol{B} 为 m×mm \times m 矩阵，且 C,D\boldsymbol{C},\boldsymbol{D} 为 n×nn \times n 矩阵，则

(A±B)⊕(C±D)=(A⊕C)±(B⊕D)(A⊕C)(B⊕D)=AB⊕CD\begin{array}{c} (\boldsymbol{A} \pm \boldsymbol{B}) \oplus (\boldsymbol{C} \pm \boldsymbol{D}) = (\boldsymbol{A} \oplus \boldsymbol{C}) \pm (\boldsymbol{B} \oplus \boldsymbol{D}) \\ (\boldsymbol{A} \oplus \boldsymbol{C})(\boldsymbol{B} \oplus \boldsymbol{D}) = \boldsymbol{AB} \oplus \boldsymbol{CD} \end{array}

  • 若 A,B,C\boldsymbol{A},\boldsymbol{B},\boldsymbol{C} 分别是 m×m,n×n,p×pm \times m, n \times n, p \times p 矩阵，则

A⊕(B⊕C)=(A⊕B)⊕C=A⊕B⊕C\begin{array}{c} \boldsymbol{A} \oplus (\boldsymbol{B} \oplus \boldsymbol{C}) = (\boldsymbol{A} \oplus \boldsymbol{B}) \oplus \boldsymbol{C} = \boldsymbol{A} \oplus \boldsymbol{B} \oplus \boldsymbol{C} \end{array}

  • 若 Am×mA_{m \times m} 和 Bn×nB_{n \times n} 均为正交矩阵，则 A⊕B \boldsymbol{A} \oplus \boldsymbol{B} 是 (m+n)×(m+n)(m+n) \times (m+n) 正交矩阵。

  • 矩阵直和的复共轭、转置、复共轭转置与逆矩阵（若 A,B\boldsymbol{A}, \boldsymbol{B} 可逆）的关系：

(A⊕B)∗=A∗⊕B∗(A⊕B)T=AT⊕BT(A⊕B)H=AH⊕BH(A⊕B)−1=A−1⊕B−1\begin{array}{c} (\boldsymbol{A} \oplus \boldsymbol{B})^* = \boldsymbol{A}^* \oplus \boldsymbol{B}^* \\ (\boldsymbol{A} \oplus \boldsymbol{B})^T = \boldsymbol{A}^T \oplus \boldsymbol{B}^T \\ (\boldsymbol{A} \oplus \boldsymbol{B})^H = \boldsymbol{A}^H \oplus \boldsymbol{B}^H \\ (\boldsymbol{A} \oplus \boldsymbol{B})^{-1} = \boldsymbol{A}^{-1} \oplus \boldsymbol{B}^{-1} \end{array}

2. Hadamard 积

2.1 定义

m×nm \times n 矩阵 A=[aij]\boldsymbol{A} = [a_{ij}] 与 m×nm \times n 矩阵 B=[bij]\boldsymbol{B} = [b_{ij}] 的 Hadamard 积（也称为 Schur 积或对应元素乘积）记作 A⊙B \boldsymbol{A} \odot \boldsymbol{B} （或 A∘B\boldsymbol{A} \circ \boldsymbol{B}），它仍然是一个 m×nm \times n 矩阵，其元素定义为两个矩阵对应元素的乘积：

(A⊙B)ij=aijbij\begin{array}{c} (\boldsymbol{A} \odot \boldsymbol{B})_{ij} = a_{ij}b_{ij} \end{array}

即 Hadamard 积是一映射 Rm×n×Rm×n↦Rm×n \mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \mapsto \mathbb{R}^{m \times n} 。

2.2 性质

2.2.1 基本性质

  • A⊙B=B⊙AA⊙(B⊙C)=(A⊙B)⊙CA⊙(B±C)=A⊙B±A⊙C \boldsymbol{A} \odot \boldsymbol{B} = \boldsymbol{B} \odot \boldsymbol{A} \\ \boldsymbol{A} \odot (\boldsymbol{B} \odot \boldsymbol{C}) = (\boldsymbol{A} \odot \boldsymbol{B}) \odot \boldsymbol{C} \\ \boldsymbol{A} \odot (\boldsymbol{B} \pm \boldsymbol{C}) = \boldsymbol{A} \odot \boldsymbol{B} \pm \boldsymbol{A} \odot \boldsymbol{C}

  • 若 A,B\boldsymbol{A},\boldsymbol{B} 均为 m×nm \times n 矩阵，则

(A⊙B)T=AT⊙BT(A⊙B)H=AH⊙BH(A⊙B)∗=A∗⊙B∗\begin{array}{c} (\boldsymbol{A} \odot \boldsymbol{B})^T = \boldsymbol{A}^T \odot \boldsymbol{B}^T \\ (\boldsymbol{A} \odot \boldsymbol{B})^H = \boldsymbol{A}^H \odot \boldsymbol{B}^H \\ (\boldsymbol{A} \odot \boldsymbol{B})^* = \boldsymbol{A}^* \odot \boldsymbol{B}^* \end{array}

  • A⊙0=0⊙A=0 \boldsymbol{A} \odot \boldsymbol{0} = \boldsymbol{0} \odot \boldsymbol{A} = \boldsymbol{0}

  • 若 cc 为常数，则 c(A⊙B)=(cA)⊙B c(\boldsymbol{A} \odot \boldsymbol{B}) = (c\boldsymbol{A}) \odot \boldsymbol{B}

  • 正定（半正定）矩阵 A,B\boldsymbol{A},\boldsymbol{B} 的 Hadamard 积 A⊙B\boldsymbol{A} \odot \boldsymbol{B} 也是正定（半正定）的。

2.2.2 其他性质

  • 若 A,C\boldsymbol{A},\boldsymbol{C} 为 m×mm \times m 矩阵，B,D\boldsymbol{B}, \boldsymbol{D} 为 n×nn \times n 矩阵，则

(A⊕B)⊙(C⊕D)=(A⊙C)⊕(B⊙D)\begin{array}{c} (\boldsymbol{A} \oplus \boldsymbol{B}) \odot (\boldsymbol{C} \oplus \boldsymbol{D}) = (\boldsymbol{A} \odot \boldsymbol{C}) \oplus (\boldsymbol{B} \odot \boldsymbol{D}) \end{array}

  • 若 A,B,C,D\boldsymbol{A}, \boldsymbol{B}, \boldsymbol{C}, \boldsymbol{D} 均为 m×nm \times n 矩阵，则

(A+B)⊙(C+D)=A⊙C+A⊙D+B⊙C+B⊙D\begin{array}{c} (\boldsymbol{A} + \boldsymbol{B}) \odot (\boldsymbol{C} + \boldsymbol{D}) = \boldsymbol{A} \odot \boldsymbol{C} + \boldsymbol{A} \odot \boldsymbol{D} + \boldsymbol{B} \odot \boldsymbol{C} + \boldsymbol{B} \odot \boldsymbol{D} \end{array}

  • 若 A,B,C\boldsymbol{A}, \boldsymbol{B}, \boldsymbol{C} 均为 m×nm \times n 矩阵，则

tr(AT(B⊙C))=tr((AT⊙BT)C)\begin{array}{c} \mathrm{tr}(\boldsymbol{A}^T(\boldsymbol{B} \odot \boldsymbol{C})) = tr((\boldsymbol{A}^T \odot \boldsymbol{B}^T) \boldsymbol{C} ) \end{array}

2.2.3 不等式性质

  • Oppenheim 不等式：A,B\boldsymbol{A}, \boldsymbol{B} 是 n×nn \times n 半正定矩阵，则

∣A⊙B∣≥a11⋯ann∣B∣\begin{array}{c} |\boldsymbol{A} \odot \boldsymbol{B}| \geq a_{11} \cdots a_{nn} |\boldsymbol{B}| \end{array}

  • ∣A⊙B∣≥∣AB∣|\boldsymbol{A} \odot \boldsymbol{B}| \geq |\boldsymbol{AB}|

  • 特征值不等式：A,B\boldsymbol{A}, \boldsymbol{B} 是 n×nn \times n 半正定矩阵，λ1,⋯ ,λn\lambda_1, \cdots, \lambda_n 是 A⊙B\boldsymbol{A} \odot \boldsymbol{B} 的特征值，λ1^,⋯ ,λn^\hat{\lambda_1}, \cdots, \hat{\lambda_n} 是矩阵乘积 AB\boldsymbol{AB} 的特征值，则

∏i=knλi≥∏i=knλi^,  k=1,⋯ ,n\begin{array}{c} \prod_{i=k}^n \lambda_i \geq \prod_{i=k}^n \hat{\lambda_i}, \ \ k = 1, \cdots, n \end{array}

  • 秩不等式：A,B\boldsymbol{A},\boldsymbol{B} 是 n×nn \times n 矩阵，则

rank(A⊙B)≤rank(A)rank(B)\begin{array}{c} \mathrm{rank}(\boldsymbol{A} \odot \boldsymbol{B}) \leq \mathrm{rank}(\boldsymbol{A}) \mathrm{rank}(\boldsymbol{B}) \end{array}

3. Kronecker 积（直积/张量积）

3.1 定义

3.1.1 Kronecker 积

两个矩阵的 Kronecker 积（也称为直积/张量积）分为右 Kronecker 积和左 Kronecker 积。

  • 右 Kronecker 积：m×nm \times n 矩阵 A=[a1,⋯ ,an] \boldsymbol{A} = [\boldsymbol{a}_1, \cdots, \boldsymbol{a}_n] 和 p×qp \times q 矩阵 B\boldsymbol{B} 的右 Kronecker 积记作 A⊗B \boldsymbol{A} \otimes \boldsymbol{B} ，它是一个 mp×nqmp \times nq 矩阵，定义为：

A⊗B=[a1B,⋯ ,anB]=[aijB]i=1,j=1m,n=[a11Ba12B⋯a1nBa21Ba22B⋯a2nB⋮⋮⋱⋮am1Bam2B⋯amnB]\begin{array}{c} \boldsymbol{A} \otimes \boldsymbol{B} = [\boldsymbol{a}_1 \boldsymbol{B}, \cdots, \boldsymbol{a}_n \boldsymbol{B}] = [a_{ij}\boldsymbol{B}]_{i=1,j=1}^{m,n} = \left[ \begin{matrix} a_{11} \boldsymbol{B} & a_{12} \boldsymbol{B} & \cdots & a_{1n} \boldsymbol{B} \\ a_{21} \boldsymbol{B} & a_{22} \boldsymbol{B} & \cdots & a_{2n} \boldsymbol{B} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} \boldsymbol{B} & a_{m2} \boldsymbol{B} & \cdots & a_{mn} \boldsymbol{B} \end{matrix} \right] \end{array}

  • 左 Kronecker 积：m×nm \times n 矩阵 A \boldsymbol{A} 和 p×qp \times q 矩阵 B==[b1,⋯ ,bq] \boldsymbol{B} = = [\boldsymbol{b}_1, \cdots, \boldsymbol{b}_q] 的左 Kronecker 积记作 [A⊗B]left [\boldsymbol{A} \otimes \boldsymbol{B}]_\mathrm{left} ，它是一个 mp×nqmp \times nq 矩阵，定义为：

[A⊗B]left=[Ab1,⋯ ,Abq]=[bijB]i=1,j=1p,q=[Ab11Ab12⋯Ab1qAb21Ab22⋯Ab2q⋮⋮⋱⋮Abp1Abp2⋯Abpq]\begin{array}{c} [\boldsymbol{A} \otimes \boldsymbol{B}]_\mathrm{left} = [\boldsymbol{A} \boldsymbol{b}_1, \cdots, \boldsymbol{A} \boldsymbol{b}_q] = [b_{ij}\boldsymbol{B}]_{i=1,j=1}^{p,q} = \left[ \begin{matrix} \boldsymbol{A} b_{11} & \boldsymbol{A} b_{12} & \cdots & \boldsymbol{A} b_{1q} \\ \boldsymbol{A} b_{21} & \boldsymbol{A} b_{22} & \cdots & \boldsymbol{A} b_{2q} \\ \vdots & \vdots & \ddots & \vdots \\ \boldsymbol{A} b_{p1} & \boldsymbol{A} b_{p2} & \cdots & \boldsymbol{A} b_{pq} \end{matrix} \right] \end{array}

可以看出，Kronecker 积是一映射：Rm×n×Rm×n↦Rmp×nq \mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \mapsto \mathbb{R}^{mp \times nq} ，且左右 Kronecker 积满足：

[A⊗B]left=B⊗A\begin{array}{c} [\boldsymbol{A} \otimes \boldsymbol{B}]_{\mathrm{left}} = \boldsymbol{B} \otimes \boldsymbol{A} \end{array}

一般常用右 Kronecker 积。

3.1.2 广义 Kronecker 积

给定 NN 个 m×rm \times r 矩阵 Ai,i=1,⋯ ,N\boldsymbol{A}_i, i=1,\cdots,N 组成矩阵组 {A}N\{A\}_N 。该矩阵组与 N×lN \times l 矩阵 BB 的 Kronecker 积称为广义 Kronecker 积，定义为：

{A}N⊗B=[A1⊗b1⋮AN⊗bN]\begin{array}{c} \{\boldsymbol{A}\}_N \otimes \boldsymbol{B} = \left[ \begin{matrix} \boldsymbol{A}_1 \otimes \boldsymbol{b}_1 \\ \vdots \\ \boldsymbol{A}_N \otimes \boldsymbol{b}_N \end{matrix} \right] \end{array}

其中，bi\boldsymbol{b}_i 是矩阵 B\boldsymbol{B} 的第 ii 个行向量。

3.2 性质

3.2.1 基本性质

  • 不满足交换律：对矩阵 Am×n\boldsymbol{A}_{m \times n} 和 Bp×q\boldsymbol{B}_{p \times q}，一般有 A⊗B≠B⊗A \boldsymbol{A} \otimes \boldsymbol{B} \ne \boldsymbol{B} \otimes \boldsymbol{A}

  • A⊗0=0⊗A=0 \boldsymbol{A} \otimes \boldsymbol{0} = \boldsymbol{0} \otimes \boldsymbol{A} = \boldsymbol{0}

  • 若 a,ba,b 为常数，则 aA⊗bB=ab(A⊗B) a\boldsymbol{A} \otimes b\boldsymbol{B} = ab(\boldsymbol{A} \otimes \boldsymbol{B})

  • Im⊗In=Imn \boldsymbol{I}_m \otimes \boldsymbol{I}_n = \boldsymbol{I}_{mn}

  • 对于矩阵 Am×n,Bn×k,Cl×p,Dp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{n \times k}, \boldsymbol{C}_{l \times p}, \boldsymbol{D}_{p \times q}，有

(AB)⊗(CD)=(A⊗C)(B⊗D)\begin{array}{c} (\boldsymbol{AB}) \otimes (\boldsymbol{CD}) = (\boldsymbol{A} \otimes \boldsymbol{C})(\boldsymbol{B} \otimes \boldsymbol{D}) \end{array}

  • 对于矩阵 Am×n,Bp×q,Cp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{p \times q}, \boldsymbol{C}_{p \times q}，有

A⊗(B±C)=A⊗B±A⊗C(B±C)⊗A=B⊗A±C⊗A\begin{array}{c} \boldsymbol{A} \otimes (\boldsymbol{B} \pm \boldsymbol{C}) = \boldsymbol{A} \otimes \boldsymbol{B} \pm \boldsymbol{A} \otimes \boldsymbol{C} \\ (\boldsymbol{B} \pm \boldsymbol{C}) \otimes \boldsymbol{A} = \boldsymbol{B} \otimes \boldsymbol{A} \pm \boldsymbol{C} \otimes \boldsymbol{A} \end{array}

  • 对于矩阵 Am×n,Bm×n,Cp×q,Dp×q\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{m \times n}, \boldsymbol{C}_{p \times q}, \boldsymbol{D}_{p \times q}，有

(A+B)⊗(C+D)=A⊗C+A⊗D+B⊗C+B⊗D\begin{array}{c} (\boldsymbol{A} + \boldsymbol{B}) \otimes (\boldsymbol{C} + \boldsymbol{D}) = \boldsymbol{A} \otimes \boldsymbol{C} + \boldsymbol{A} \otimes \boldsymbol{D} + \boldsymbol{B} \otimes \boldsymbol{C} + \boldsymbol{B} \otimes \boldsymbol{D} \end{array}

  • 对于矩阵 Am×n,Bp×q,Ck×l\boldsymbol{A}_{m \times n}, \boldsymbol{B}_{p \times q}, \boldsymbol{C}_{k \times l}，有

(A⊗B)⊗C=A⊗(B⊗C)=A⊗B⊗C\begin{array}{c} (\boldsymbol{A} \otimes \boldsymbol{B}) \otimes \boldsymbol{C} = \boldsymbol{A} \otimes (\boldsymbol{B} \otimes \boldsymbol{C}) = \boldsymbol{A} \otimes \boldsymbol{B} \otimes \boldsymbol{C} \end{array}

  • Kronecker 积的转置与复共轭转置

(A⊗B)T=AT⊗BT(A⊗B)H=AH⊗BH\begin{array}{c} (\boldsymbol{A} \otimes \boldsymbol{B})^T = \boldsymbol{A}^T \otimes \boldsymbol{B}^T \\ (\boldsymbol{A} \otimes \boldsymbol{B})^H = \boldsymbol{A}^H \otimes \boldsymbol{B}^H \end{array}

3.2.2 其他性质

  • Kronecker 积的秩

rank(A⊗B)=rank(A)rank(B)\begin{array}{c} \mathrm{rank}(\boldsymbol{A} \otimes \boldsymbol{B}) = \mathrm{rank}(\boldsymbol{A}) \mathrm{rank}(\boldsymbol{B}) \end{array}

  • Kronecker 积的行列式

det(An×n⊗Bm×m)=(detA)m(detB)n\begin{array}{c} \mathrm{det}(\boldsymbol{A}_{n \times n} \otimes \boldsymbol{B}_{m \times m}) = (\mathrm{det} \boldsymbol{A})^m (\mathrm{det} \boldsymbol{B})^n \end{array}

  • Kronecker 积的迹

tr(A⊗B)=tr(A)tr(B)\begin{array}{c} \mathrm{tr}(\boldsymbol{A} \otimes \boldsymbol{B}) = \mathrm{tr}(\boldsymbol{A}) \mathrm{tr}(\boldsymbol{B}) \end{array}

4. Khatri-Rao 积

4.1 定义

两个具有相同列数的矩阵 G∈Rp×n \boldsymbol{G} \in \mathbb{R}^{p \times n} 和 F∈Rq×n \boldsymbol{F} \in \mathbb{R}^{q \times n} 的 Khatri-Rao 积记为 F∗G \boldsymbol{F} * \boldsymbol{G} ，它是一个 pq×npq \times n 矩阵，定义为：

F∗G=[f1⊗g1,⋯ ,fn⊗gn]∈Rpq×n\begin{array}{c} \boldsymbol{F} * \boldsymbol{G} = [\boldsymbol{f}_1 \otimes \boldsymbol{g}_1 , \cdots, \boldsymbol{f}_n \otimes \boldsymbol{g}_n] \in \mathbb{R}^{pq \times n} \end{array}

它由两个矩阵对应列向量的 Kronecker 积排列而成（fi\boldsymbol{f}_i 和 gi\boldsymbol{g}_i 分别表示 F\boldsymbol{F} 和 G\boldsymbol{G} 的列向量），因此 Khatri-Rao 积又称为对应列 Kronecker 积。

4.2 性质

4.2.1 基本性质

  • 分配律：(A+B)∗D=A∗D+B∗D (\boldsymbol{A} + \boldsymbol{B}) * \boldsymbol{D} = \boldsymbol{A} * \boldsymbol{D} + \boldsymbol{B} * \boldsymbol{D}

  • 结合律：A∗B∗C=(A∗B)∗C=A∗(B∗C) \boldsymbol{A} * \boldsymbol{B} * \boldsymbol{C} = (\boldsymbol{A} * \boldsymbol{B}) * \boldsymbol{C} = \boldsymbol{A} * (\boldsymbol{B} * \boldsymbol{C})

  • 交换律：A∗B=Knn(B∗A) \boldsymbol{A} * \boldsymbol{B} = \boldsymbol{K}_{nn} (\boldsymbol{B} * \boldsymbol{A})

4.2.2 Khatri-Rao 积与 Hadamard 积的关系

  • (A∗B)⊙(C∗D)=(A⊙C)∗(B⊙D) (\boldsymbol{A} * \boldsymbol{B}) \odot (\boldsymbol{C} * \boldsymbol{D}) = (\boldsymbol{A} \odot \boldsymbol{C}) * (\boldsymbol{B} \odot \boldsymbol{D})

  • (A∗B)T(A∗B)=(ATA)⊙(BTB) (\boldsymbol{A} * \boldsymbol{B})^T(\boldsymbol{A} * \boldsymbol{B}) = (\boldsymbol{A}^T \boldsymbol{A}) \odot (\boldsymbol{B}^T \boldsymbol{B})

  • (A∗B)†=[(ATA)⊙(BTB)]†(A∗B)T (\boldsymbol{A} * \boldsymbol{B})^{\dagger} = [(\boldsymbol{A}^T \boldsymbol{A}) \odot (\boldsymbol{B}^T \boldsymbol{B})]^{\dagger} (\boldsymbol{A} * \boldsymbol{B})^T

4.2.3 Khatri-Rao 积与 Kronecker 积的关系

  • (A⊗B)(F∗G)=AF∗BG (\boldsymbol{A} \otimes \boldsymbol{B})(\boldsymbol{F} * \boldsymbol{G}) = \boldsymbol{AF} * \boldsymbol{BG}

5. Face-Splitting 积

5.1 定义

两个具有相同行数的矩阵 G∈Rn×p \boldsymbol{G} \in \mathbb{R}^{n \times p} 和 F∈Rn×q \boldsymbol{F} \in \mathbb{R}^{n \times q} 的 Face-Splitting 积记为 F∗G \boldsymbol{F} * \boldsymbol{G} ，它是一个 n×pqn \times pq 矩阵，定义为：

F∙G=[f1⊗g1⋮fn⊗gn]∈Rn×pq\begin{array}{c} \boldsymbol{F} \bullet \boldsymbol{G} = \left[ \begin{matrix} \boldsymbol{f}_1 \otimes \boldsymbol{g}_1 \\ \vdots \\ \boldsymbol{f}_n \otimes \boldsymbol{g}_n \end{matrix} \right] \in \mathbb{R}^{n \times pq} \end{array}

它由两个矩阵对应行向量的 Kronecker 积排列而成（fi\boldsymbol{f}_i 和 gi\boldsymbol{g}_i 分别表示 F\boldsymbol{F} 和 G\boldsymbol{G} 的行向量），因此 Face-Splitting 积又称为对应行 Kronecker 积。

5.2 性质

  • (A∙B)T=AT∗BT(\boldsymbol{A} \bullet \boldsymbol{B})^T = \boldsymbol{A}^T * \boldsymbol{B}^T

其余性质和 Khatri-Rao 类似。

附录

  • 《矩阵分析与应用（第二版）》by 张贤达