线性代数

地址

基本知识

import numpy as np
import torch

向量

$$\vec x = (x_{1}, x_{2},...x_{n})^{T}$$

x = np.array([1, 2, 3])
y = torch.Tensor([1, 2, 3])
print(x, y)

[1 2 3] tensor([1., 2., 3.])

矩阵

x = np.array([[1, 2, 3],[4, 5, 6]])
y = torch.Tensor([[1, 2, 3], [4, 5, 6]])
print(x)
print(y)

[[1 2 3]
 [4 5 6]]
tensor([[1., 2., 3.],
        [4., 5., 6.]])

矩阵的F范数

设矩阵 $A=(a_{i,j})_{m,n}$,则其F范数为$||A||_{F}=\sqrt{\sum_{i,j}a^2_{i,j}}$

l2_x = np.linalg.norm(x)
l2_y = torch.norm(y)
print("norm of x is {}".format(l2_x))
print("norm of y is {}".format(l2_y))

norm of x is 9.539392014169456
norm of y is 9.539392471313477

矩阵的迹

矩阵$A=(a_{i,j})_{m,n}$，则$A$的迹为:

$$tr(A)=\sum_{i}{a_{i,i}}$$

x_trace = x.trace()
y_trace = torch.trace(y)
print("trace of x is {}".format(x_trace))
print("trace of y is {}".format(y_trace))

trace of x is 6
trace of y is 6.0

矩阵运算

逐元素积（哈达玛积）

$$\vec u * \vec v = [u_{i,j}v_{i,j},...]$$

# 向量
a = np.arange(0,9)
b = a[::-1]
c = a * b
print("dot of vector a,b is {}".format(c))

dot of vector a,b is [ 0  7 12 15 16 15 12  7  0]

# 矩阵
a = np.arange(1, 5).reshape(2,2)
b = np.arange(5, 9).reshape(2,2)
c = a * b
print("dot of matrix a,b is {}".format(c))

dot of matrix a,b is [[ 5 12]
 [21 32]]

# pytorch
a = torch.arange(1,5).reshape(2,2)
b = torch.arange(5,9).view(2, 2)
print(a * b)
print(a.mul(b))

tensor([[ 5, 12],
        [21, 32]])
tensor([[ 5, 12],
        [21, 32]])

点积（矩阵乘法）

$$(AB)_{ij} = \sum_{k=1}^{p}a_{ik}b_{kj}$$

a = np.arange(1, 5).reshape(2,2)
b = np.arange(5, 9).reshape(2,2)
np.dot(a,b)

array([[19, 22],
       [43, 50]])

a = torch.arange(1,5).reshape(2,2)
b = torch.arange(5,9).view(2, 2)
a.mm(b)

tensor([[19, 22],
        [43, 50]])

克罗内克积

$$A \bigotimes B = \begin{bmatrix} a_{1,1}B &... &a_{1,n}B \\ ... & ... &...\\a_{m,1}B & ... &a_{m,n}B \end{bmatrix}$$

a = np.arange(1, 5).reshape(2,2)
b = np.arange(5, 9).reshape(2,2)
np.kron(a,b)

array([[ 5,  6, 10, 12],
       [ 7,  8, 14, 16],
       [15, 18, 20, 24],
       [21, 24, 28, 32]])

特殊函数

sigmoid函数

$$\sigma (x) = \frac{1}{1+exp(-x)}$$

• 导数 $$\sigma^{'}(x) = \sigma(x) \cdot (1-\sigma(x))$$
• 值域在0和1之间，用于二分类
• 函数具有非常好的对称性
• 在趋于无穷时，函数值变化小，容易产生梯度消失

import matplotlib.pyplot as plt
%matplotlib inline
def sigmoid(x):
    return 1. / (1.+np.exp(-x))
x = np.arange(-5, 5, 0.2)
y = sigmoid(x)
plt.plot(x,y)
plt.show()

softplus函数

$$\zeta(x) = log(1+exp(x)$$

• 平滑版的ReLu函数
• 导数: $$\frac{d}{dx}\zeta(x) = \sigma(x)$$

def softplus(x):
    return np.log(1.+np.exp(x))
x = np.arange(-10, 10, 0.2)
y = softplus(x)
plt.plot(x,y)
plt.show()

伽玛函数

$$\Gamma(x) = \int_{0}^{\infty}t^{x-1}e^{-t} \mathrm{d}x$$

• 对于整数$n$：$\Gamma(n)=(n-1)!$
• $\Gamma(x+1)=x\Gamma(x)$
• 对于$x \in (0,1)$: $$\Gamma(1-x)\Gamma(x) = \frac{\pi}{sin\pi x}$$ 可以推出重要公式: $\Gamma(\frac{1}{2}) = \sqrt{\pi}$

from scipy.special import gamma
x = np.linspace(0, 5, 100)
y = gamma(x)
plt.plot(x,y)
plt.show()

贝塔函数

$$B(m,n) = \int_{0}{1} x^{m-1}(1-x)^{n-1} \mathrm{d}x$$

• 与伽玛函数关系: $$B(m,n) = \frac{\Gamma(m)\Gamma(n)}{\Gamma(m+n)}$$ $$B(m,1-m) = \Gamma(m)\Gamma(1-m)$$
• 对称性: $B(m,n) = B(n,m)$