-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsearch.xml
200 lines (99 loc) · 178 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>深度学习Pytorch/Pycharm等 踩坑记录</title>
<link href="/2024/10/20/Pytorch%20%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/"/>
<url>/2024/10/20/Pytorch%20%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/</url>
<content type="html"><![CDATA[<h1>深度学习代码/软件 踩坑记录</h1><h2 id="pycharm">pycharm</h2><h3 id="1-远程调试时，调试文件使用的是C盘JetBrains文件夹下的映射文件，而不是实际项目文件">1. 远程调试时，调试文件使用的是C盘JetBrains文件夹下的映射文件，而不是实际项目文件</h3><p>在调试时，会跳到断点位置，但是是C盘映射部分的对应文件，而不是pycharm中实际的打断点的文件</p><p>解决方案：在运行/调试设置中，增加路径映射：</p><img src="/2024/10/20/Pytorch%20%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/image-20241014144606967.png" class="" title="image-20241014144606967"><h3 id="2-远程调试设置文件路径出现FileNotFoundError-Errno-2-No-such-file-or-directory，找不到文件">2. 远程调试设置文件路径出现FileNotFoundError: [Errno 2] No such file or directory，找不到文件</h3><p>在经过1的路径映射设置和远程服务器正确配置后如果仍出现该问题，一般是工作目录设置成了子目录或者一些奇怪的地方，把工作目录改成项目目录即可，如下</p><img src="/2024/10/20/Pytorch%20%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/image-20241014144727017.png" class="" title="image-20241014144727017"><ul><li>这里原来是…/detrex/tools，默认设置成子目录了，导致相对路径错误</li></ul><h3 id="3-远程调用jupyternotebook报错">3. 远程调用jupyternotebook报错</h3><p><strong>报错信息：</strong></p><ul><li>如：Jupyter 服务器进程退出，代码为 1 [I 2024-10-21 02:50:57.667 ServerApp] jupyter_lsp | extension was successfully</li><li>主要最后一行含有：<code>Running as root is not recommended. Use --allow-root to bypass.</code></li></ul><p>这个报错的意思是你作为<code>root</code>访问远程服务器，但是 jupyter 不允许running as root</p><p><strong>解决方案：</strong></p><p>修改jupyter配置文件</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">conda activate your_environment_name</span><br><span class="line">vim ~/.jupyter/jupyter_notebook_config.py</span><br></pre></td></tr></table></figure><p>输入上述指令后，如果发现 config 文件不存在（即vim打开为空），则需用如下指令重新创建一个：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">jupyter notebook --generate-config</span><br></pre></td></tr></table></figure><p>如果 config 文件已存在，则会进入到<a href="https://so.csdn.net/so/search?q=%E9%85%8D%E7%BD%AE%E6%96%87%E4%BB%B6&spm=1001.2101.3001.7020">配置文件</a>界面的中，我们需要使用vim指令对其进行编辑：<br>可以看到，<code>c.NotebookApp.allow_root/c.ServerApp.allow_root</code>默认是<code>False</code>，改成<code>True</code>即可解决问题。</p><img src="/2024/10/20/Pytorch%20%E8%B8%A9%E5%9D%91%E8%AE%B0%E5%BD%95/image-20241021111742034.png" class="" title="image-20241021111742034"><h2 id="pytorch">pytorch</h2><h3 id="1-4090最低可以用cuda11-0">1. 4090最低可以用cuda11.0</h3><h3 id="2-pip-install-e-报错：OSError">2. pip install -e ./ 报错：OSError</h3><p>完整报错如下：</p><p></p><p>主要是两个错误：</p><ol><li>OSError: /root/miniconda3/envs/cdfsod/lib/python3.9/site-packages/torch/lib/…/…/nvidia/cublas/lib/libcublas.so.11: symbol cublasLtGetStatusString version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference</li><li>OSError: /root/miniconda3/envs/cdfsod/lib/python3.9/site-packages/nvidia/cublas/lib/libcublas.so.11: symbol cublasLtGetStatusString version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference</li></ol><p>这俩本质一样</p><p>解决方案：增加一个临时变量</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">export</span> LD_LIBRARY_PATH=/root/miniconda3/envs/cdfsod/lib/python3.9/site-packages/nvidia/cublas/lib/:<span class="variable">$LD_LIBRARY_PATH</span></span><br></pre></td></tr></table></figure><ul><li>这里的路径参考上面报错的路径，对应一下即可</li></ul><h3 id="3-RuntimeError-weight-tensor-should-be-defined-either-for-all-or-no-classes">3.RuntimeError: weight tensor should be defined either for all or no classes</h3><ul><li>主要是weight的维度和实际不符，比如10个类，你的分类头是81维</li><li>修改维度或者取消weight的初始化即可</li></ul>]]></content>
<categories>
<category> 深度学习服务器 </category>
</categories>
<tags>
<tag> Pytorch </tag>
</tags>
</entry>
<entry>
<title>深度学习服务器常用Linux命令小结</title>
<link href="/2024/10/20/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%9C%8D%E5%8A%A1%E5%99%A8%E5%B8%B8%E7%94%A8Linux%E5%91%BD%E4%BB%A4%E5%B0%8F%E7%BB%93/"/>
<url>/2024/10/20/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%9C%8D%E5%8A%A1%E5%99%A8%E5%B8%B8%E7%94%A8Linux%E5%91%BD%E4%BB%A4%E5%B0%8F%E7%BB%93/</url>
<content type="html"><![CDATA[<h1>深度学习服务器常用Linux命令小结</h1><h2 id="说明">说明</h2><p>本博客记录了在利用Xshell，Xftp等访问服务器时用到的一些常用命令，持续更新中。</p><h2 id="更改文件夹访问权限">更改文件夹访问权限</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">chmod</span> 777 /path</span><br></pre></td></tr></table></figure><h2 id="pip配置清华源">pip配置清华源</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pip config <span class="built_in">set</span> global.index-url https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple</span><br></pre></td></tr></table></figure><h2 id="删除conda环境">删除conda环境</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">conda remove --name your_env_name --all</span><br></pre></td></tr></table></figure><h2 id="解压tar-gz文件">解压tar.gz文件</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">tar -xzvf file.tar.gz</span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> 深度学习服务器 </category>
</categories>
<tags>
<tag> DL服务器命令 </tag>
</tags>
</entry>
<entry>
<title>PyTorch Tensor 操作全面教程</title>
<link href="/2024/10/14/PyTorch%20Tensor%20%E5%9F%BA%E7%A1%80%E6%95%99%E7%A8%8B/"/>
<url>/2024/10/14/PyTorch%20Tensor%20%E5%9F%BA%E7%A1%80%E6%95%99%E7%A8%8B/</url>
<content type="html"><![CDATA[<h1>PyTorch Tensor 操作全面教程</h1><p>这篇博客将讨论PyTorch 中的tensor相关操作，分为四个主要部分：</p><ol><li>tensor的初始化</li><li>tensor的数学操作和比较</li><li>tensor的索引操作</li><li>tensor的重塑</li></ol><h2 id="1-初始化张量">1. 初始化张量</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">device = <span class="string">"cuda"</span> <span class="keyword">if</span> torch.cuda.is_available() <span class="keyword">else</span> <span class="string">"cpu"</span> <span class="comment"># 如果有GPU可用，则使用GPU</span></span><br></pre></td></tr></table></figure><h3 id="1-1-张量的初始化">1.1 张量的初始化</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">my_tensor = torch.tensor(</span><br><span class="line"> [[<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>], [<span class="number">4</span>, <span class="number">5</span>, <span class="number">6</span>]], dtype=torch.float32, device=device, requires_grad=<span class="literal">True</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li><code>torch.tensor()</code>：用于创建张量。</li><li><code>[[1, 2, 3], [4, 5, 6]]</code>：定义张量的数据，形状为 2 行 3 列。</li><li><code>dtype=torch.float32</code>：指定张量的数据类型为 32 位浮点数。</li><li><code>device=device</code>：将张量分配到之前设置的设备上（GPU 或 CPU）。</li><li><code>requires_grad=True</code>：表示是否需要对该张量进行求导，这在反向传播中非常重要。</li></ul><h3 id="1-2-张量属性">1.2 张量属性</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">print</span>(<span class="string">f"Information about tensor: <span class="subst">{my_tensor}</span>"</span>) <span class="comment"># 打印张量的数据信息，包括设备和梯度信息</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">"Type of Tensor {my_tensor.dtype}"</span>) <span class="comment"># 打印张量的数据类型</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Device Tensor is on <span class="subst">{my_tensor.device}</span>"</span>) <span class="comment"># 打印张量所在的设备</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Shape of tensor <span class="subst">{my_tensor.shape}</span>"</span>) <span class="comment"># 打印张量的形状</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Requires gradient: <span class="subst">{my_tensor.requires_grad}</span>"</span>) <span class="comment"># 打印是否需要计算梯度</span></span><br></pre></td></tr></table></figure><ul><li><code>my_tensor.dtype</code>：获取张量的数据类型。</li><li><code>my_tensor.device</code>：获取张量所在的设备（CPU 或 CUDA）。</li><li><code>my_tensor.shape</code>：获取张量的形状。</li><li><code>my_tensor.requires_grad</code>：获取张量是否需要计算梯度。</li></ul><h3 id="1-3-常见的初始化方法">1.3 常见的初始化方法</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">x = torch.empty(size=(<span class="number">3</span>, <span class="number">3</span>)) <span class="comment"># 生成一个形状为 3x3 的张量，其中数据未初始化</span></span><br><span class="line">x = torch.zeros((<span class="number">3</span>, <span class="number">3</span>)) <span class="comment"># 生成一个形状为 3x3 的全零张量</span></span><br><span class="line">x = torch.rand((<span class="number">3</span>, <span class="number">3</span>)) <span class="comment"># 生成一个形状为 3x3 的张量，元素为 [0, 1) 区间的均匀分布随机数</span></span><br><span class="line">x = torch.ones((<span class="number">3</span>, <span class="number">3</span>)) <span class="comment"># 生成一个形状为 3x3 的全为 1 的张量</span></span><br><span class="line">x = torch.eye(<span class="number">5</span>, <span class="number">5</span>) <span class="comment"># 生成一个 5x5 的单位矩阵</span></span><br><span class="line">x = torch.arange(start=<span class="number">0</span>, end=<span class="number">5</span>, step=<span class="number">1</span>) <span class="comment"># 生成从 0 到 4 的张量，步长为 1</span></span><br><span class="line">x = torch.linspace(start=<span class="number">0.1</span>, end=<span class="number">1</span>, steps=<span class="number">10</span>) <span class="comment"># 在 [0.1, 1] 区间内生成 10 个等间隔的数</span></span><br><span class="line">x = torch.empty(size=(<span class="number">1</span>, <span class="number">5</span>)).normal_(mean=<span class="number">0</span>, std=<span class="number">1</span>) <span class="comment"># 生成一个均值为 0，标准差为 1 的正态分布张量</span></span><br><span class="line">x = torch.empty(size=(<span class="number">1</span>, <span class="number">5</span>)).uniform_(<span class="number">0</span>, <span class="number">1</span>) <span class="comment"># 生成一个均匀分布在 [0, 1) 的张量</span></span><br><span class="line">x = torch.diag(torch.ones(<span class="number">3</span>)) <span class="comment"># 生成一个 3x3 的对角矩阵</span></span><br></pre></td></tr></table></figure><ul><li><strong><code>torch.empty(size)</code></strong>：创建未初始化的 Tensor，内存中的随机值会填充。</li><li><strong><code>torch.zeros(size)</code></strong>：创建全 0 Tensor。</li><li><strong><code>torch.ones(size)</code></strong>：创建全 1 Tensor。</li><li><strong><code>torch.eye(n)</code></strong>：创建<strong>单位矩阵</strong>，对角线为 1，其他位置为 0。</li><li><strong><code>torch.arange(start, end, step)</code></strong>：生成从 <code>start</code> 到 <code>end</code>（不含 <code>end</code>）的序列，步长为 <code>step</code>。</li><li><strong><code>torch.linspace(start, end, steps)</code></strong>：在 <code>start</code> 和 <code>end</code> 之间生成 <code>steps</code> 个均匀间隔的数。</li><li><strong><code>torch.randn(size)</code></strong>：生成符合标准正态分布的随机数。</li><li><strong><code>torch.diag(tensor)</code></strong>：生成对角矩阵。</li></ul><h3 id="1-4-数据类型转换">1.4 数据类型转换</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tensor = torch.arange(<span class="number">4</span>) <span class="comment"># [0, 1, 2, 3]，默认类型为 int64</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted Boolean: <span class="subst">{tensor.<span class="built_in">bool</span>()}</span>"</span>) <span class="comment"># 转换为布尔类型：非零为 True</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted int16 <span class="subst">{tensor.short()}</span>"</span>) <span class="comment"># 转换为 int16 类型</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted int64 <span class="subst">{tensor.long()}</span>"</span>) <span class="comment"># 转换为 int64 类型（非常常用）</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted float16 <span class="subst">{tensor.half()}</span>"</span>) <span class="comment"># 转换为 float16 类型</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted float32 <span class="subst">{tensor.<span class="built_in">float</span>()}</span>"</span>) <span class="comment"># 转换为 float32 类型（非常常用）</span></span><br><span class="line"><span class="built_in">print</span>(<span class="string">f"Converted float64 <span class="subst">{tensor.double()}</span>"</span>) <span class="comment"># 转换为 float64 类型</span></span><br></pre></td></tr></table></figure><ul><li><code>tensor.bool()</code>：将张量的元素转换为布尔类型。</li><li><code>tensor.short()</code>：将张量的元素转换为 16 位整数。</li><li><code>tensor.long()</code>：将张量的元素转换为 64 位整数，这是非常常用的类型。</li><li><code>tensor.half()</code>：将张量的元素转换为 16 位浮点数。</li><li><code>tensor.float()</code>：将张量的元素转换为 32 位浮点数，这是深度学习中非常常用的类型。</li><li><code>tensor.double()</code>：将张量的元素转换为 64 位浮点数。</li></ul><h3 id="1-5-张量与-NumPy-数组的转换">1.5 张量与 NumPy 数组的转换</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">np_array = np.zeros((<span class="number">5</span>, <span class="number">5</span>))</span><br><span class="line">tensor = torch.from_numpy(np_array)</span><br><span class="line">np_array_again = tensor.numpy() <span class="comment"># 将张量转换回 NumPy 数组</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.from_numpy(np_array)</code>：将 NumPy 数组转换为 PyTorch 张量。</li><li><code>tensor.numpy()</code>：将张量转换为 NumPy 数组。这种转换是共享内存的，因此更改其中一个也会更改另一个。</li></ul><h2 id="2-张量的数学与比较操作">2. 张量的数学与比较操作</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">x = torch.tensor([<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>])</span><br><span class="line">y = torch.tensor([<span class="number">9</span>, <span class="number">8</span>, <span class="number">7</span>])</span><br><span class="line"></span><br><span class="line"><span class="comment"># 加法</span></span><br><span class="line">z1 = torch.empty(<span class="number">3</span>)</span><br><span class="line">torch.add(x, y, out=z1) <span class="comment"># 通过 `out` 参数指定输出张量</span></span><br><span class="line">z2 = torch.add(x, y) <span class="comment"># 直接相加</span></span><br><span class="line">z = x + y <span class="comment"># 最简洁的加法方式</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 减法</span></span><br><span class="line">z = x - y</span><br><span class="line"></span><br><span class="line"><span class="comment"># 除法</span></span><br><span class="line">z = torch.true_divide(x, y) <span class="comment"># 元素间逐一相除</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.add(x, y, out=z1)</code>：通过 <code>out</code> 参数指定将结果存储在 <code>z1</code> 中。</li><li><code>z = x + y</code>：最简洁的加法操作方式。</li><li><code>torch.true_divide(x, y)</code>：<strong>元素级</strong>逐一相除。</li></ul><h3 id="2-1-原地操作">2.1 原地操作</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">t = torch.zeros(<span class="number">3</span>)</span><br><span class="line">t.add_(x) <span class="comment"># 原地相加，修改 `t` 本身</span></span><br><span class="line">t += x <span class="comment"># 同样是原地操作</span></span><br></pre></td></tr></table></figure><ul><li><code>t.add_(x)</code> 和 <code>t += x</code>：原地操作会直接修改变量本身，这些操作在变量名后面带有 <code>_</code> 后缀。</li></ul><h3 id="2-2-幂运算与比较">2.2 幂运算与比较</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">z = x.<span class="built_in">pow</span>(<span class="number">2</span>) <span class="comment"># 计算 x 的每个元素的平方</span></span><br><span class="line">z = x**<span class="number">2</span> <span class="comment"># 等价于 `x.pow(2)`</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 比较操作</span></span><br><span class="line">z = x > <span class="number">0</span> <span class="comment"># 返回 [True, True, True]</span></span><br><span class="line">z = x < <span class="number">0</span> <span class="comment"># 返回 [False, False, False]</span></span><br></pre></td></tr></table></figure><ul><li><code>x.pow(2)</code> 和 <code>x**2</code>：计算张量每个元素的平方，结果相同。</li><li><code>x > 0</code> 和 <code>x < 0</code>：比较操作会逐元素返回布尔值。</li></ul><h3 id="2-3-矩阵乘法">2.3 矩阵乘法</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">x1 = torch.rand((<span class="number">2</span>, <span class="number">5</span>))</span><br><span class="line">x2 = torch.rand((<span class="number">5</span>, <span class="number">3</span>))</span><br><span class="line">x3 = torch.mm(x1, x2) <span class="comment"># 矩阵乘法，输出形状为 2x3</span></span><br><span class="line">x3 = x1.mm(x2) <span class="comment"># 与上一行等价</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.mm(x1, x2)</code>：执行矩阵乘法，<code>x1</code> 的列数必须等于 <code>x2</code> 的行数。</li><li><code>x1.mm(x2)</code>：另一种调用矩阵乘法的方法，结果与 <code>torch.mm(x1, x2)</code> 相同。</li></ul><h3 id="2-4-矩阵的幂运算">2.4 矩阵的幂运算</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">matrix_exp = torch.rand(<span class="number">5</span>, <span class="number">5</span>)</span><br><span class="line"><span class="built_in">print</span>(matrix_exp.matrix_power(<span class="number">3</span>)) <span class="comment"># 计算矩阵的三次幂，相当于 matrix_exp (mm) matrix_exp (mm) matrix_exp</span></span><br></pre></td></tr></table></figure><ul><li><code>matrix_exp.matrix_power(3)</code>：计算矩阵的三次幂，即将矩阵自身与自身相乘三次。</li></ul><h3 id="2-5-元素级乘法与点积">2.5 元素级乘法与点积</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">z = x * y <span class="comment"># 元素级乘法，z = [9, 16, 21] = [1*9, 2*8, 3*7]</span></span><br><span class="line"></span><br><span class="line">z = torch.dot(x, y) <span class="comment"># 计算向量的点积，结果为 1*9 + 2*8 + 3*7</span></span><br></pre></td></tr></table></figure><ul><li><code>x * y</code>：元素级乘法，每个元素对应相乘。</li><li><code>torch.dot(x, y)</code>：计算两个向量的点积。</li></ul><h3 id="2-6-批量矩阵乘法">2.6 批量矩阵乘法</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">batch = <span class="number">32</span></span><br><span class="line">n = <span class="number">10</span></span><br><span class="line">m = <span class="number">20</span></span><br><span class="line">p = <span class="number">30</span></span><br><span class="line">tensor1 = torch.rand((batch, n, m))</span><br><span class="line">tensor2 = torch.rand((batch, m, p))</span><br><span class="line">out_bmm = torch.bmm(tensor1, tensor2) <span class="comment"># 输出形状为 (batch, n, p)</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.bmm(tensor1, tensor2)</code>：执行批量矩阵乘法，<code>tensor1</code> 和 <code>tensor2</code> 的第一个维度是批量大小，后面的维度表示矩阵的形状。</li></ul><h3 id="2-7-广播机制示例">2.7 广播机制示例</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">x1 = torch.rand((<span class="number">5</span>, <span class="number">5</span>))</span><br><span class="line">x2 = torch.ones((<span class="number">1</span>, <span class="number">5</span>))</span><br><span class="line">z = x1 - x2 <span class="comment"># 形状为 5x5，x2 会广播以匹配 x1 的形状</span></span><br><span class="line">z = x1**x2 <span class="comment"># 形状为 5x5，元素级指数运算，x2 会广播</span></span><br></pre></td></tr></table></figure><ul><li><code>x1 - x2</code>：<code>x2</code> 的形状是 <code>(1, 5)</code>，会自动广播为 <code>(5, 5)</code>，然后逐元素相减。</li><li><code>x1**x2</code>：<code>x2</code> 同样会广播为 <code>(5, 5)</code>，然后<strong>逐元素</strong>进行指数运算。</li></ul><h3 id="2-8-其他有用的张量操作">2.8 其他有用的张量操作</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">sum_x = torch.<span class="built_in">sum</span>(x, dim=<span class="number">0</span>) <span class="comment"># 在维度 0 上求和，结果为 6</span></span><br><span class="line">values, indices = torch.<span class="built_in">max</span>(x, dim=<span class="number">0</span>) <span class="comment"># 获取最大值及其索引</span></span><br><span class="line">values, indices = torch.<span class="built_in">min</span>(x, dim=<span class="number">0</span>) <span class="comment"># 获取最小值及其索引</span></span><br><span class="line">abs_x = torch.<span class="built_in">abs</span>(x) <span class="comment"># 对每个元素取绝对值</span></span><br><span class="line">z = torch.argmax(x, dim=<span class="number">0</span>) <span class="comment"># 获取最大值的索引</span></span><br><span class="line">z = torch.argmin(x, dim=<span class="number">0</span>) <span class="comment"># 获取最小值的索引</span></span><br><span class="line">mean_x = torch.mean(x.<span class="built_in">float</span>(), dim=<span class="number">0</span>) <span class="comment"># 求平均值，x 需要转换为浮点数</span></span><br><span class="line">z = torch.eq(x, y) <span class="comment"># 逐元素比较是否相等，返回布尔值张量</span></span><br><span class="line">sorted_y, indices = torch.sort(y, dim=<span class="number">0</span>, descending=<span class="literal">False</span>) <span class="comment"># 对张量排序</span></span><br><span class="line"></span><br><span class="line">z = torch.clamp(x, <span class="built_in">min</span>=<span class="number">0</span>) <span class="comment"># 将所有小于 0 的值设为 0，大于 0 的值保持不变</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.sum(x, dim=0)</code>：在指定维度上求和。</li><li><code>torch.max(x, dim=0)</code> 和 <code>torch.min(x, dim=0)</code>：获取最大值或最小值及其索引。</li><li><code>torch.abs(x)</code>：计算每个元素的绝对值。</li><li><code>torch.argmax(x, dim=0)</code> 和 <code>torch.argmin(x, dim=0)</code>：获取最大值或最小值的索引。</li><li><code>torch.mean(x.float(), dim=0)</code>：计算平均值，需要将整数类型转换为浮点数。</li><li><code>torch.eq(x, y)</code>：逐元素比较是否相等。</li><li><code>torch.sort(y, dim=0)</code>：对张量进行排序。</li><li><code>torch.clamp(x, min=0)</code>：将张量中的值限制在指定范围内。</li></ul><h3 id="2-9-布尔值操作">2.9 布尔值操作</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">x = torch.tensor([<span class="number">1</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">1</span>, <span class="number">1</span>], dtype=torch.<span class="built_in">bool</span>) <span class="comment"># 布尔值张量</span></span><br><span class="line">z = torch.<span class="built_in">any</span>(x) <span class="comment"># 如果任一元素为 True，则返回 True</span></span><br><span class="line">z = torch.<span class="built_in">all</span>(x) <span class="comment"># 如果所有元素为 True，则返回 True</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.any(x)</code>：检查张量中是否有任一元素为 <code>True</code>。</li><li><code>torch.all(x)</code>：检查张量中是否所有元素都为 <code>True</code>。</li></ul><h2 id="3-张量的索引操作">3. 张量的索引操作</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">batch_size = <span class="number">10</span></span><br><span class="line">features = <span class="number">25</span></span><br><span class="line">x = torch.rand((batch_size, features))</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取第一个样本的所有特征</span></span><br><span class="line"><span class="built_in">print</span>(x[<span class="number">0</span>].shape) <span class="comment"># 输出形状为 [25]，等同于 x[0, :]</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取所有样本的第一个特征</span></span><br><span class="line"><span class="built_in">print</span>(x[:, <span class="number">0</span>].shape) <span class="comment"># 输出形状为 [10]</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取第三个样本的前十个特征</span></span><br><span class="line"><span class="built_in">print</span>(x[<span class="number">2</span>, <span class="number">0</span>:<span class="number">10</span>].shape) <span class="comment"># 输出形状为 [10]</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 修改特定元素的值</span></span><br><span class="line">x[<span class="number">0</span>, <span class="number">0</span>] = <span class="number">100</span></span><br></pre></td></tr></table></figure><ul><li><code>x[0]</code>：获取第一个样本的所有特征。</li><li><code>x[:, 0]</code>：获取所有样本的第一个特征。</li><li><code>x[2, 0:10]</code>：获取第三个样本的前十个特征。</li><li><code>x[0, 0] = 100</code>：将第一个样本的第一个特征值修改为 100。</li></ul><h3 id="3-1-花式索引">3.1 花式索引</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">x = torch.arange(<span class="number">10</span>)</span><br><span class="line">indices = [<span class="number">2</span>, <span class="number">5</span>, <span class="number">8</span>]</span><br><span class="line"><span class="built_in">print</span>(x[indices]) <span class="comment"># 输出 [2, 5, 8]</span></span><br><span class="line"></span><br><span class="line">x = torch.rand((<span class="number">3</span>, <span class="number">5</span>))</span><br><span class="line">rows = torch.tensor([<span class="number">1</span>, <span class="number">0</span>])</span><br><span class="line">cols = torch.tensor([<span class="number">4</span>, <span class="number">0</span>])</span><br><span class="line"><span class="built_in">print</span>(x[rows, cols]) <span class="comment"># 获取第二行第五列和第一行第一列的值</span></span><br></pre></td></tr></table></figure><ul><li><code>x[indices]</code>：使用索引列表进行索引。</li><li><code>x[rows, cols]</code>：获取指定行和列的元素值。</li></ul><h3 id="3-2-更多高级索引">3.2 更多高级索引</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">x = torch.arange(<span class="number">10</span>)</span><br><span class="line"><span class="built_in">print</span>(x[(x < <span class="number">2</span>) | (x > <span class="number">8</span>)]) <span class="comment"># 输出 [0, 1, 9]</span></span><br><span class="line"><span class="built_in">print</span>(x[x.remainder(<span class="number">2</span>) == <span class="number">0</span>]) <span class="comment"># 输出 [0, 2, 4, 6, 8]</span></span><br></pre></td></tr></table></figure><ul><li><code>(x < 2) | (x > 8)</code>：逻辑或操作符，用于筛选满足条件的元素。</li><li><code>x.remainder(2) == 0</code>：筛选出能够被 2 整除的元素。</li></ul><h3 id="3-3-索引操作的有用函数">3.3 索引操作的有用函数</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">print</span>(torch.where(x > <span class="number">5</span>, x, x * <span class="number">2</span>)) <span class="comment"># 如果 x > 5，则返回 x，否则返回 x * 2</span></span><br><span class="line">x = torch.tensor([<span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>, <span class="number">2</span>, <span class="number">2</span>, <span class="number">3</span>, <span class="number">4</span>]).unique() <span class="comment"># 返回 [0, 1, 2, 3, 4]</span></span><br><span class="line"><span class="built_in">print</span>(x.ndimension()) <span class="comment"># 返回张量的维度数量</span></span><br><span class="line">x = torch.arange(<span class="number">10</span>)</span><br><span class="line"><span class="built_in">print</span>(x.numel()) <span class="comment"># 返回张量中的元素总数</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.where(condition, x, y)</code>：根据条件选择返回 <code>x</code> 或 <code>y</code>。</li><li><code>x.unique()</code>：返回张量中唯一的元素。</li><li><code>x.ndimension()</code>：返回张量的维度数量。</li><li><code>x.numel()</code>：返回张量中的元素总数。</li></ul><h2 id="4-张量的重塑">4. 张量的重塑</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">x = torch.arange(<span class="number">9</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 重塑为 3x3</span></span><br><span class="line">x_3x3 = x.view(<span class="number">3</span>, <span class="number">3</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># view 和 reshape 的区别</span></span><br><span class="line">x_3x3 = x.reshape(<span class="number">3</span>, <span class="number">3</span>)</span><br></pre></td></tr></table></figure><ul><li><code>x.view(3, 3)</code>：将张量重塑为 3x3。</li><li><code>x.reshape(3, 3)</code>：与 <code>view</code> 类似，但 <code>reshape</code> 可以处理非连续存储的张量。</li></ul><h3 id="4-1-其他重塑操作">4.1 其他重塑操作</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">y = x_3x3.t()</span><br><span class="line"><span class="built_in">print</span>(y.is_contiguous()) <span class="comment"># 检查张量是否是连续存储的，结果为 False</span></span><br><span class="line"><span class="built_in">print</span>(y.contiguous().view(<span class="number">9</span>)) <span class="comment"># 调用 `.contiguous()` 使其连续存储后再使用 view</span></span><br></pre></td></tr></table></figure><ul><li><code>y.is_contiguous()</code>：检查张量是否是连续存储的。</li><li><code>y.contiguous().view(9)</code>：将张量转换为连续存储后再重塑。</li></ul><h3 id="4-2-张量拼接与展开">4.2 张量拼接与展开</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">x1 = torch.rand(<span class="number">2</span>, <span class="number">5</span>)</span><br><span class="line">x2 = torch.rand(<span class="number">2</span>, <span class="number">5</span>)</span><br><span class="line"><span class="built_in">print</span>(torch.cat((x1, x2), dim=<span class="number">0</span>).shape) <span class="comment"># 拼接后形状为 4x5</span></span><br><span class="line"><span class="built_in">print</span>(torch.cat((x1, x2), dim=<span class="number">1</span>).shape) <span class="comment"># 拼接后形状为 2x10</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 展开张量</span></span><br><span class="line">z = x1.view(-<span class="number">1</span>) <span class="comment"># 展开为一维向量</span></span><br></pre></td></tr></table></figure><ul><li><code>torch.cat((x1, x2), dim=0)</code>：在指定维度上拼接张量。</li><li><code>x1.view(-1)</code>：将张量展开为一维向量。</li></ul><h3 id="4-3-改变维度顺序与添加-移除维度">4.3 改变维度顺序与添加/移除维度</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">batch = <span class="number">64</span></span><br><span class="line">x = torch.rand((batch, <span class="number">2</span>, <span class="number">5</span>))</span><br><span class="line"></span><br><span class="line"><span class="comment"># 改变维度顺序</span></span><br><span class="line">z = x.permute(<span class="number">0</span>, <span class="number">2</span>, <span class="number">1</span>) <span class="comment"># 交换维度 1 和维度 2</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 添加维度</span></span><br><span class="line">x = torch.arange(<span class="number">10</span>)</span><br><span class="line"><span class="built_in">print</span>(x.unsqueeze(<span class="number">0</span>).shape) <span class="comment"># 形状变为 [1, 10]</span></span><br><span class="line"><span class="built_in">print</span>(x.unsqueeze(<span class="number">1</span>).shape) <span class="comment"># 形状变为 [10, 1]</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 移除维度</span></span><br><span class="line">x = torch.arange(<span class="number">10</span>).unsqueeze(<span class="number">0</span>).unsqueeze(<span class="number">1</span>)</span><br><span class="line">z = x.squeeze(<span class="number">1</span>) <span class="comment"># 移除维度，形状变为 [1, 10]</span></span><br></pre></td></tr></table></figure><ul><li><code>x.permute(0, 2, 1)</code>：重新排列张量的维度。</li><li><code>x.unsqueeze(dim)</code>：在指定位置添加一个维度。</li><li><code>x.squeeze(dim)</code>：移除指定的维度。</li></ul>]]></content>
<categories>
<category> Pytorch应用 </category>
</categories>
<tags>
<tag> Pytorch </tag>
</tags>
</entry>
<entry>
<title>基于docker的服务器深度学习环境构建——pycharm连接调试全流程</title>
<link href="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/"/>
<url>/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/</url>
<content type="html"><![CDATA[<h1>基于docker的服务器深度学习环境构建——pycharm连接调试全流程</h1><h2 id="说明">说明</h2><p>该博客教程基于2024最新版pycharm。由于网上的配置教程一般都是几年前的，导致在按照往年教程配置深度学习环境过程中，会出现各种难以解决的问题。</p><p>本教程基于最新版pycharm，docker等，帮你一站式解决深度学习环境配置问题，包括：</p><ul><li>在服务器上构建docker容器</li><li>在docker容器上配置conda深度学习环境</li><li><strong>在本机pycharm上远程连接docker容器的深度学习环境，并可在主机直接进行运行，调试等</strong></li></ul><p><strong>You’re just a few steps away from success!</strong></p><h2 id="环境">环境</h2><p>服务器：Ubuntu20.04</p><p>Pycharm Professtional：2024.1.6</p><ul><li>注意，如果要远程连接服务器，pycharm必须使用professtional版，community版无法进行远程连接</li></ul><h2 id="零-基本环境配置">零. 基本环境配置</h2><p>默认你的深度学习服务器已经安装了<strong>Nvidia Driver, CUDA和cuDNN</strong>等深度学习组件，该教程不进行上述基本环境的配置讲解，请自行搜索（资源太多了）。</p><p>ps：如果你已经安装了docker容器，但是配置错误，建议根据本教程将已有docker容器删除后进行重新配置，删除命令如下，其中我的docker容器起名为<code>pycharm_lkq</code>，请根据你的需要自己取名。</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013101809660.png" class="" title="image-20241013101809660"><h2 id="一-在远程服务器上构建docker容器">一. 在远程服务器上构建docker容器</h2><h3 id="1-启动并运行容器">1. 启动并运行容器</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">sudo</span> docker run -it --name pycharm_lkq --shm-size 8g -v /home/lkq/PycharmProjects/qa/:/workspace/qa -p 8080:22 --gpus all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all registry.cn-hangzhou.aliyuncs.com/alter_images/pytorch:1.10.0-cuda11.3-cudnn8-devel /bin/bash</span><br></pre></td></tr></table></figure><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">sudo</span> docker run -it \</span><br><span class="line"> --name pycharm_lkq \</span><br><span class="line"> --shm-size 8g \</span><br><span class="line"> -v /home/lkq/PycharmProjects/:/workspace \</span><br><span class="line"> -v /mnt/data/lkq:/workspace/data \</span><br><span class="line"> -p 8080:22 \</span><br><span class="line"> --gpus all \</span><br><span class="line"> -e NVIDIA_DRIVER_CAPABILITIES=compute,utility \</span><br><span class="line"> -e NVIDIA_VISIBLE_DEVICES=all \</span><br><span class="line"> registry.cn-hangzhou.aliyuncs.com/alter_images/pytorch:1.10.0-cuda11.3-cudnn8-devel \</span><br><span class="line"> /bin/bash</span><br></pre></td></tr></table></figure><p>注意几个重要参数</p><ul><li><code>-p 8080:22</code>：主机端口是8080，容器端口是22。建议不要更改容器端口，如果8080被占用，可以更改主机端口为8081等合法端口号</li><li><code>registry.cn-hangzhou.aliyuncs.com/alter_images/pytorch:1.10.0-cuda11.3-cudnn8-devel /bin/bash</code>：这里的基础镜像请与你需要的pytorch环境保持一致，不用与示例一样</li><li><code>--shm-size 8g</code>：可选，如果服务器内存够大建议添加，可扩大docker容器内存，避免后期容器内存不够的情况</li><li><code>-e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all</code>：可选，建议添加，作用可自行检索</li><li>其他参数请自行搜索</li></ul><p><strong>在执行上述命令后，会默认进入到docker容器中。下面的步骤2-6都在容器中执行。</strong></p><h3 id="2-进入容器后修改root密码">2. 进入容器后修改root密码</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">passwd</span><br></pre></td></tr></table></figure><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013104628151.png" class="" title="image-20241013104628151"><h3 id="3-更新-etc-apt-source-list中的源">3. 更新/etc/apt/source.list中的源</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">apt-get update</span><br></pre></td></tr></table></figure><p>（可选）可能出现错误：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013102528684.png" class="" title="image-20241013102528684"><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease' is no longer signed.</span><br></pre></td></tr></table></figure><p>解决方案：</p><p>后两行代码需要注意ubuntu对应的版本，如果是18.04就可以直接复制，如果不是就改成当前ubuntu系统对应的版本，我已经将其改成了ubuntu2004</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">rm</span> /etc/apt/sources.list.d/cuda.list</span><br><span class="line"><span class="built_in">rm</span> /etc/apt/sources.list.d/nvidia-ml.list</span><br><span class="line">apt-key del 7fa2af80</span><br><span class="line">apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub</span><br><span class="line">apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu2004/x86_64/7fa2af80.pub</span><br></pre></td></tr></table></figure><p>再次运行：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013105028085.png" class="" title="image-20241013105028085"><ul><li>运行成功</li></ul><h3 id="4-安装容器远程连接等必要的组件">4. 安装容器远程连接等必要的组件</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">apt-get install openssh-server</span><br><span class="line">apt-get install openssh-client</span><br><span class="line">apt-get install vim</span><br><span class="line">apt-get install iputils-ping</span><br><span class="line">apt-get install wget</span><br><span class="line">apt-get install procps</span><br></pre></td></tr></table></figure><h3 id="5-修改SSH配置文件">5. 修改SSH配置文件</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim /etc/ssh/sshd_config</span><br></pre></td></tr></table></figure><p>以上命令打开sshd_config文件，在文件最后面添加：<code>PermitRootLogin yes</code></p><ul><li>即允许root用户使用ssh登录</li></ul><h3 id="6-重启ssh服务：">6. 重启ssh服务：</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">/etc/init.d/ssh restart</span><br></pre></td></tr></table></figure><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013105347991.png" class="" title="image-20241013105347991"><h3 id="7-退出容器进行连接测试">7. 退出容器进行连接测试</h3><p>使用exit命令退出容器</p><p>执行命令：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh [email protected] -p 8080</span><br></pre></td></tr></table></figure><ul><li>root：容器内部系统的root账号，不是服务器的用户账号</li><li>127.0.0.1：服务器本机ip</li><li>-p 端口号：这里的端口就是我们在步骤1 启动容器时映射的主机端口号8080，不是容器的端口22<ul><li>主机端口号根据你的设置进行修改</li></ul></li></ul><p>（可选）<strong>执行上述命令后</strong>可能会出现很多问题，按照时间顺序梳理如下：</p><ol><li><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013110121446.png" class="" title="image-20241013110121446"></li></ol><p>这是因为你的docker容器没有启动，在主机上执行<code>sudo docker start pycharm_lkq</code></p><ol start="2"><li><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013110158834.png" class="" title="image-20241013110158834"></li></ol><p>需要重新进入docker容器，在容器内执行<code>service ssh restart</code></p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241028160844098.png" class="" title="image-20241028160844098"><ol start="3"><li><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013110336770.png" class="" title="image-20241013110336770"></li></ol><p>该问题大多数情况出现在你重新配置了已删除的docker容器。在主机上执行<code>vim ~/.ssh/known_hosts</code>，而后清空该文件中的内容，从而去除过时的认证</p><p>成功连接如下图所示：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013105745406.png" class="" title="image-20241013105745406"><p><strong>如果该步如图所示，那么表明第一阶段的docker容器配置已经成功！</strong></p><h2 id="二-在docker容器上配置conda环境">二. 在docker容器上配置conda环境</h2><ul><li>请你重新进入docker容器，而后在容器内执行以下步骤</li></ul><h3 id="1-确定CUDA版本">1. 确定CUDA版本</h3><p>执行<code>nvcc -V</code>确定CUDA版本，并检查docker容器内环境是否成功配置</p><p>（可选）可能出现<code>-bash: nvcc: command not found</code>错误。不要慌，如果服务器上的环境配置正确，那么该docker容器中的环境大概率仍是正确的，只需要进行如下配置</p><p>首先，查看cuda的bin目录下是否有nvcc：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> /usr/local/cuda/bin</span><br></pre></td></tr></table></figure><p>如果存在，直接将cuda路径加入系统路径即可：</p><p><code>vim ~/.bashrc</code>进入配置文件，而后在文件末尾添加以下两行：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">export</span> PATH=/usr/local/cuda/bin:<span class="variable">$PATH</span></span><br><span class="line"><span class="built_in">export</span> LD_LIBRARY_PATH=/usr/local/cuda/lib64:<span class="variable">$LD_LIBRARY_PATH</span></span><br></pre></td></tr></table></figure><p>最后更新配置文件：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">source</span> ~/.bashrc</span><br></pre></td></tr></table></figure><p>再次执行nvcc -V就能看到相应cuda版本了，例如：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013110933273.png" class="" title="image-20241013110933273"><h3 id="2-安装miniconda">2. 安装miniconda</h3><p>进入<a href="https://docs.conda.io/projects/miniconda/en/latest/">miniconda官网</a>，选择命令行下载，获取命令行下载的指令。</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013111102171.png" class="" title="image-20241013111102171"><p>（可选）：miniconda官网的下载指令可能会更新，可以使用以下指令，在目前阶段经过测试一直是可用的</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">mkdir</span> -p ~/miniconda3</span><br><span class="line">wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh</span><br><span class="line">bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3</span><br><span class="line"><span class="built_in">rm</span> ~/miniconda3/miniconda.sh</span><br></pre></td></tr></table></figure><p>初始化miniconda和shell</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">~/miniconda3/bin/conda init bash</span><br><span class="line">~/miniconda3/bin/conda init zsh</span><br></pre></td></tr></table></figure><p>此时使用<code>exit</code>退出容器，而后重新进入，就可以看到base环境已经成功构建了<img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013111226256.png" class="" title="image-20241013111226256"></p><h3 id="3-配置conda环境">3. 配置conda环境</h3><ul><li>这里根据你的需要配置环境即可，这里举一个简单的例子</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">conda create -n detrex python=3.8</span><br><span class="line">conda activate detrex</span><br></pre></td></tr></table></figure><ul><li>此时你就拥有了一个名为<code>detrex</code>，同时python版本为3.8的conda环境，你可以在这里面进行其他包的安装，<strong>后续的教程默认你已经配置好完成的conda环境</strong></li></ul><h2 id="三-在pycharm上远程连接docker容器的conda环境">三. 在pycharm上远程连接docker容器的conda环境</h2><h3 id="1-配置远程连接到服务器">1. 配置远程连接到服务器</h3><p>点击工具，进入部署的配置选项</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013094632487.png" class=""><p>点击加号，选择SFTP</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013111505596.png" class="" title="image-20241013111505596"><p>首先看到的是右图，点击其中SSH配置框右侧的<code>...</code>，弹出左图</p><ul><li>主机：远程服务器ip</li><li>端口号：docker容器映射到的主机端口号</li><li>用户名：docker容器内部的root账户</li><li>身份验证类型：密码（之前设置过的docker容器内的密码）</li></ul><p>而后点击<strong>测试连接</strong>查看是否能成功连接</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013094807262.png" class="" title="image-20241013094807262"><p>而后配置路径映射：</p><p>本地路径即为该项目所在路径，一般不用改</p><p>部署路径即为你希望将你的项目部署在服务器的位置</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013094921865.png" class="" title="image-20241013094921865"><h3 id="2-上传本地代码数据到docker容器中">2. 上传本地代码数据到docker容器中</h3><p>右键项目根目录，点击部署，上传到…，将本地代码数据上传到服务器上的docker容器中</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013094957291.png" class="" title="image-20241013094957291"><p>（可选）：在工具栏中点击<strong>自动上传（始终）<strong>和</strong>浏览远程主机</strong>，方便之后代码的修改和调试</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013095052338.png" class="" title="image-20241013095052338"><h3 id="3-配置python解释器">3. 配置python解释器</h3><p>点击文件——设置</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013112205222.png" class="" title="image-20241013112205222"><p>选择项目——python解释器，点击右侧的“添加解释器”——SSH</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013100139446.png" class="" title="image-20241013100139446"><p>点击“现有”，选择刚刚配置好的docker容器</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013113609442.png" class="" title="image-20241013113609442"><p>点击下一步</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013100228212.png" class="" title="image-20241013100228212"><p>现在SSH已经设置好，接下来就该配置conda环境了</p><p>下面是最坑的地方之一。由于pycharm2024的更新，<strong>导致原有的很多教程的设置都是错误的</strong>。</p><p>如下图：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013113646993.png" class="" title="image-20241013113646993"><p>如果此时你按照常理选择了”Conda环境“，你会发现不管你使用教程的方法配置，还是自己折腾，基本全会报错，比如下图的错误：</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013114012190.png" class="" title="image-20241013114012190"><p><strong>解决方法：</strong></p><ol><li>在docker容器内执行<code>which conda</code>，找到conda路径</li></ol><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013100510257.png" class="" title="image-20241013100510257"><ul><li>如上图，我们可以知道conda位于<code>/root/miniconda3/bin</code>目录下</li></ul><ol start="2"><li>回到pycharm，选择默认的<strong>Virtualenv环境</strong>，点击“现有”，点击解释器右边的<code>...</code>，根据上述conda路径选择对应envs中的<code>/bin/python</code>文件</li></ol><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013100729725.png" class="" title="image-20241013100729725"><p>同步文件夹：映射目录根据你的设置选择容器中的目录</p><img src="/2024/10/13/%E5%9F%BA%E4%BA%8Edocker%E4%BA%8E%E6%9C%8D%E5%8A%A1%E5%99%A8%E6%9E%84%E5%BB%BA%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E7%8E%AF%E5%A2%83%E2%80%94%E2%80%94pycharm%E8%BF%9E%E6%8E%A5%E8%B0%83%E8%AF%95%E5%85%A8%E6%B5%81%E7%A8%8B/image-20241013100933057.png" class="" title="image-20241013100933057"><h2 id="四、小结">四、小结</h2><p>至此，如果你已经成功完成了以上三步，那么恭喜你，已经成功完成了pycharm连接docker容器深度学习环境的全流程！</p>]]></content>
<categories>
<category> 深度学习服务器 </category>
</categories>
<tags>
<tag> DL环境配置 </tag>
</tags>
</entry>
<entry>
<title>RevisitingML_从数学角度重游机器学习</title>
<link href="/2024/10/07/Revisiting_ML/"/>
<url>/2024/10/07/Revisiting_ML/</url>
<content type="html"><![CDATA[<h1>Revisiting_ML</h1><h2 id="基础回顾">基础回顾</h2><h3 id="向量的模与范数">向量的模与范数</h3><p>在机器学习中，<strong>范数（Norm）</strong> 是一个非常重要的概念，用来衡量向量的长度或大小。常见的范数包括 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>L</mi><mn>1</mn></msup></mrow><annotation encoding="application/x-tex">L^1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8141em;"></span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span></span></span></span></span></span></span></span> 范数、 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>L</mi><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">L^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8141em;"></span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span></span> 范数和一般的 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>L</mi><mi>p</mi></msup></mrow><annotation encoding="application/x-tex">L^p</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">p</span></span></span></span></span></span></span></span></span></span></span> 范数，它们在不同的场景下有不同的作用。下面我们详细解释几种常见的范数及其在机器学习中的应用。</p><ol><li>一般的<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><msup><mi>L</mi><mi>p</mi></msup><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">(L^p)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">p</span></span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span>范数</li></ol><p>范数的通用公式为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi mathvariant="normal">∥</mi><mi>x</mi><msub><mi mathvariant="normal">∥</mi><mi>p</mi></msub><mo>=</mo><msup><mrow><mo fence="true">(</mo><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mi mathvariant="normal">∣</mi><msub><mi>x</mi><mi>i</mi></msub><msup><mi mathvariant="normal">∣</mi><mi>p</mi></msup><mo fence="true">)</mo></mrow><mfrac><mn>1</mn><mi>p</mi></mfrac></msup></mrow><annotation encoding="application/x-tex">\|x\|_p = \left( \sum_{i=1}^{n} |x_i|^p \right)^{\frac{1}{p}}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.0361em;vertical-align:-0.2861em;"></span><span class="mord">∥</span><span class="mord mathnormal">x</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">p</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.2861em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:3.3716em;vertical-align:-1.2777em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top:0em;"><span class="delimsizing size4">(</span></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.2777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord">∣</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.7144em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">p</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top:0em;"><span class="delimsizing size4">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:2.0939em;"><span style="top:-4.5029em;margin-right:0.05em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mopen nulldelimiter sizing reset-size3 size6"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8443em;"><span style="top:-2.656em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">p</span></span></span></span><span style="top:-3.2255em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line mtight" style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.4829em;"><span></span></span></span></span></span><span class="mclose nulldelimiter sizing reset-size3 size6"></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p><p>其中，向量 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi></mrow><annotation encoding="application/x-tex">x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal">x</span></span></span></span> 的分量为 ( <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>x</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">x_1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>x</mi><mn>2</mn></msub></mrow><annotation encoding="application/x-tex">x_2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>…</mo></mrow><annotation encoding="application/x-tex">\dots</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.123em;"></span><span class="minner">…</span></span></span></span>, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>x</mi><mi>n</mi></msub></mrow><annotation encoding="application/x-tex">x_n</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> )，<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi></mrow><annotation encoding="application/x-tex">p</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.1944em;"></span><span class="mord mathnormal">p</span></span></span></span> 是一个大于等于1的实数，表示不同的范数。范数的通用定义适用于所有 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mi>p</mi><mo>≥</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">( p \geq 1 )</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal">p</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">≥</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">1</span><span class="mclose">)</span></span></span></span>，它是对向量分量绝对值的某种程度上的综合度量。</p><ol start="2"><li>特殊的范数： <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>L</mi><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">L^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8141em;"></span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span></span> 和 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msup><mi>L</mi><mn>1</mn></msup></mrow><annotation encoding="application/x-tex">L^1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8141em;"></span><span class="mord"><span class="mord mathnormal">L</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span></span></span></span></span></span></span></span></li></ol><p>在机器学习中，最常用的范数是 ( L^2 ) 范数和 ( L^1 ) 范数。</p><h5 id="L-2-范数（欧几里得范数）">( L^2 ) 范数（欧几里得范数）</h5><p>当 ( p = 2 ) 时，范数称为<strong>欧几里得范数</strong>，其定义为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi mathvariant="normal">∥</mi><mi>x</mi><msub><mi mathvariant="normal">∥</mi><mn>2</mn></msub><mo>=</mo><msup><mrow><mo fence="true">(</mo><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mi mathvariant="normal">∣</mi><msub><mi>x</mi><mi>i</mi></msub><msup><mi mathvariant="normal">∣</mi><mn>2</mn></msup><mo fence="true">)</mo></mrow><mfrac><mn>1</mn><mn>2</mn></mfrac></msup></mrow><annotation encoding="application/x-tex">\|x\|_2 = \left( \sum_{i=1}^{n} |x_i|^2 \right)^{\frac{1}{2}}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∥</span><span class="mord mathnormal">x</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:3.3716em;vertical-align:-1.2777em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top:0em;"><span class="delimsizing size4">(</span></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.2777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord">∣</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top:0em;"><span class="delimsizing size4">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:2.0939em;"><span style="top:-4.5029em;margin-right:0.05em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mopen nulldelimiter sizing reset-size3 size6"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8443em;"><span style="top:-2.656em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span><span style="top:-3.2255em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line mtight" style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.344em;"><span></span></span></span></span></span><span class="mclose nulldelimiter sizing reset-size3 size6"></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p><p>这个公式实际上就是我们在几何中常见的<strong>欧几里得距离</strong>，它表示向量 ( x ) 在 ( n )-维空间中的长度，或从原点出发到向量 ( x ) 所在点的距离。</p><ul><li><strong>应用</strong>：欧几里得范数在机器学习中十分常见，特别是在梯度下降优化、神经网络权重的正则化等问题中，它常常用于度量误差（如均方误差, Mean Squared Error, MSE）。由于其几何性质，( L^2 ) 范数可以捕捉到向量各个分量的平方和，这意味着远离零的元素对范数的贡献较大，因此 ( L^2 ) 范数对大分量更为敏感。</li></ul><h5 id="L-1-范数（曼哈顿范数）">( L^1 ) 范数（曼哈顿范数）</h5><p>当 ( p = 1 ) 时，范数称为<strong>曼哈顿范数</strong>，其定义为：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi mathvariant="normal">∥</mi><mi>x</mi><msub><mi mathvariant="normal">∥</mi><mn>1</mn></msub><mo>=</mo><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mi mathvariant="normal">∣</mi><msub><mi>x</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">\|x\|_1 = \sum_{i=1}^{n} |x_i|</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∥</span><span class="mord mathnormal">x</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:2.9291em;vertical-align:-1.2777em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.2777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span></span></span></span></span></p><p>它表示向量中每个分量的绝对值之和。( L^1 ) 范数通常用于强调稀疏性，因为它对零和非零元素之间的变化敏感，因此有助于筛选出重要的特征。</p><ul><li><strong>应用</strong>：$( L^1 ) 范数在机器学习中通常用于构建<strong>稀疏模型</strong>，例如 Lasso 回归（L1 正则化）中，它通过对系数向量施加 ( L^1 ) 正则化约束，使得某些系数变为零，从而实现特征选择和模型的稀疏化。在数据集稀疏化问题中（例如稀疏编码、稀疏矩阵分解等），( L^1 ) 范数经常用于作为正则化项，因为它能有效地引导模型忽略不重要的特征，从而得到稀疏解。</li></ul><h5 id="范数的几何解释">范数的几何解释</h5><ul><li><strong>欧几里得距离 vs 曼哈顿距离</strong>：当我们使用 ( L^2 ) 范数时，距离的几何解释是直线距离，也就是两个点之间的最短路径，这就是欧几里得距离。而使用 ( L^1 ) 范数时，距离的几何解释为“曼哈顿距离”，即在格子网格上行走时，沿水平和垂直方向移动的总距离。曼哈顿距离更注重维度之间的单独影响，而欧几里得距离更关心整体距离。</li></ul><h5 id="不同范数的选择">不同范数的选择</h5><p>选择不同的范数会对机器学习算法的效果产生不同的影响：</p><ol><li><p><strong>( L^2 ) 范数（欧几里得范数）</strong>：</p><ul><li><strong>优点</strong>：对大分量敏感，因此能够惩罚离散值。常用于机器学习中的误差度量和正则化，帮助模型更平滑。</li><li><strong>缺点</strong>：对异常值较为敏感，因为大值的平方会导致较大的影响。</li></ul></li><li><p><strong>( L^1 ) 范数（曼哈顿范数）</strong>：</p><ul><li><strong>优点</strong>：对稀疏性敏感，可以有效减少模型中不重要的特征。广泛应用于特征选择和稀疏模型中。</li><li><strong>缺点</strong>：由于只看每个分量的绝对值，对整体几何信息的捕捉不如 ( L^2 ) 范数。</li></ul></li></ol><h5 id="p-趋于无穷的-L-infty-范数">( p ) 趋于无穷的 ( L^\infty ) 范数</h5><p>当 ( p ) 趋于无穷时，( L^p ) 范数将趋向于 <strong>最大值范数</strong>，即：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi mathvariant="normal">∥</mi><mi>x</mi><msub><mi mathvariant="normal">∥</mi><mi mathvariant="normal">∞</mi></msub><mo>=</mo><munder><mrow><mi>max</mi><mo></mo></mrow><mi>i</mi></munder><mi mathvariant="normal">∣</mi><msub><mi>x</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">\|x\|_\infty = \max_i |x_i|</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∥</span><span class="mord mathnormal">x</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">∞</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.4777em;vertical-align:-0.7277em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.4306em;"><span style="top:-2.3723em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop">max</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.7277em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span></span></span></span></span></p><p>这表示向量中最大的分量的绝对值。它专注于单一维度的极值，而忽略了其他维度的贡献。在某些优化问题中使用 ( L^\infty ) 范数可以使模型更为保守，因为它只关注最极端的误差。</p><h3 id="总结">总结</h3><p>在机器学习中，范数是用于衡量向量大小的基本工具，选择合适的范数可以帮助我们更好地控制模型的行为：</p><ul><li><strong>( L^2 ) 范数</strong> 用于度量整体的误差和距离，强调较大值的影响；</li><li><strong>( L^1 ) 范数</strong> 强调稀疏性，适合稀疏模型的构建；</li><li><strong>( L^\infty ) 范数</strong> 专注于极值，适合某些保守的优化问题。</li></ul><h3 id="loss">loss</h3><h2 id="Introduction">Introduction</h2><p>对概率的诠释有两大学派，一种是频率派另一种是贝叶斯派。后面我们对观测集采用下面记号：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>X</mi><mrow><mi>N</mi><mo>×</mo><mi>p</mi></mrow></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>x</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>x</mi><mn>2</mn></msub><mo separator="true">,</mo><mo>⋯</mo><mtext> </mtext><mo separator="true">,</mo><msub><mi>x</mi><mi>N</mi></msub><msup><mo stretchy="false">)</mo><mi>T</mi></msup><mo separator="true">,</mo><msub><mi>x</mi><mi>i</mi></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>i</mi><mn>1</mn></mrow></msub><mo separator="true">,</mo><msub><mi>x</mi><mrow><mi>i</mi><mn>2</mn></mrow></msub><mo separator="true">,</mo><mo>⋯</mo><mtext> </mtext><mo separator="true">,</mo><msub><mi>x</mi><mrow><mi>i</mi><mi>p</mi></mrow></msub><msup><mo stretchy="false">)</mo><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">X_{N\times p}=(x_{1},x_{2},\cdots,x_{N})^{T},x_{i}=(x_{i1},x_{i2},\cdots,x_{ip})^{T}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.9694em;vertical-align:-0.2861em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">N</span><span class="mbin mtight">×</span><span class="mord mathnormal mtight">p</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.2861em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.1413em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">N</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.1774em;vertical-align:-0.2861em;"></span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mathnormal mtight">p</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.2861em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span></span></span></p><p>这个记号表示有 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi></mrow><annotation encoding="application/x-tex">N</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.10903em;">N</span></span></span></span> 个样本，每个样本都是 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi></mrow><annotation encoding="application/x-tex">p</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.1944em;"></span><span class="mord mathnormal">p</span></span></span></span> 维向量。其中每个观测都是由 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">p(x|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span> 生成的。</p><h3 id="频率派的观点">频率派的观点</h3><p><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">p(x|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span>中的 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 是一个常量。对于 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi></mrow><annotation encoding="application/x-tex">N</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.10903em;">N</span></span></span></span> 个观测来说观测集的概率为 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><munder><mo><mo>=</mo></mo><mrow><mi>i</mi><mi>i</mi><mi>d</mi></mrow></munder><msubsup><mo>∏</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></msubsup><mi>p</mi><mo stretchy="false">(</mo><msub><mi>x</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">p(X|\theta)\mathop{=}\limits _{iid}\prod\limits _{i=1}^{N}p(x_{i}|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:2.506em;vertical-align:-0.9777em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3669em;"><span style="top:-2.3479em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">ii</span><span class="mord mathnormal mtight">d</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop">=</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.7521em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.5283em;"><span style="top:-2.1223em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop op-symbol small-op">∏</span></span></span><span style="top:-3.95em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">N</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.9777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span> 。为了求 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 的大小，我们采用最大对数似然MLE的方法：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>θ</mi><mrow><mi>M</mi><mi>L</mi><mi>E</mi></mrow></msub><mo>=</mo><munder><mo><mi>a</mi><mi>r</mi><mi>g</mi><mi>m</mi><mi>a</mi><mi>x</mi></mo><mi>θ</mi></munder><mi>log</mi><mo></mo><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><munder><mo><mo>=</mo></mo><mrow><mi>i</mi><mi>i</mi><mi>d</mi></mrow></munder><munder><mo><mi>a</mi><mi>r</mi><mi>g</mi><mi>m</mi><mi>a</mi><mi>x</mi></mo><mi>θ</mi></munder><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></munderover><mi>log</mi><mo></mo><mi>p</mi><mo stretchy="false">(</mo><msub><mi>x</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\theta_{MLE}=\mathop{argmax}\limits _{\theta}\log p(X|\theta)\mathop{=}\limits _{iid}\mathop{argmax}\limits _{\theta}\sum\limits _{i=1}^{N}\log p(x_{i}|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8444em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">M</span><span class="mord mathnormal mtight">L</span><span class="mord mathnormal mtight" style="margin-right:0.05764em;">E</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:3.106em;vertical-align:-1.2777em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.4306em;"><span style="top:-2.1535em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop"><span class="mord mathnormal">a</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">x</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.9465em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop">lo<span style="margin-right:0.01389em;">g</span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3669em;"><span style="top:-2.3479em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">ii</span><span class="mord mathnormal mtight">d</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop">=</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.7521em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.4306em;"><span style="top:-2.1535em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop"><span class="mord mathnormal">a</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">x</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.9465em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.8283em;"><span style="top:-1.8723em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">N</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.2777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop">lo<span style="margin-right:0.01389em;">g</span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span></span></p><h3 id="贝叶斯派的观点">贝叶斯派的观点</h3><p>贝叶斯派认为 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">p(x|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span> 中的 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 不是一个常量。这个 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 满足一个预设的先验的分布 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi><mo>∼</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\theta\sim p(\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">∼</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span> 。于是根据贝叶斯定理依赖观测集参数的后验可以写成：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mi mathvariant="normal">∣</mi><mi>X</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo></mrow></mfrac><mo>=</mo><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><mrow><munder><mo>∫</mo><mi>θ</mi></munder><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo><mi>d</mi><mi>θ</mi></mrow></mfrac></mrow><annotation encoding="application/x-tex">p(\theta|X)=\frac{p(X|\theta)\cdot p(\theta)}{p(X)}=\frac{p(X|\theta)\cdot p(\theta)}{\int\limits _{\theta}p(X|\theta)\cdot p(\theta)d\theta}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:2.363em;vertical-align:-0.936em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.427em;"><span style="top:-2.314em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mclose">)</span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.936em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:3.1802em;vertical-align:-1.7532em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.427em;"><span style="top:-2.3044em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8056em;"><span style="top:-2.0423em;margin-left:-0.1945em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3.0006em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop op-symbol small-op" style="margin-right:0.19445em;">∫</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.0577em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mord mathnormal">d</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.7532em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></p><p>为了求 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 的值，我们要最大化这个参数后验MAP：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi>θ</mi><mrow><mi>M</mi><mi>A</mi><mi>P</mi></mrow></msub><mo>=</mo><munder><mo><mi>a</mi><mi>r</mi><mi>g</mi><mi>m</mi><mi>a</mi><mi>x</mi></mo><mi>θ</mi></munder><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mi mathvariant="normal">∣</mi><mi>X</mi><mo stretchy="false">)</mo><mo>=</mo><munder><mo><mi>a</mi><mi>r</mi><mi>g</mi><mi>m</mi><mi>a</mi><mi>x</mi></mo><mi>θ</mi></munder><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\theta_{MAP}=\mathop{argmax}\limits _{\theta}p(\theta|X)=\mathop{argmax}\limits _{\theta}p(X|\theta)\cdot p(\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8444em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.10903em;">M</span><span class="mord mathnormal mtight">A</span><span class="mord mathnormal mtight" style="margin-right:0.13889em;">P</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.6965em;vertical-align:-0.9465em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.4306em;"><span style="top:-2.1535em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop"><span class="mord mathnormal">a</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">x</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.9465em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.6965em;vertical-align:-0.9465em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.4306em;"><span style="top:-2.1535em;margin-left:0em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span><span class="mop"><span class="mord mathnormal">a</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal">ma</span><span class="mord mathnormal">x</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.9465em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span></span></p><p>其中第二个等号是由于分母和 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 没有关系。求解这个 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>θ</mi></mrow><annotation encoding="application/x-tex">\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span> 值后计算<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo></mrow><mrow><munder><mo>∫</mo><mi>θ</mi></munder><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mo stretchy="false">)</mo><mi>d</mi><mi>θ</mi></mrow></mfrac></mrow><annotation encoding="application/x-tex">\frac{p(X|\theta)\cdot p(\theta)}{\int\limits _{\theta}p(X|\theta)\cdot p(\theta)d\theta}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:2.1613em;vertical-align:-1.1513em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.01em;"><span style="top:-2.6261em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mop op-limits mtight"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8056em;"><span style="top:-1.8374em;margin-left:-0.1945em;"><span class="pstrut" style="height:2.805em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-2.8056em;"><span class="pstrut" style="height:2.805em;"></span><span><span class="mop op-symbol small-op mtight" style="margin-right:0.19445em;">∫</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.1106em;"><span></span></span></span></span></span><span class="mspace mtight" style="margin-right:0.1952em;"></span><span class="mord mathnormal mtight">p</span><span class="mopen mtight">(</span><span class="mord mathnormal mtight" style="margin-right:0.07847em;">X</span><span class="mord mtight">∣</span><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span><span class="mclose mtight">)</span><span class="mbin mtight">⋅</span><span class="mord mathnormal mtight">p</span><span class="mopen mtight">(</span><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span><span class="mclose mtight">)</span><span class="mord mathnormal mtight">d</span><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.485em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">p</span><span class="mopen mtight">(</span><span class="mord mathnormal mtight" style="margin-right:0.07847em;">X</span><span class="mord mtight">∣</span><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span><span class="mclose mtight">)</span><span class="mbin mtight">⋅</span><span class="mord mathnormal mtight">p</span><span class="mopen mtight">(</span><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span><span class="mclose mtight">)</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.1513em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span> ，就得到了参数的后验概率。其中 <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>X</mi><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">p(X|\theta)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span></span></span></span> 叫似然，是我们的模型分布。得到了参数的后验分布后，我们可以将这个分布用于预测贝叶斯预测：</p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>p</mi><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>n</mi><mi>e</mi><mi>w</mi></mrow></msub><mi mathvariant="normal">∣</mi><mi>X</mi><mo stretchy="false">)</mo><mo>=</mo><munder><mo>∫</mo><mi>θ</mi></munder><mi>p</mi><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>n</mi><mi>e</mi><mi>w</mi></mrow></msub><mi mathvariant="normal">∣</mi><mi>θ</mi><mo stretchy="false">)</mo><mo>⋅</mo><mi>p</mi><mo stretchy="false">(</mo><mi>θ</mi><mi mathvariant="normal">∣</mi><mi>X</mi><mo stretchy="false">)</mo><mi>d</mi><mi>θ</mi></mrow><annotation encoding="application/x-tex">p(x_{new}|X)=\int\limits _{\theta}p(x_{new}|\theta)\cdot p(\theta|X)d\theta</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span><span class="mord mathnormal mtight">e</span><span class="mord mathnormal mtight" style="margin-right:0.02691em;">w</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:2.9744em;vertical-align:-1.6132em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.3611em;"><span style="top:-1.8468em;margin-left:-0.4445em;"><span class="pstrut" style="height:3.36em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.02778em;">θ</span></span></span></span><span style="top:-3.3611em;"><span class="pstrut" style="height:3.36em;"></span><span><span class="mop op-symbol large-op" style="margin-right:0.44445em;">∫</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.6132em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span><span class="mord mathnormal mtight">e</span><span class="mord mathnormal mtight" style="margin-right:0.02691em;">w</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">p</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mclose">)</span><span class="mord mathnormal">d</span><span class="mord mathnormal" style="margin-right:0.02778em;">θ</span></span></span></span></span></p><p>其中积分中的被乘数是模型，乘数是后验分布。</p><h3 id="小结">小结</h3><p>频率派和贝叶斯派分别给出了一系列的机器学习算法。频率派的观点导出了一系列的统计机器学习算法而贝叶斯派导出了概率图理论。在应用频率派的 MLE 方法时最优化理论占有重要地位。而贝叶斯派的算法无论是后验概率的建模还是应用这个后验进行推断时积分占有重要地位。因此采样积分方法如 MCMC 有很多应用。</p><h2 id="Math">Math</h2><h3 id="先验、后验与似然概率">先验、后验与似然概率</h3><p>属于<strong>朴素贝叶斯</strong>（Naive Bayes）相关概念</p><p><strong>1、先验概率(prior probability)</strong></p><p>百度百科：先验概率（prior probability）是指<strong>根据以往经验和分析得到的概率</strong>，如全概率公式，它往往作为“由因求果”问题中的“因”出现的概率。</p><p>维基百科：在贝叶斯统计中，某一不确定量p的先验概率（prior probability）分布是<strong>在考虑“观测数据”前，能表达p不确定性的概率分布</strong>。它旨在描述这个不确定量的不确定程度，而不是这个不确定量的随机性。这个不确定量可以是一个参数，或者是一个隐含变量（英语：latent variable）。</p><p>我们可以发现这两个定义有一个共同点，即<strong>先验概率是不依靠观测数据的概率分布</strong>，也就是与其他因素独立的分布。在<a href="https://so.csdn.net/so/search?q=%E6%9C%B4%E7%B4%A0%E8%B4%9D%E5%8F%B6%E6%96%AF&spm=1001.2101.3001.7020">朴素贝叶斯</a>中，<strong>类别c的概率就是先验概率，表示为<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mo stretchy="false">(</mo><mi>C</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">P(C)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.13889em;">P</span><span class="mopen">(</span><span class="mord mathnormal" style="margin-right:0.07153em;">C</span><span class="mclose">)</span></span></span></span>。</strong></p><p><strong>2、后验概率(posterior probability)</strong></p><p>百度百科：后验概率是指<strong>在得到“结果”的信息后重新修正的概率</strong>，是“执果寻因”问题中的"果"。</p><p>维基百科：在贝叶斯统计中，一个随机事件或者一个不确定事件的后验概率（posterior probability）是在考虑和给出相关证据或数据后所得到的条件概率。同样，后验概率分布是一个未知量（视为随机变量）基于试验和调查后得到的概率分布。“后验”在本文中代表考虑了被测试事件的相关证据。</p><p>在朴素贝叶斯中，<strong>后验概率指给定数据x后，类别<img src="https://latex.csdn.net/eq?c%5Cin%20C" alt="cn C">的概率<img src="https://latex.csdn.net/eq?P%28c%20%7C%20x%29" alt="P(c | x)">。</strong></p><p>先验概率与后验概率有不可分割的联系，后验概率的计算要以先验概率为基础。<strong>事情还没有发生，要求这件事情发生的可能性的大小，是先验概率。事情已经发生，要求这件事情发生的原因是由某个因素引起的可能性的大小，是后验概率。</strong></p><p><strong>3、似然概率(likelihood)</strong></p><p>百度百科：统计学中，似然函数是一种关于统计模型参数的函数。<strong>给定输出<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>x</mi></mrow><annotation encoding="application/x-tex">x</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal">x</span></span></span></span>时，关于参数<img src="https://latex.csdn.net/eq?%5Ctheta" alt="heta">的似然函数 <img src="https://latex.csdn.net/eq?L%28%5Ctheta%7Cx%29" alt="L(heta|x)">（在数值上）等于给定参数<img src="https://latex.csdn.net/eq?%5Ctheta" alt="heta">后变量X的概率：<img src="https://latex.csdn.net/eq?L%28%5Ctheta%7Cx%29%3DP%28X%3Dx%7C%5Ctheta%29" alt="L(heta|x)=P(X=x|heta)">。</strong></p><p>维基百科：在数理统计学中，似然函数（英语：likelihood function）是一种关于统计模型中的参数的函数，表示模型参数中的似然性（英语：likelihood）。</p><p>似然概率其实很好理解，就是说我们现在有一堆数据，现在需要构建一组参数对这些数据建模，以使得模型能够尽可能地拟合这些数据。所以我们要做的就是从很多组参数中选出一组使得模型对数据的拟合程度最高，所以也常常说最大似然概率。</p><p>注意“似然”与“概率”意思相近，都是指某种事件发生的可能性，但是在统计学中，“似然”和“概率”又有明确的区分：</p><ul><li><strong>“概率”描述了给定模型参数后，描述结果的合理性，而不涉及任何观察到的数据</strong></li><li><strong>“似然”描述了给定了特定观测值后，描述模型参数是否合理</strong></li></ul><p>举个栗子，抛一枚均匀的硬币，拋20次，问15次拋得正面的可能性有多大？这里的可能性就是“概率”；而拋一枚硬币，拋20次，结果15次正面向上，问其为均匀的可能性？这里的可能性就是“似然”。</p><p><strong>4、先验、后验概率与似然之间的关系</strong></p><ul><li>先验概率：<img src="https://latex.csdn.net/eq?P%20%28%20c%20%29" alt="P ( c )"></li><li>后验概率：<img src="https://latex.csdn.net/eq?P%20%28c%7Cx%29" alt="P (c|x)"></li><li>似然：<img src="https://latex.csdn.net/eq?P%28X%3Dx%7C%5Ctheta%20%3Dc%29" alt="P(X=x|heta =c)"></li></ul><p>存在的关系</p><p><img src="https://i-blog.csdnimg.cn/blog_migrate/c8d6c3d64446feba7d8143d0ecc1d85c.png" alt="img"></p><p>一般而言数据X的分布是已知的，因此</p><p><img src="https://i-blog.csdnimg.cn/blog_migrate/f586b7b52b252b58fe387526bdc4bc08.png" alt="img"></p><p>此外，当参数<img src="https://latex.csdn.net/eq?%5Ctheta" alt="heta">是均匀分布时，后验概率与似然概率成正比，即</p><p><img src="https://i-blog.csdnimg.cn/blog_migrate/f9be3263524a33ed33ca4dec068776b2.png" alt="img"></p><h3 id="极大似然估计">极大似然估计</h3><p>考虑在一个正态分布中随机抽样，抽样出一组值<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>X</mi><mo>=</mo><mo stretchy="false">[</mo><msub><mi>x</mi><mn>1</mn></msub><mo separator="true">,</mo><msub><mi>x</mi><mn>2</mn></msub><mo separator="true">,</mo><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mi mathvariant="normal">.</mi><mo separator="true">,</mo><msub><mi>x</mi><mi>n</mi></msub><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">X=[x_1,x_2,...,x_n]</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">[</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">...</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">n</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">]</span></span></span></span>，这组值独立同分布，都符合某一个正态分布。此时我希望通过这组抽样出来的值（结果）反推出<strong>最有可能抽出这一组结果的原有分布</strong>（参数），这个过程即为似然。</p><p>对于这组抽样出来的值的概率，即为在<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>μ</mi><mo separator="true">,</mo><mi>σ</mi></mrow><annotation encoding="application/x-tex">\mu,\sigma</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.625em;vertical-align:-0.1944em;"></span><span class="mord mathnormal">μ</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal" style="margin-right:0.03588em;">σ</span></span></span></span>的分布<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>N</mi><mo stretchy="false">(</mo><mi>μ</mi><mo separator="true">,</mo><msup><mi>σ</mi><mn>2</mn></msup><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">N(\mu,\sigma^2)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.0641em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.10903em;">N</span><span class="mopen">(</span><span class="mord mathnormal">μ</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.03588em;">σ</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span>下的条件概率。由于他们是相互独立的，那么条件概率即为各个抽样值的概率的连乘：<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>max</mi><mo></mo><mrow><mo fence="true">[</mo><msubsup><mo>∏</mo><mi>i</mi><mi>n</mi></msubsup><mi mathvariant="script">N</mi><mrow><mo fence="true">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>∣</mo><mi>μ</mi><mo separator="true">,</mo><mi>σ</mi><mo fence="true">)</mo></mrow><mo fence="true">]</mo></mrow></mrow><annotation encoding="application/x-tex">\max \left[\prod_{i}^{n} \mathcal{N}\left(x_{i} \mid \mu, \sigma\right)\right]</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.104em;vertical-align:-0.2997em;"></span><span class="mop">max</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top:0em;">[</span><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∏</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8043em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span style="top:-3.2029em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.2997em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathcal" style="margin-right:0.14736em;">N</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top:0em;">(</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mord mathnormal">μ</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathnormal" style="margin-right:0.03588em;">σ</span><span class="mclose delimcenter" style="top:0em;">)</span></span><span class="mclose delimcenter" style="top:0em;">]</span></span></span></span></span>，将其取log即为<strong>对数似然</strong>，所以极大似然估计即为通过求对数似然的最大值，来找到最合适的参数。</p><p>以线性回归举例</p><p>在给定x的情况下，因为y是不确定的，相当于给y增加了一个均值为0的高斯噪声<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>ϵ</mi></mrow><annotation encoding="application/x-tex">\epsilon</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal">ϵ</span></span></span></span>。<img src="/2024/10/07/Revisiting_ML/undefined" alt="image-20241003001600366" style="zoom:50%;" /><img src="/2024/10/07/Revisiting_ML/undefined" alt="image-20241003001615573" style="zoom:50%;" /></p><p>现在y的分布即为一个如图的高斯分布</p><p>在给定散点x，y的情况下，我要求线性回归模型，即求最合适的<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>w</mi></mrow><annotation encoding="application/x-tex">w</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal" style="margin-right:0.02691em;">w</span></span></span></span>。这不就是对其做极大似然估计嘛？</p><p>过程如下：</p><p>有极大似然函数：<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mo>∑</mo><mi>i</mi></msub><mi>ln</mi><mo></mo><mi mathvariant="script">N</mi><mo stretchy="false">(</mo><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi><mi>ω</mi><msub><mi>x</mi><mi>i</mi></msub><mo separator="true">,</mo><msup><mi>σ</mi><mn>2</mn></msup><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\sum_{i} \ln \mathcal{N}(y_i | \omega x_i, \sigma^2)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.1138em;vertical-align:-0.2997em;"></span><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∑</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.162em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.2997em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop">ln</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord mathcal" style="margin-right:0.14736em;">N</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mord mathnormal" style="margin-right:0.03588em;">ω</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.03588em;">σ</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span><span class="mclose">)</span></span></span></span></p><p>概率密度：<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mi>σ</mi><msqrt><mrow><mn>2</mn><mi>π</mi></mrow></msqrt></mrow></mfrac><msup><mi>e</mi><mrow><mo>−</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><msup><mrow><mo fence="true">(</mo><mfrac><mrow><mi>x</mi><mo>−</mo><mi>μ</mi></mrow><mi>σ</mi></mfrac><mo fence="true">)</mo></mrow><mn>2</mn></msup></mrow></msup></mrow><annotation encoding="application/x-tex">f(x) = \frac{1}{\sigma \sqrt{2 \pi}} e^{-\frac{1}{2} \left( \frac{x - \mu}{\sigma} \right)^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.10764em;">f</span><span class="mopen">(</span><span class="mord mathnormal">x</span><span class="mclose">)</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.721em;vertical-align:-0.538em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8451em;"><span style="top:-2.551em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">σ</span><span class="mord sqrt mtight"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.9128em;"><span class="svg-align" style="top:-3em;"><span class="pstrut" style="height:3em;"></span><span class="mord mtight" style="padding-left:0.833em;"><span class="mord mtight">2</span><span class="mord mathnormal mtight" style="margin-right:0.03588em;">π</span></span></span><span style="top:-2.8728em;"><span class="pstrut" style="height:3em;"></span><span class="hide-tail mtight" style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="1.08em" viewBox="0 0 400000 1080" preserveAspectRatio="xMinYMin slice"><path d="M95,702c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429c69,-144,104.5,-217.7,106.5,-221l0 -0c5.3,-9.3,12,-14,20,-14H400000v40H845.2724s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47zM834 80h400000v40h-400000z"/></svg></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.1272em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.538em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord"><span class="mord mathnormal">e</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:1.183em;"><span style="top:-3.4103em;margin-right:0.05em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight"><span class="mopen nulldelimiter sizing reset-size3 size6"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8443em;"><span style="top:-2.656em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span><span style="top:-3.2255em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line mtight" style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.344em;"><span></span></span></span></span></span><span class="mclose nulldelimiter sizing reset-size3 size6"></span></span><span class="minner mtight"><span class="minner mtight"><span class="mopen sizing reset-size3 size6 mtight delimcenter" style="top:0.075em;"><span class="mtight">(</span></span><span class="mord mtight"><span class="mopen nulldelimiter sizing reset-size3 size6"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.879em;"><span style="top:-2.656em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.03588em;">σ</span></span></span></span><span style="top:-3.2255em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line mtight" style="border-bottom-width:0.049em;"></span></span><span style="top:-3.4624em;"><span class="pstrut" style="height:3em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">x</span><span class="mbin mtight">−</span><span class="mord mathnormal mtight">μ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.344em;"><span></span></span></span></span></span><span class="mclose nulldelimiter sizing reset-size3 size6"></span></span><span class="mclose sizing reset-size3 size6 mtight delimcenter" style="top:0.075em;"><span class="mtight">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:1.1039em;"><span style="top:-3.1436em;margin-right:0.0714em;"><span class="pstrut" style="height:2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></p><p>最终将噪声的参数部分去掉，仅留下包含w的部分，即为下图，<strong>与最小二乘法的结果完全一致</strong></p><p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>n</mi></munderover><mo>−</mo><mo stretchy="false">(</mo><msub><mi>y</mi><mi>i</mi></msub><mo>−</mo><mi>ω</mi><msub><mi>x</mi><mi>i</mi></msub><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">\sum_{i=1}^{n} - (y_i - \omega x_i)^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:2.9291em;vertical-align:-1.2777em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:1.2777em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord">−</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.03588em;">y</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1.1141em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.03588em;">ω</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span></span></span></p><h2 id="Model">Model</h2>]]></content>
<categories>
<category> 机器学习理论 </category>
</categories>
<tags>
<tag> ML </tag>
</tags>
</entry>
<entry>
<title>友链</title>
<link href="/link/index.html"/>
<url>/link/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>分类</title>
<link href="/categories/index.html"/>
<url>/categories/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>电影</title>
<link href="/movies/index.html"/>
<url>/movies/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>关于</title>
<link href="/about/index.html"/>
<url>/about/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>音乐</title>
<link href="/music/index.html"/>
<url>/music/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>标签</title>
<link href="/tags/index.html"/>
<url>/tags/index.html</url>
<content type="html"><![CDATA[]]></content>
</entry>
</search>