Implement all the algorithms introduced by Statistical Learning Methods, Li Hang.
- Complete. All the algorithms introduced by this book are implemented, including
- kNN powered by kd-tree.
- max entropy model. I cannot find any other repo that implements this algorithm.
- linear chain conditional random field. I cannot find this model in any other similar repo.
- HMM powered by baum-welch. Most repos only provide with HMM trained by counting.
- Detailed. All the algorithms are implemented thoroughly. I try my best not to skip any detail. For example,
- about how to select the best one of trimmed CART by cross-validation, I asked Dr. Li Hang by e-mail and got detailed answer. Thanks a lot to Dr. Li Hang for his patiance and kindness.
- Matrix calculation. Strip off
forloops. Implement most of the algorithms with matrix calculation supported bynumpy. - Extensible. It is easy to fit the codes with new datasets because all the algorithms are controllable through parameters, to a large extent.
- Examples Each algorithm comes with some examples. Just run the model file and you will see the examples. If you have better examples for others to understand the model, please feel free to start a PR.
- Python3
- numpy
- matplotlib
- rich
Just run any single file located in each chapter. You will see examples of the algorithm.
李航博士《统计学习方法》一书的硬核 Python 实现。
GitHub 上有许多实现《统计学习方法》的仓库。本仓库与它们的不同之处在于：
- 完整性。实现了所有模型。包括
- KD 树支持的 KNN 模型。
- 最大熵模型。我没有找到其他任何一个仓库实现了该算法。
- 线性链条件随机场。我同样没有找到其他任何一个仓库实现了该算法。这个模型花费了我一个月的时间去理解和实现。
- Baum-Welch 算法支持的 HMM 算法。大多数仓库实现的 HMM 算法都是简单的计数模型。
- 细节。所有的算法我都在尽力完全实现。比如说
- 有关如何用交叉验证法选取剪枝的 CART 树，我特意邮件询问了李航博士并得到了耐心的解答。在此非常感谢李航博士的支持！
- 矩阵运算。我不喜欢用循环。你可以看到本仓库中的算法使用了大量的矩阵运算来避免使用循环。
- 可扩展性。其他仓库的算法可能会在可扩展性上偷懒。比如 GMM 模型可能只实现了两个聚类的简单版本用于演示。而本仓库中的算法尽量将所有可调节部分作为模型参数，以供自由修改使用。
- 示例。每个算法都加上了我认为会增强读者对算法理解的例子。当然我认为这部分目前还是不太完善的。如果你对如何举例有更好的见解，欢迎给我提 PR。
- Python3
- numpy
- matplotlib
- rich
直接使用 Python 运行任意一个文件夹内的模型文件，你就可以看到算法示例了。