惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

J
Java Code Geeks
Hugging Face - Blog
Hugging Face - Blog
博客园_首页
爱范儿
爱范儿
罗磊的独立博客
美团技术团队
Jina AI
Jina AI
量子位
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
酷 壳 – CoolShell
酷 壳 – CoolShell
有赞技术团队
有赞技术团队
V
V2EX
阮一峰的网络日志
阮一峰的网络日志
小众软件
小众软件
IT之家
IT之家
雷峰网
雷峰网
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
博客园 - 司徒正美
大猫的无限游戏
大猫的无限游戏
博客园 - 聂微东
月光博客
月光博客
人人都是产品经理
人人都是产品经理
博客园 - 三生石上(FineUI控件)

Keep the gradient flowing

Policy Gradients Part 1: The REINFORCE Estimator On the Link Between Optimization and Polynomials, Part 6. Optimization Nuggets: Stochastic Polyak Step-size, Part 2 Optimization Nuggets: Stochastic Polyak Step-size On the Convergence of the Unadjusted Langevin Algorithm The Russian Roulette: An Unbiased Estimator of the Limit Notes on the Frank-Wolfe Algorithm, Part III: backtracking line-search On the Link Between Optimization and Polynomials, Part 5 Optimization Nuggets: Implicit Bias of Gradient-based Methods Optimization Nuggets: Exponential Convergence of SGD On the Link Between Optimization and Polynomials, Part 4 On the Link Between Optimization and Polynomials, Part 3 On the Link Between Optimization and Polynomials, Part 2 On the Link Between Polynomials and Optimization, Part 1 How to Evaluate the Logistic Loss and not NaN trying Notes on the Frank-Wolfe Algorithm, Part II: A Primal-dual Analysis Three Operator Splitting Notes on the Frank-Wolfe Algorithm, Part I Optimization inequalities cheatsheet A fully asynchronous variant of the SAGA algorithm Hyperparameter optimization with approximate gradient Lightning v0.1 scikit-learn-contrib, an umbrella for scikit-learn related projects. SAGA algorithm in the lightning library On the consistency of ordinal regression methods Holdout cross-validation generator IPython/Jupyter notebook gallery PyData Paris - April 2015 Data-driven hemodynamic response function estimation Plot memory usage as a function of time
qr_multiply function in scipy.linalg
Fabian Pedregosa · 2011-10-14 · via Keep the gradient flowing

In scipy's development version there's a new function closely related to the QR-decomposition of a matrix and to the least-squares solution of a linear system. What this function does is to compute the QR-decomposition of a matrix and then multiply the resulting orthogonal factor by another arbitrary matrix. In pseudocode:

def qr_multiply(X, Y):
    Q, R = qr(X)
    return dot(Q.T, Y)

but unlike this naive implementation, qr_multiply is able to do all this without explicitly computing the orthogonal Q matrix, resulting both in memory and time saving. In the following picture I measured the memory consumption as a function of time of running this computation on a 1.000 x 1.000 matrix X and a vector Y (full code can be found here):

It can be seen that not only qr_multiply is almost twice as fast as the naive approach, but also that the memory consumption is significantly reduced, since the orthogonal factor is never explicitly computed. Credit for implementing the qr_multiply function goes to Martin Teichmann.