惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

aimingoo的专栏
aimingoo的专栏
G
Google Developers Blog
B
Blog RSS Feed
A
About on SuperTechFans
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
V
V2EX
Stack Overflow Blog
Stack Overflow Blog
C
Check Point Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
Engineering at Meta
Engineering at Meta
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
博客园 - 司徒正美
D
Docker
F
Fortinet All Blogs
Hugging Face - Blog
Hugging Face - Blog
Last Week in AI
Last Week in AI
H
Help Net Security
WordPress大学
WordPress大学
MyScale Blog
MyScale Blog
博客园 - Franky
人人都是产品经理
人人都是产品经理
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Blog — PlanetScale
Blog — PlanetScale
L
LangChain Blog

Keep the gradient flowing

Policy Gradients Part 1: The REINFORCE Estimator On the Link Between Optimization and Polynomials, Part 6. Optimization Nuggets: Stochastic Polyak Step-size, Part 2 Optimization Nuggets: Stochastic Polyak Step-size On the Convergence of the Unadjusted Langevin Algorithm The Russian Roulette: An Unbiased Estimator of the Limit Notes on the Frank-Wolfe Algorithm, Part III: backtracking line-search On the Link Between Optimization and Polynomials, Part 5 Optimization Nuggets: Implicit Bias of Gradient-based Methods Optimization Nuggets: Exponential Convergence of SGD On the Link Between Optimization and Polynomials, Part 4 On the Link Between Optimization and Polynomials, Part 3 On the Link Between Optimization and Polynomials, Part 2 On the Link Between Polynomials and Optimization, Part 1 How to Evaluate the Logistic Loss and not NaN trying Notes on the Frank-Wolfe Algorithm, Part II: A Primal-dual Analysis Three Operator Splitting Notes on the Frank-Wolfe Algorithm, Part I Optimization inequalities cheatsheet A fully asynchronous variant of the SAGA algorithm Hyperparameter optimization with approximate gradient Lightning v0.1 scikit-learn-contrib, an umbrella for scikit-learn related projects. SAGA algorithm in the lightning library On the consistency of ordinal regression methods Holdout cross-validation generator IPython/Jupyter notebook gallery PyData Paris - April 2015 Data-driven hemodynamic response function estimation Plot memory usage as a function of time
Least squares with equality constrain
Fabian Pedregosa · 2011-04-14 · via Keep the gradient flowing

The following algorithm computes the Least squares solution || Ax - b|| subject to the equality constrain Bx = d. It's a classic algorithm that can be implemented only using a QR decomposition and a least squares solver. This implementation uses numpy and scipy. It makes use of the new linalg.solve_triangular function in scipy 0.9, although degrades to linalg.solve on older versions.

import numpy as np
def lse(A, b, B, d, cond=None):
    """
    Equality-contrained least squares.The following algorithm minimizes
    ||Ax - b|| subject to the constrain Bx = d.

    Parameters
    ----------
    A : array-like, shape=[m, n]
    b : array-like, shape=[m]
    B : array-like, shape=[p, n]
    d : array-like, shape=[p]
    cond : float, optional Cutoff for 'small' singular
    values; used to determine effective rank of A. Singular values smaller
    than \`\`rcond \* largest\_singular\_value\`\` are considered zero.

    Reference
    ---------
    Matrix Computations, Golub & van Loan, algorithm 12.1.2

    Examples
    --------
    >>> A, b = [[0, 2, 3], [1, 3, 4.5]], [1, 1]
    >>> B, d = [[1, 1, 0]], [1]
    >>> lse(A, b, B, d) array([-0.5 , 1.5 , -0.66666667])
    """
    from scipy import linalg
    if not hasattr(linalg, 'solve_triangular'): # compatibility for old scipy
        def solve_triangular(X, y, **kwargs):
            return linalg.solve(X, y)
        else:
            solve_triangular = linalg.solve_triangular
    A, b, B, d = map(np.asanyarray, (A, b, B, d))
    p = B.shape[0]
    Q, R = linalg.qr(B.T)
    y = solve\_triangular(R[:p, :p], d, trans='T', lower=False)
    A = np.dot(A, Q)
    z = linalg.lstsq(A[:, p:], b - np.dot(A[:, :p], y), cond=cond)[0].ravel()
    return np.dot(Q[:, :p], y) + np.dot(Q[:, p:], z)

Update: now scipy has a function qr_multiply which would considerably speed up this code