<p dir="auto">Hi,<br>
I am trying to optimize a function using fmin_bfgs, and logistic regression algorithm. There's no problem in convergence using the older version (scipy 0.10). However when using newer version 0.15 the optimizer crashes with warning: "Desired error not necessarily achieved due to precision loss”.<br>
I compared optimize.py code of older, and newer versions, and noticed initial stepsize is updated from <strong>old_old_fval = old_fval + 5000</strong> to <strong>old_old_fval = None</strong>. (<a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/scipy/scipy/commit/81474eb34d99bba0b2908e35ba832ca735e40154/hovercard" href="https://github.com/scipy/scipy/commit/81474eb34d99bba0b2908e35ba832ca735e40154"><tt>81474eb</tt></a>)<br>
Then I tried changing the stepsize back to <strong>old_old_fval = old_fval + 5000</strong>, and this time fmin_bfgs worked perfectly. I am guessing there might be an issue with the initial stepsize (<strong>old_old_fval = None</strong>) for fmin_bfgs, and this stepsize may not be flexible for all gradient sizes. I see a similar issue still opened here: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="32281799" data-permission-text="Title is private" data-url="https://github.com/scipy/scipy/issues/3581" data-hovercard-type="issue" data-hovercard-url="/scipy/scipy/issues/3581/hovercard" href="https://github.com/scipy/scipy/issues/3581">#3581</a>.</p>
<p dir="auto"><em>Original ticket <a href="http://projects.scipy.org/scipy/ticket/1473" rel="nofollow">http://projects.scipy.org/scipy/ticket/1473</a> on 2011-07-07 by trac user collinstocks, assigned to unknown.</em></p>
<p dir="auto">The qr function in the linear algebra library is lacking some of the functionality that exists in the matlab function of the same name. While the purpose of scipy is certainly not duplicate the functionality of matlab, qr decomposition with pivoting is a rather important feature, as it can be used, for example, to estimate the rank of a matrix.</p>
<p dir="auto">I am not familiar enough with the scipy code to create a new lapack wrapper, but I have implemented the functionality described above by using cvxopt.lapack. Clearly, a production version should wrap lapack.geqp3 properly.</p>
<p dir="auto">Unfortunately, the reference implementation I made is based off linalg.qr from scipy version 0.7.1. I can probably do the same for the current version without too much difficulty.</p>