Implement to_vec for matrix factorizations #128

Kolaru · 2020-12-25T21:10:56Z

Fixes #114 and should help cleaning the tests in JuliaDiff/ChainRules.jl#306.

willtebbutt · 2020-12-26T10:22:10Z

LGTM. Could you please bump the patch version?

willtebbutt · 2020-12-28T00:07:24Z

I'm happy with this, but could someone else quickly take a look to make sure I've not missed anything obvious. @oxinabox @nickrobinson251 @wesselb ?

oxinabox · 2020-12-28T22:14:45Z

I am on leave.
If noone has looked at it before the 4th of Jan, I will look at it that week.

wesselb

Looks good to me! Thanks for adding this. :) I've left two small comments.

.gitignore

test/to_vec.jl

rkube · 2021-07-08T16:43:38Z

src/to_vec.jl

+    end
+    return x_vec, QRCompact_from_vec
+end
+
 # Non-array data structures


x.T consists of upper triangular blocks: JuliaLang/julia#41363 (comment)
The lower part is undefined on purpose, NaNs can hide there. This line won't work:
x_vec, back = to_vec([x.factors, x.T])

For application purposes it may be preferable to think of derivatives in Q and R, not in factors and T.
So to_vec would unroll Q and R into a matrix. To instantiate a new QR object from updated values, one would need to re-calculate the QR factorization:

function FiniteDifferences.to_vec(x::S) where {S <: Union{LinearAlgebra.QRCompactWYQ, LinearAlgebra.QRCompactWY}} x_vec, x_back = to_vec([x.Q x.R]) function QRCompact_from_vec(v) Q_new, R_new = x_back(v) QR = S(Q_new * R_new) return S(QR.factors, QR.T) end return x_vec, QRCompact_from_vec end

I tried that but unfortunately recomputing the QR factorization is enough to introduce numerical errors, and make the to_vec test fail (since it test for exact equality).

As far as I understand the internal representation used by to_vec should not change the result and it should be fine. If it is not the case, I will need to be smarter, somehow.

As far as I understand the internal representation used by to_vec should not change the result and it should be fine. If it is not the case, I will need to be smarter, somehow.

If it contains NaNs, it definitely does. This should be reusing the logic added in JuliaLang/julia#41363, in particular _triuppers_qr, that way no numerical instabilities will be introduced.

I am not sure to understand. The numerical instability I am referring to are those that prevent are introduced by recomputing the QR decomposition internally:

julia> X = rand(10, 10) ; julia> F = qr(X) ; julia> Xnew = F.Q * F.R ; julia> Xnew == X false julia> Fnew = qr(Xnew) ; julia> Fnew == F false julia> Fnew.Q == F.Q false

The tests require from_vec(to_vec(F)) == F which is not guaranteed there.

To deal with NaNs in T is it preferrable to change the supported version of Julia and use LinearAlgebra._triuppers_qr or to reproduce here part of the logic?

I just meant that by filtering out the undef values, we get both the correct behavior and avoid any numerical issues related to having to refactorize the recalculated matrix.

To deal with NaNs in T is it preferrable to change the supported version of Julia and use LinearAlgebra._triuppers_qr or to reproduce here part of the logic?

It's probably easiest if we just copy that code from there for now. Would be good to mention in a comment that this is taken from Base though and that it can be removed once we drop support for older versions of Julia.

BTW, a good test here would be to check that to_vec(qr(A)) == to_vec(qr(A))

I incorporated that, and added a test of NaN in test_to_vec along the way.

QR must unfortunaely be tested separately for now, because it compares the T matrices that have those arbitrary values.

rkube · 2021-07-08T16:46:16Z

What is the status of this PR? I see that some checks are failing but the logs have expired. Do you need help with this?

oxinabox · 2021-07-08T17:50:03Z

What is the status of this PR?

It is waiting for @Kolaru 's to replay to Wessel's comments.
It seems it's dropped off their radar.

It is reasonable to make another PR to branching off there branch, then rebase and fix the tests, and the comments, and we would be able to merge it.
(They would still show up in the contributors since would keep there commit, if anyone is keeping score.)

Kolaru · 2021-07-13T13:55:46Z

Indeed it kind of went off my radar. I think it may have been partially superseeded by #156. I will try to come back to it ASAP.

Kolaru · 2021-07-20T15:54:50Z

This should be ready now.

simeonschaub

The QRCompactWY case is still broken, see my comment above.

simeonschaub · 2021-07-21T19:49:41Z

src/to_vec.jl

@@ -162,11 +162,24 @@ function to_vec(x::Cholesky)
    return x_vec, Cholesky_from_vec
 end

-function to_vec(x::S) where {S <: Union{LinearAlgebra.QRCompactWYQ, LinearAlgebra.QRCompactWY}}
-    x_vec, back = to_vec([x.factors, x.T])
+function to_vec(x::S) where {U, S <: Union{LinearAlgebra.QRCompactWYQ{U}, LinearAlgebra.QRCompactWY{U}}}


Are you sure this is valid for QRCompactWYQ? If it is, it should at least be tested.

QRCompactWYQ is represented with the same T and factors matrices than QRCompactWY, yes. I added the tests.

Perfect, thanks!

simeonschaub · 2021-07-21T19:51:30Z

src/to_vec.jl

-        factors, T = back(v)
-        return S(factors, T)
+        factors, Tback = back(v)
+        return S(factors, Tback)


Note that using S like this here will not be type stable. Since it's not super performance critical, it might not be worth all the trouble though.

Out of curiosity, why is that so?

See JuliaLang/julia#23618

simeonschaub · 2021-07-21T19:52:14Z

test/to_vec.jl

+                @test F.Q == F.Q
+                @test F.R == F.R


What does this test? Would be good to also add the to_vec(qr(A)) == to_vec(qr(A)) test I proposed above.

This was an oversight from me, it should read F_back.Q == F.Q, to test that back(to_vec(qr(A))) correctly gives back the input.

Your suggested test was on the next line, I have reformulated it to make more explicit what is tested.

test/to_vec.jl

simeonschaub

One more comment, otherwise looks good to me. Thanks for pulling through!

test/to_vec.jl

Co-authored-by: Simeon Schaub <[email protected]>

Kolaru · 2021-07-29T20:27:56Z

I integrated the last suggestion, that should indeed be enough to cover all cases.

simeonschaub · 2021-07-29T21:04:54Z

Cool, thanks for the contribution!

wesselb approved these changes Dec 29, 2020

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

test/to_vec.jl Outdated Show resolved Hide resolved

rkube reviewed Jul 8, 2021

View reviewed changes

oxinabox closed this Jul 8, 2021

oxinabox reopened this Jul 8, 2021

Kolaru added 2 commits July 19, 2021 04:55

Implement to_vec for factorizations

f40b48a

Fix inferrence for svd

7e3010d

Kolaru force-pushed the factorization_to_vec branch from dbcbb7d to 7e3010d Compare July 19, 2021 03:57

simeonschaub requested changes Jul 20, 2021

View reviewed changes

Fix problem from qr(X).T undef values

677d77b

simeonschaub reviewed Jul 21, 2021

View reviewed changes

Fix tests

7bec614

simeonschaub approved these changes Jul 22, 2021

View reviewed changes

test/to_vec.jl Outdated Show resolved Hide resolved

Reduce number of tests

3f85a8c

Co-authored-by: Simeon Schaub <[email protected]>

simeonschaub merged commit 6ee273e into JuliaDiff:master Jul 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement to_vec for matrix factorizations #128

Implement to_vec for matrix factorizations #128

Kolaru commented Dec 25, 2020

willtebbutt commented Dec 26, 2020

willtebbutt commented Dec 28, 2020

oxinabox commented Dec 28, 2020

wesselb left a comment

rkube Jul 8, 2021

Kolaru Jul 19, 2021

simeonschaub Jul 20, 2021 •

edited

Loading

Kolaru Jul 20, 2021

simeonschaub Jul 20, 2021

simeonschaub Jul 20, 2021

Kolaru Jul 20, 2021

rkube commented Jul 8, 2021

oxinabox commented Jul 8, 2021

Kolaru commented Jul 13, 2021

Kolaru commented Jul 20, 2021

simeonschaub left a comment

simeonschaub Jul 21, 2021

Kolaru Jul 22, 2021

simeonschaub Jul 22, 2021

simeonschaub Jul 21, 2021

Kolaru Jul 22, 2021

simeonschaub Jul 22, 2021

simeonschaub Jul 21, 2021

Kolaru Jul 22, 2021

simeonschaub left a comment

Kolaru commented Jul 29, 2021

simeonschaub commented Jul 29, 2021

Implement to_vec for matrix factorizations #128

Implement to_vec for matrix factorizations #128

Conversation

Kolaru commented Dec 25, 2020

willtebbutt commented Dec 26, 2020

willtebbutt commented Dec 28, 2020

oxinabox commented Dec 28, 2020

wesselb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simeonschaub Jul 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rkube commented Jul 8, 2021

oxinabox commented Jul 8, 2021

Kolaru commented Jul 13, 2021

Kolaru commented Jul 20, 2021

simeonschaub left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simeonschaub left a comment

Choose a reason for hiding this comment

Kolaru commented Jul 29, 2021

simeonschaub commented Jul 29, 2021

simeonschaub Jul 20, 2021 •

edited

Loading