Could you share a bit more details on your config?

You can easily test a gradient using only the forward pass by doing f(x+h) ~ f(x) + dot(g, h) for a random h

I just skimmed the article and didn't watch the video, but the bit about backpropagation is just wrong. Backpropagation doesn't compute an inverse of the jacobian, it computes its transpose. (although a similar idea to backpropagation could possibly be used to compute an inverse of several reversible layers, but that's not typically how neural networks work)

I'm willing to bet the actual reason the transition went so smoothly and the library doesn't pay that much in per-article fees is because everybody just used sci-hub.