Based on an initial read, this looks like a significant breakthrough to me.
I wonder if the techniques developed by the authors could make it feasible to take other piecewise linear/constant algorithms (which until now have been considered "non-differentiable" for practical purposes) and turn them into differentiable algorithms.
itβs not completely unprecedented because there were other ways of getting equivalent results before (the Sinkhorn based optimal transport approach cited, for one), which have been used for all kinds of interesting tasks. the contribution is that it does so more efficiently.
I wonder if the techniques developed by the authors could make it feasible to take other piecewise linear/constant algorithms (which until now have been considered "non-differentiable" for practical purposes) and turn them into differentiable algorithms.
Think beyond sorting and ranking.