It was a bit more than that. He implied that the performance differences were trivial, so it wasn't worth the trouble.

I think, unfortunately, Linus' argument amounts to demanding that API's conform to how they are used rather than how they are specified, which leads to O_PONIES type problems.