Thank you for pointing that out! I did misinterpret that part of the abstract!
On a perhaps related note, it seem a bit surprising to me because when I first started with neural networks about a year ago, I tried to shortcut hyperparameter search by reusing weights and noticed that independently trained models with the same hyperparameters would produce model with different performance. I naively assumed that such correlation is something I don't want and it was something that everyone knows about so I just moved on.
I've noticed this too. I've got a paper coming up on the arxiv soon that discusses this phenomenon, and structured random architecture search, in the context of semantic segmentation networks.
On a perhaps related note, it seem a bit surprising to me because when I first started with neural networks about a year ago, I tried to shortcut hyperparameter search by reusing weights and noticed that independently trained models with the same hyperparameters would produce model with different performance. I naively assumed that such correlation is something I don't want and it was something that everyone knows about so I just moved on.
Edit: typo (pointed --> pointing)