Because on the prestige, you sit back, relax and ponder if you really care about 20 seconds.
It could be many things you could be pondering, tyres and taking in the bumps that would slow you down, how easy they are to roll or just simply pure cadence to wheel rotation at the tyre.
Assuming Cadence (same pedalling rhythm) is exactly the same and the gear ratio are identical, but the smokes create a slightly smaller working diameter, at a 15mins 2 mile, that's about 5 mm less on the tyre to get a 20 second loss (I think , somebody can check the mathematics of this)