Guard the access of h.size as this is now a data race.

Fixes #2571
Signed-off-by: Miek Gieben <miek@miek.nl>