The problem with this idea is that it complicates the buffer size
calculations in the backends. It also results in situations where
the buffer size is inconsistent which is annoying.

Future commits will be adding support for specifying the size of the
buffer in milliseconds which should offer more flexibility.