As part of the dataplane operation, Ondat uses Linux AIO (Asynchronous Input Output) contexts to serve I/O requests without blocking. Ondat requires 4 AIO contexts per deployment (i.e. an Ondat volume deployment, whether master or replica).
Max AIO prerequisite.
By default there is a maximum number of AIO contexts that can be allocated at once.
The current and maximum number of AIO requests is visible in the virtual
The default context limit has been set at 2^16 or 65536. This figure may vary
so please check your
io_setup syscall will fail with
EAGAIN. For more information please see the Linux kernel docs
Why is this relevant?
As Ondat requires 4 AIO contexts per deployed volume, there is a limit to
the number of volumes that can be deployed per node. Trying to provision
additional deployments once the
aio-max-nr has been reached will fail as the
kernel will be unable to create enough new AIO contexts.
Increasing your AIO context cap.
If your nodes
aio-max-nr is set too low you can either provision additional
nodes to reduce the number of deployments per node, or increase the
You can do this by editing your
/etc/sysctl.conf file with the following
fs.aio-max-nr = 1048576
To activate the new settings, run the following command:
$ sysctl -p /etc/sysctl.conf