Prerequisites

Overview

• This page provides information on the requirements that are required to successfully run Ondat on your container orchestrator. Ensure that you have met the requirements before attempting to do an Ondat deployment.

System Requirements

Hardware

• For Ondat to successfully pool the attached storage disks in a Kubernetes cluster, an Ondat daemonset will be deployed where each pod will run on worker nodes:

Number Of Worker Nodes (Virtual or Bare-metal)

• Each worker node in cluster where Ondat will be deployed, should meet the following hardware requirements (minimum hardware requirements are recommended for evaluation purposes only):

Node Hardware Requirements

💡 The requirements above will also be dependant on the stateful applications that will be running in your cluster, therefore, it is recommended to review your stateful applications resources requirements when factoring in requirements for Ondat.

Architecture

Linux Kernel

• Below is the list of supported Linux kernel versions and the maximum number of active volumes that can run per node in an Ondat cluster:

Linux Asynchronous I/O (AIO)

• When the Ondat Data plane component is operational, Ondat leverages Linux Asynchronous I/O (AIO) contexts to initiate a number of I/O requests without having to block or wait for any to complete to process the next task - thus allowing you to boost performance for applications that are able to simultaneously overlap processing and I/O.
• The Linux kernel uses the /proc/sys/fs/aio-max-nr virtual file to set the value of the maximum number of allowable AIO concurrent requests. The /proc/sys/fs/aio-nr virtual file also provides information on the current number of asynchronous requests on the node.
• Ondat requires 4 AIO contexts per deployed volume, whether it is a master or replica volume.
  • Trying to provision additional volumes once the aio-nr value reaches the aio-max-nr, will cause a io_setup system call to fail and return a EAGAIN error, meaning that the resource becomes temporarily unavailable.
• Therefore, if your nodes aio-max-nr kernel parameter value is less than 1048576, a strong recommendation will be to set a new value in your /etc/sysctl.conf or through a custom configuration file in /etc/sysctl.d/ to increase the number of AIO contexts that can be requested:

Required Kernel Modules

• Ondat requires the open source LinuxIO (LIO) SCSI target engine that has been built into the Linux kernel mainline, since kernel version 2.6.38.
• Below are the following kernel modules that are required to be loaded on your worker nodes operating system where Ondat will be deployed:

⚠️ You cannot concurrently run Ondat and other applications that utilise TCMU. Doing so will result in data loss and corruption. On startup, Ondat will automatically detect if there are other applications that utilise TCMU to avoid data loss and corruption.

• Once the kernel modules are available, during the deployment process - Ondat will run an init container which conducts preflight checks to ensure that the kernel modules are loaded and ready to be used by the Ondat daemonset that runs on each worker node.

Supported Filesystems

Host/Node Filesystems

• Ondat uses the /var/lib/storageos path on each node as a base directory for storing its configuration file and and blob files related to the persistent volumes running in the cluster. Below are the list of node filesystem types that are supported by Ondat:

Ondat Persistent Volume Filesystems

• Ondat provides a block device on which a filesystem can be created. The creation of that filesystem is either handled by Ondat or by Kubernetes, which affects what filesystems can be created. When using the Ondat CSI (Container Storage Interface) driver, it will be responsible for running mkfs against the block device that the pod will mount. Below are the list of filesystems that Ondat is able to create for the persistent volumes:

💡 If the listed filesystems above are different from the one you are using, you can raise a feature request for it by contacting us through through our Community Slack or through the Support Portal.

Operating System

• Below is the following list of current Linux distributions (non-EOL) that are supported by Ondat:

💡 As a general rule of thumb, Ondat is agnostic and will run on any Linux distribution as long as the required kernel modules prerequisites are available and can be successfully loaded. In most modern Linux distributions, the key kernel modules are distributed as part of the default Linux kernel packages. In some older distributions, the kernel modules were part of the kernel extras package that needed to be installed separately.

💡 If you need help with a specific issue with one of the listed distributions we support, raise an issue up on GitHub or reach out to us through our Community Slack.

Container Orchestrators

Supported Kubernetes Distributions

Container Runtimes

Mount Propagation

• It is required that mount propagation is enabled for the container orchestrator where Ondat will be deployed in. Mount propagation is enabled by default in the newer release versions of Kubernetes and OpenShift.

💡 If your container orchestrator has mount propagation disabled, and you are looking for guidance on how to enable it, review the Kubernetes and OpenShift documentation on mount propagation for more information.

Process ID (PID) Limits

• Ondat pods that are running in a Kubernetes or OpenShift cluster are part of a PID cgroup that may limit the maximum number of PIDs that all containers in the PID cgroup slice can spawn.
• As the Linux kernel assigns a PID to processes and Light-Weight Processes (LWPs), a low PID limit can be easily reached without breaching any other resource limits, which could then potentially cause instability issues as Ondat won’t be able to spawn new processes.
• Depending on the container orchestrator you are using, the PID limit is set by the distribution or the container runtime being used. Therefore, ensure that the PID limit is set to at least 32768 for Ondat pods in your cluster:

💡 To check if the PID limit of the PID cgroup slice that the Ondat pods runs in is set to at least 32768, Ondat will run an init container which conducts preflight checks to ensure that the correct limit is set. If your cluster defaults to a low PID limit, it is recommended to follow your distribution’s documentation on how to configure and set a higher limit. Review the Kubernetes - Configure Pod PID Limits and OpenShift - Creating a ContainerRuntimeConfig CR to Edit CRI-O Parameters documentation for guidance.

⚠️ OpenShift uses CRI-O as the container runtime, which has a PID limit that defaults to 1024. It is strongly recommended that you raise the PID limit to at least 32768 to avoid instability issues.

Networking

Firewall Rules

For Ondat components to be able to successfully communicate with each other in a cluster, ensure that you add the following firewall rules or web proxy exceptions between nodes. Ensure that these ports are only accessible inside the scope of a cluster in your environment.

💡 Ondat also uses ephemeral ports to dial-out to the ports listed below to other Ondat nodes in the cluster. For this reason, egress/outgoing traffic flows to other nodes is allowed.

💡 Ondat does not expose any service externally by default, ie each service is published with the ClusterIP service type which makes services only reachable from within a cluster.

IPv6 Availability

• Although Ondat does not require IPv6 addressing or routing to be configured in order to run successfully, specific Ondat components do require to be able to listen on a standard dual-stack AF_INET6 socket type and accept client requests from either IPv4 or IPv6 nodes.
• The IPv6 address family must be supported in your cluster so that Ondat can leverage the AF_INET6 socket type.

Hardware Clock Synchronisation

• It is recommended that the hardware/system clock for the nodes in your cluster are correctly synchronised to use reliable Network Time Protocol (NTP) servers.
• While Ondat’s distributed consensus algorithm does not require synchronised hardware clocks, it is useful for troubleshooting through logs by being able to easily correlate logs across multiple Ondat nodes and prevent clock drift in your cluster.