Prepare Linux for Edge Volumes

The article describes how to prepare Linux for Edge Volumes using Azure Kubernetes Service (AKS) enabled by Azure Arc, Edge Essentials, or Ubuntu.

These instructions assume that you already have an Arc-enabled Kubernetes cluster. To connect an existing Kubernetes cluster to Azure Arc, see these instructions.

If you want to use Azure Container Storage enabled by Azure Arc with Azure IoT Operations, follow the instructions to create a cluster for Azure IoT Operations.

You must also install a cert-manager as described in Install Edge Volumes.

Prerequisites

The following table lists the prerequisites for Azure Container Storage enabled by Azure Arc:

Requirement	Single-node/Two-node cluster	Multi-node cluster
Operating System
Kernel version	5.15 and above (minimum supported)	5.15 and above (minimum supported)
NFSv4.2 support	Enabled in the kernel	Enabled in the kernel
Hardware Requirements
CPU	4 CPUs minimum	8 CPUs minimum
RAM	16 GB minimum	32 GB recommended (16 GB minimum)
VM recommendation	Standard_D8ds_v5 or equivalent	Standard_D8as_v5 or equivalent
Storage Requirements
Storage provisioner	Local-path storage class required	N/A
Reserved system volume	1 GB per Edge Volume	N/A
System Configuration
sysctl configuration	`fs.inotify.max_user_instances >= 1024`	`fs.inotify.max_user_instances >= 1024`
NVME over TCP kernel module	Not required	Required
Hugepages configuration	Not required	Set to 512

Additional considerations

Kernel compatibility: Known issues exist with kernel versions 6.4 and 6.2.
Regional availability: Azure Container Storage enabled by Azure Arc is only available in: East US, East US 2, West US, West US 2, West US 3, North Europe, West Europe
Ensure that the required disk storage is available and properly mounted
For multi-node clusters, 32 GB RAM serves as a buffer; however, 16 GB RAM should suffice. Edge Essentials configurations require 8 CPUs with 10 GB RAM per node, making 16 GB RAM the minimum requirement.

Single-node clusters

A single-node cluster is commonly used for development or testing purposes due to its simplicity in setup and minimal resource requirements. These clusters offer a lightweight and straightforward environment for developers to experiment with Kubernetes without the complexity of a multi-node setup. Additionally, in situations where resources such as CPU, memory, and storage are limited, a single-node cluster is more practical. Its ease of setup and minimal resource requirements make it a suitable choice in resource-constrained environments.

However, single-node clusters come with limitations, including their lack of high availability, fault tolerance, scalability, and decreased performance.

Multi-node clusters

A multi-node Kubernetes configuration is typically used for production, staging, or large-scale scenarios because of features such as high availability, fault tolerance, scalability, and performance. A multi-node cluster also introduces challenges and trade-offs, including complexity, overhead, cost, and efficiency considerations. For example, setting up and maintaining a multi-node cluster requires extra knowledge, skills, tools, and resources (network, storage, compute). The cluster must handle coordination and communication among nodes, leading to potential latency and errors. Additionally, running a multi-node cluster is more resource-intensive and is costlier than a single-node cluster. Optimization of resource usage among nodes is crucial for maintaining cluster and application efficiency and performance.

In summary, a single-node Kubernetes cluster might be suitable for development, testing, and resource-constrained environments. A multi-node cluster is more appropriate for production deployments, high availability, scalability, and scenarios in which distributed applications are a requirement. This choice ultimately depends on your specific needs and goals for your deployment.

Azure Container Storage enabled by Azure Arc doesn't natively support multi-node deployments. If you need a multi-node Kubernetes cluster, consider using the Azure Local service.

Minimum storage requirements

When you provision an Edge Volume, a static system volume of 1 GB is reserved for the system itself to run; this is in addition to the provisioned size chosen by the user in the Custom Resource Definition option.

When you use Azure Local for fault tolerance, Azure Local offers a ReadWriteOnce file system, so you can allocate resources to it in the same way you would for a single node cluster; no other constraints apply.

Prepare Linux for Edge Volumes using a single-node or two-node cluster

This section describes how to prepare Linux using a single-node or two-node cluster, and assumes you fulfilled the prerequisites. Select the appropriate tab for your Linux distribution and Kubernetes platform.

Prepare Linux with AKS enabled by Azure Arc

If you run a single-node or two-node cluster on Linux with AKS enabled by Azure Arc, you don't need to perform any additional steps.

Prepare Linux with AKS Edge Essentials

This section describes how to prepare Linux with AKS Edge Essentials if you run a single-node or two-node cluster.

For Edge Essentials to support Azure IoT Operations and Azure Container Storage enabled by Azure Arc, the Kubernetes hosts must be modified to support more memory. You can also increase vCPU and disk allocations at this time if you anticipate requiring additional resources for your Kubernetes uses.

Start by following the How-To guide here. The QuickStart uses the default configuration and should be avoided.

Following Step 1: single machine configuration parameters, you have a file in your working directory called aksedge-config.json. Open this file in Notepad or another text editor:
```
"SchemaVersion": "1.11",
"Version": "1.0",
"DeploymentType": "SingleMachineCluster",
"Init": {
    "ServiceIPRangeSize": 0
},
"Machines": [
{
    "LinuxNode": {
        "CpuCount": 4,
        "MemoryInMB": 4096,
        "DataSizeInGB": 10,
    }
}
]
```
Increase MemoryInMB to at least 16384 and DataSizeInGB to 40G. Set ServiceIPRangeSize to 15. If you intend to run many PODs, you can increase the CpuCount as well. For example:
```
"Init": {
    "ServiceIPRangeSize": 15
   },
"Machines": [
{
    "LinuxNode": {
        "CpuCount": 4,
        "MemoryInMB": 16384,
        "DataSizeInGB": 40,
    }
}
]
```
Continue with the remaining steps starting with create a single machine cluster. Next, connect your AKS Edge Essentials cluster to Arc.
Check for and install Local Path Provisioner storage if it's not already installed. Check if the local-path storage class is already available on your node by running the following cmdlet:
```
kubectl get StorageClass
```
If the local-path storage class is not available, run the following command:
```
kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/storage/local-path-provisioner/local-path-storage.yaml
```
Note

Local-Path-Provisioner and Busybox images are not maintained by Microsoft and are pulled from the Rancher Labs repository. Local-Path-Provisioner and BusyBox are only available as a Linux container image.

If everything is correctly configured, you should see the following output:
```
NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  21h
```
If you have multiple disks and want to redirect the path, use:
```
kubectl edit configmap -n kube-system local-path-config
```

Run the following command to determine if you set fs.inotify.max_user_instances to 1024:

Invoke-AksEdgeNodeCommand -NodeType "Linux" -Command "sysctl fs.inotify.max_user_instances

After you run this command, if it outputs less than 1024, run the following command to increase the maximum number of files:

Invoke-AksEdgeNodeCommand -NodeType "Linux" -Command "echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.conf && sudo sysctl -p"

Prepare Linux with Ubuntu

This section describes how to prepare Linux with Ubuntu if you run a single-node or two-node cluster.

Run the following command to determine if you set fs.inotify.max_user_instances to 1024:
```
sysctl fs.inotify.max_user_instances
```
After you run this command, if it outputs less than 1024, run the following command to increase the maximum number of files and reload the sysctl settings:
```
echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
```

Prepare Linux with other platforms

The available platform options are production-like environments that Microsoft validated. These platforms aren't necessarily the only environments on which Azure Container Storage enabled by Azure Arc can run. Azure Container Storage can run on any Arc-enabled Kubernetes cluster that meets the Azure Arc-enabled Kubernetes system requirements. If you're running on an environment not listed, here are a few suggestions to increase the likelihood of a successful installation:

Run the following commands to increase the user watch and instance limits:

echo fs.inotify.max_user_instances=8192 | sudo tee -a /etc/sysctl.conf
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Run the following commands to increase the file descriptor limit for better performance:
```
echo fs.file-max = 100000 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
```

Run the following command to install the local path provisioner:

kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/storage/local-path-provisioner/local-path-storage.yaml

Next steps

Feedback

Was this page helpful?

Last updated on 2026-02-05