The minimum hardware and infrastructure required to run ArcGIS Enterprise on Kubernetes 11.1 are described below. These requirements also apply when deploying in a disconnected environment.
Supported environments
System requirements and specifications apply across all supported environments except where noted. For this release, the following environments are supported:
- On-premises data center
- Red Hat OpenShift Container Platform
- Managed Kubernetes services on the cloud
- Amazon Elastic Kubernetes Service (EKS)
- Google Kubernetes Engine (GKE)
- Microsoft Azure Kubernetes Service (AKS)
It is strongly recommended that you disable auto-upgrade in your Kubernetes cluster. When your cluster is enabled with auto-upgrade, the nodes will automatically be updated with the latest version of Kubernetes and these future versions may not yet be supported for ArcGIS Enterprise.
Caution:
When upgrading your environment, you must upgrade ArcGIS Enterprise on Kubernetes before upgrading Kubernetes to a supported version.The following versions for each environment have been tested and are supported:
| Supported environment | Supported Kubernetes version |
|---|---|
Managed Kubernetes services on the cloud (AKS, EKS, GKE) | 1.23 - 1.25 |
| Red Hat OpenShift Container Platform | 4.10 - 4.12 |
Note:
ArcGIS Enterprise on Kubernetes is only supported on CPUs that adhere to the x86_64 architecture (64 bit). Worker nodes must be Linux-based.
Container registry
Container images for ArcGIS Enterprise on Kubernetes are accessible from a private Docker Hub organization. Esri will provide access to this organization's repositories to those who are deploying ArcGIS Enterprise on Kubernetes. When deploying in a disconnected environment, you will need to push images from the private Docker Hub organization to your organization's container registry that is accessible from your cluster.
Obtain an Esri license
To authorize your ArcGIS Enterprise organization during deployment, you need an ArcGIS Enterprise on Kubernetes license file in JSON format (.json file). To obtain this license file, visit My Esri with privileges to take licensing action.
CPU and memory
ArcGIS Enterprise on Kubernetes is deployed with one of three architecture profiles. Recommendations for resource (CPU and memory) requests and limits and overall compute requirements vary based on the selected profile. Recommendations for each profile are provided below.
The following are the minimum node requirements for each architecture profile. It is recommended that each worker/agent node have a minimum of 8 CPU and 32 GiB of memory.
| Architecture profile | Minimum worker/agent nodes | Total minimum CPU | Total minimum GiB |
|---|---|---|---|
Enhanced availability | 5 | 40 | 160 |
Standard availability | 4 | 32 | 128 |
Development | 3 | 24 | 96 |
The pods in the ArcGIS Enterprise on Kubernetes deployment are distributed across the worker nodes in the cluster. When scaling the deployment or adding another ArcGIS Enterprise deployment to the cluster, you need to provision hardware accordingly. This may require an increase in the default maximum number of pods per node. The number of pods that are initially created varies with each architecture profile. As you scale horizontally or add new functionality, the number of pods increases.
Security
The security requirements for ArcGIS Enterprise on Kubernetes are described below.
Role-based access control
Role-based access control (RBAC) must be enabled on the Kubernetes cluster. To deploy ArcGIS Enterprise on Kubernetes, you do not need cluster-admin privileges. If you do not have cluster-admin privileges, the user must have minimum namespace administrative privileges. You can assign the user a default ClusterRole by creating a RoleBinding in the namespace.
Register a data folder
To publish items using file-based data, like items published from a file geodatabase, you will need to place the data in an NFS shared location. This NFS share must be registered with the ArcGIS Enterprise organization to avoid copying the data to the server while publishing. To register the shared folder successfully, you will need to grant file level read permissions to others. You can secure the NFS share at the network or infrastructure level by allowing network access to the pod IP range.
Network
Network requirements include a FQDN and load balancer. Details for each are provided below.
Fully qualified domain name
ArcGIS Enterprise on Kubernetes requires a FQDN (for example, map.company.com). You can use an existing domain name system (DNS) to create one or use a Cloud DNS service such as Amazon Route 53. You can create the DNS record after deployment; however, you must provide its value during deployment. At this release, the FQDN cannot be modified after deployment.
Load balancer
A load balancer is required to direct traffic across each worker node. You can provision the following load balancers from the deployment script without manual configuration:
- Azure Load Balancer (public or internal)—A preprovisioned static public IP address and DNS label can be specified in the deployment script.
- AWS Network
Load Balancer (internet-facing or internal)—Other load balancing
services can be used; however, they must
be configured manually
with each cluster
node.
Note:
The AWS Load Balancer Controller add-on is required to create Network Load Balancers in either a public or private subnet.
- Google Cloud Platform TCP Load Balancer (internet-facing or internal)—A preprovisioned static public IP address can be specified in the deployment script.
In an OpenShift Container Platform, routes can be configured when pointing to the ingress controller service.
You can use a self-managed load balancer pointing to the worker nodes on the ingress controller service's NodePort. For details, see the deployment guide's parameter description for load balancer.
When using a self-managed load balancer or reverse proxy such as NGINX, specify the following connection: proxy_set_header X-Forwarded-Host $host;. This header is needed to ensure that traffic is properly routed to your ArcGIS Enterprise organization's URL.
Note:
ArcGIS Enterprise does not support SSL offloading through a reverse proxy/load balancer. If your configuration uses a reverse proxy, it must redirect to either the ArcGIS Web Adaptor or directly to the organization over HTTPS.
IP requirements
Planning your cluster network in advance is essential for ensuring a successful deployment, appropriate scaling requirements, and the ability to upgrade. ArcGIS Enterprise on Kubernetes initially deploys 47-66 pods, depending on the architecture profile. The number of pods will increase as additional capabilities are added, the deployment is scaled, and during the upgrading process.
Each pod is assigned a unique IP address, and depending on the cluster network configuration, pods can either get their IP addresses from a logically different address space from that of the host network (an overlay network) or from the host subnet. For example, if you configure your cluster to use Kubenet in Azure (default), pods will receive an IP address from a logically different address space and will be able to reach Azure resources using NAT.
Kubernetes supports Container Network Interface (CNI) and platforms like AKS and EKS, which use platform specific CNI plugins for cluster networking. For example, EKS clusters use Virtual Private Cloud (VPC) CNI by default. If the cluster is configured with a CNI plugin, pods will receive IP addresses from the host subnet and a corresponding pool of IPs available in the VPC/VNet.
If you do not have a sufficient number of IPs available in the host subnets, the deployment will either fail, or you will not be able to scale the deployment. For example, if an EKS cluster is configured with 2 subnets each and a /26 IPv4 address prefix (64 available IPv4 addresses each), then there cannot be more than 126 IP addresses available for the pods. While you may be able to deploy ArcGIS Enterprise on Kubernetes in this cluster you will not be able to scale the deployment to have 80 feature service pods, as this scaling requirement will exceed the number of IP addresses available.
System storage
ArcGIS Enterprise on Kubernetes requires persistent volumes (PVs) for system storage, which can be provisioned dynamically via a storage class or statically by an administrator prior to creating the organization. Stateful workloads of ArcGIS Enterprise include relational database management systems and NoSQL databases. It is recommended that you provision PVs on block storage devices that provide low latency, such as EBS volumes when using EKS, Azure Disks when using AKS, Persistent Disks when using GKE, and vSphereVolume or Longhorn volumes when deploying on-premises.
Because these PVs store your organization's data and settings, you must protect them using restrictive security policies. For PVs based on network file storage, such as NFS, Azure Files, and GlusterFS, ensure that the permissions are set to prevent unauthorized access. For block storage, such as EBS, Azure Disk, vSphereVolume, and Longhorn volumes, ensure that access to the storage volumes is restricted to only those users who need it.
The following are descriptions of storage volumes and their intended purpose:
Note:
Persistent volume requirements are stated for 11.1 and may differ from prior versions.
- In-memory—Stores temporary system resources.
- Item packages—Stores large uploads and packages to support publishing workflows.
- Object—Stores uploaded and saved content, hosted tile and image layer caches, and geoprocessing output.
- Queue—Stores asynchronous geoprocessing jobs.
- Relational—Stores hosted feature data and administrative aspects such as customization and configuration settings. Two are required for deployment.
- Spatiotemporal and index—Stores logs and indexes as well as hosted feature data.
- Usage metric data—Stores GIS service usage data.
Consider the storage requirements for your organization and define the size for each PV accordingly.
Static PVs
If you're provisioning static PVs prior to deployment, the specifications and labels described below are recommended.
The number of PVs required for each architecture profile are provided.
| Volume | Enhanced availability profile | Standard availability profile | Development profile |
|---|---|---|---|
in-memory-volume | 1 | 1 | 1 |
item-packages-volume | 2 | 2 | 1 |
object-volume | 8 | 3 | 1 |
queue-volume | 2 | 2 | 1 |
relational-volume | 2 | 2 | 2 |
spatiotemporal-and-index-volume | 5 | 3 | 1 |
usage-metric-volume | 1 | 1 | 1 |
If you are configuring your organization to use existing static PVs, the following values of each persistent volume claim (PVC) are used to bind it to the PVs via label selection: size, access mode, and labels. These values can be customized in the setup wizard or the configuration properties file to match the preprovisioned PVs to each stateful-set specification.
| Volume | Size in GiB (minimum) | Access mode | Label |
|---|---|---|---|
in-memory-volume | 16 | ReadWriteOnce | arcgis/tier=storage, arcgis/app=ignite |
item-packages-volume | 16 | ReadWriteOnce | arcgis/tier=api, arcgis/app=sharing |
object-volume | 32 | ReadWriteOnce | arcgis/tier=storage, arcgis/app=ozone |
queue-volume | 16 | ReadWriteOnce | arcgis/tier=queue, arcgis/app=rabbitmq |
relational-volume | 16 | ReadWriteOnce | arcgis/tier=storage, arcgis/app=postgres |
spatiotemporal-and-index-volume | 16 | ReadWriteOnce | arcgis/tier=storage, arcgis/app=elasticsearch |
usage-metric-volume | 30 | ReadWriteOnce | arcgis/tier=storage, arcgis/app=prometheus |
Additional considerations for static PVs
The type of provisioned storage that you configure during deployment will determine requirements for upgrades and scaling:
- Dynamic PVs—Storage is scaled and adjusted by the software, provided that sufficient storage is available, and storage-class specifications are met.
- Static PVs—If you are provisioning for your deployment, you will need to provision additional item-packages PVs with the same specifications (labels, size, and access mode) as those specified during deployment to support scaling and upgrade workflows.
Create additional PVs to scale the Portal API deployment
To scale the Portal API deployment (to increase the number of participating pods), an additional item-packages PV is required for each additional pod that is added to the deployment. For example, if the organization requires three additional pods for the Portal API deployment, a minimum of three additional item-packages PVs must be provisioned and configured with equivalent specifications to those specified during deployment.
Dynamic PVs
For dynamic provisioning, a StorageClass is required.
The reclaimPolicy parameter on the StorageClass can be set to Delete to simplify administration. This will clean up the associated PV when a PVC is deleted (for example, when you undeploy the software).
A separate StorageClass should be used for the hosted backup store configuration with its reclaimPolicy parameter set to Retain so that the PV persists when you undeploy and redeploy.
The allowVolumeExpansion parameter on the StorageClass can be set to true if your storage provider supports the expansion of PVs.
Note:
Not all VM types support premium disks in Azure. Use a premium disk when the VM type supports it.- For AKS, the following is an example of a StorageClass definition with Premium Azure Disk:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: arcgis-storage-default provisioner: disk.csi.azure.com parameters: kind: Managed storageaccounttype: Premium_LRS reclaimPolicy: Delete allowVolumeExpansion: true volumeBindingMode: WaitForFirstConsumer - For EKS, the following is an example of a StorageClass definition with GP3 type EBS volumes:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: arcgis-storage-default provisioner: ebs.csi.aws.com parameters: fsType: ext4 type: gp3 reclaimPolicy: Delete allowVolumeExpansion: true volumeBindingMode: WaitForFirstConsumer - For GKE, the following is an example of a StorageClass definition with SSD persistent disk:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: arcgis-storage-default provisioner: pd.csi.storage.gke.io parameters: type: pd-ssd reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer allowVolumeExpansion: true
You can also use the default storage classes provided with an EKS (GP2), AKS (managed or managed-premium), or GKE (persistentdisk) cluster, but you may need to install a CSI driver to support provisioning in Kubernetes version 1.23 or later releases. You must confirm that the reclaimPolicy and allowVolumeExpansion properties meet your needs prior to creating the organization.
Client workstation
The deployment scripts are bash scripts that can be run from a remote client workstation. The user running the scripts must have read and write access for the scripts to write temporary resource files to subdirectories.
Note:
Due to known compatibility issues, Linux emulators are not supported to deploy ArcGIS Enterprise on Kubernetes.
You need the following when setting up your client workstation (download links are provided):
- Kubectl
- An environment-specific command line interface (CLI)
Kubectl is a prerequisite to run the deployment script. Use Kubectl installation and setup to download the Kubernetes command line tool.
Note:
The kubectl client version must be within one minor release of the Kubernetes server version. For example, kubectl 1.24 is compatible with Kubernetes cluster versions 1.23-1.25.
When managing your deployment, you can use environment-specific command line tools. Use the following links to download an environment-specific CLI:
TLS certificate
ArcGIS Enterprise on Kubernetes uses an NGINX-based ingress controller. This ingress controller is namespace scoped and is deployed to listen to only ingress traffic for the ArcGIS Enterprise namespace. A TLS certificate is required with the FQDN in the certificate common name and subject alternate name. Either a CA-signed certificate or a self-signed certificate can be used; however, for security reasons, a CA-signed certificate is recommended. This is the default TLS certificate for the ingress controller. The following certificate options are available in the deployment script to apply a TLS certificate for ingress traffic:
- An existing TLS secret that contains a private key and certificate
- A .pfx file that contains a private key and certificate
- A PEM-formatted private key and certificate
- A self-signed certificate
ArcGIS Enterprise on Kubernetes supports using a TLS certificate for the ingress controller that is issued and managed by Kubernetes cert-manager. This certificate must be stored in a TLS secret in the same namespace as ArcGIS Enterprise. The TLS secret can then be referenced either during deployment or after the ArcGIS Enterprise organization is created.
ArcGIS Pro
- ArcGIS Pro 3.1 is the companion release for ArcGIS Enterprise on Kubernetes 11.1. To benefit from the latest features available, use ArcGIS Pro 3.1.
- To publish services to ArcGIS Enterprise on Kubernetes, ArcGIS Pro 2.8 or later is required.
- To consume services from ArcGIS Enterprise on Kubernetes, ArcGIS Pro 2.7 or later is required.
When registering a data store item from an enterprise geodatabase, the geodatabase version must be 10.9.0.2.8 or later.
Note:
To benefit from the latest features available, upgrade your geodatabase version to 11.1.0.3.1.The geodatabase version number is a combination of ArcGIS Enterprise and ArcGIS Pro release numbers. For more information, review client and geodatabase compatibility.
Upgrade and update requirements
Before performing an upgrade, you must meet several requirements, such as the following:
- When upgrading your environment, you must upgrade ArcGIS Enterprise on Kubernetes before upgrading Kubernetes to a supported version.
- If a required update is available, you must apply it before upgrading to this release. Review the release notes for details about the latest required update.
- You must have a unified ArcGIS Enterprise on Kubernetes license for this release.
- You must update the resource quota values in your namespace to meet the current requirements.
- Ensure your environment meets pod security standards such as the pod security admission standards prior to conducting the upgrade.
- If your environment is in Red Hat OpenShift Container Platform, ensure it meets the current security context constraints.
- If you have provisioned static PVs, you must provision additional storage to accommodate upgrade requirements. See the Adjust static PVS for upgrade section for further details.
- If your provisioned storage has dynamic PVs, you need to ensure that sufficient storage is available for the additional item-packages, object volume, and queue volume.
- Run the preupgrade script. This script will detect and address any functional requirements to meet the current software release.
- If you have configured a web adaptor with your organization, review installation and upgrade requirements.
- If your organization is in a disconnected environment, follow steps to apply an upgrade or update in disconnected environments.
- If you used your organization's container registry when deploying ArcGIS Enterprise on Kubernetes, you must copy the required container images from the Esri repository to your organization's registry before running the update or upgrade.
Adjust static PVs for upgrades
Prior to an upgrade, each pod in the organization's Portal API deployment has been configured with an item-packages volume PV. In preparation for an upgrade, each pod in the Portal API deployment must be configured with an additional PV.
For example, prior to an upgrade, if the Portal API deployment is configured with three running pods, three additional PVs must be provisioned and configured with equivalent specifications as those specified during deployment.
Once the upgrade is complete, the Portal API deployment will use the newly provisioned persistent volumes and persistent volume claim, and the original set can be removed.