Prepare the cluster

Review the following guidance on preparing your cluster before deploying ArcGIS Enterprise on Kubernetes.

Namespace

ArcGIS Enterprise on Kubernetes requires its own dedicated namespace. The namespace must be created before running the deployment script. Running multiple deployments of ArcGIS Enterprise on Kubernetes in a single cluster will require a unique namespace for each deployment.

Resource quota

ArcGIS Enterprise on Kubernetes pods have defined requests and limits for CPU and memory. If the namespace has a ResourceQuota object, the quota must be higher than the sum of all the pods' requests and limits. These values vary based on the architecture profile you've selected, as described below.

Note:

If you are performing an upgrade to 11.1, you must first update the resource quota values in the namespace to the 11.1 requirements.

The following recommended values are the minimum requirements for each architecture profile. Consider that you may need to increase these values to accommodate your organization's scalability requirements.

Enhanced availability profile:

spec: 
    hard: 
      limits.cpu: "230" 
      limits.memory: 394Gi 
      requests.cpu: "36" 
      requests.memory: 188Gi

Standard availability profile:

spec: 
    hard: 
      limits.cpu: "198" 
      limits.memory: 326Gi 
      requests.cpu: "32" 
      requests.memory: 130Gi

Development profile:

spec: 
    hard: 
      limits.cpu: "144" 
      limits.memory: 226Gi 
      requests.cpu: "20" 
      requests.memory: 86Gi

Namespace network requirements

If you use Kubernetes NetworkPolicies, ensure that uninterrupted pod-to-pod and pod-to-service communication is allowed in the ArcGIS Enterprise namespace.

Additionally, ensure that the pods in the namespace have access to the Kubernetes API server. The API server is accessible through a service named Kubernetes in the default namespace. ArcGIS Enterprise pods use the fully qualified domain name (FQDN) kubernetes.default.svc.cluster.local to query the API server.

Note:

cluter.local is the default domain of the cluster.

Pod Security Policies

Note:

PodSecurityPolicy was removed at Kubernetes v1.25.

ArcGIS Enterprise on Kubernetes deploys Elasticsearch to support various features of the ArcGIS Enterprise organization. By default, Elasticsearch uses the mmapfs directory to store required indices. The default operating system limits on map counts may be insufficient for deployment. Elasticsearch recommends a default vm.max_map_count value of 262144. To change the default value, an elevated (root) privilege is required on each node.

Depending on whether the Kubernetes cluster includes a pod security policy and allows containers to run as privileged or unprivileged, the following actions are required:

• If the Kubernetes cluster does not include a pod security policy but allows containers to run as privileged, no action is required.
• Run as privileged—If the Kubernetes cluster has pod security defined and allows containers to run as privileged, you must allow the Elasticsearch service account to run containers as privileged. Other service accounts do not need to run containers as privileged. ArcGIS Enterprise on Kubernetes can run a privileged init container on the node running Elasticsearch, which changes the vm.max_map_count value. The ArcGIS Enterprise on Kubernetes deployment script creates a service account under its namespace to use API Server authentication for processes inside the pods. The Elasticsearch pod uses its own service account, which is not shared with other workloads. The default Elasticsearch service account is arcgis-elastic-serviceaccount. You can grant the service account access to the pod security policy with RBAC Roles and RoleBindings. For OpenShift, you can grant the service account access to the privileged security context constraints by adding the following in the user section:
  “-system:serviceaccount: <Namespace>:arcgis-elastic-serviceaccount"
  

• Run as unprivileged—If the Kubernetes cluster has pod security defined and cannot allow the ElasticSearch service account to run containers as privileged, you must prepare each node manually by running the following command as root:
  sysctl -w vm.max_map_count=262144
  

• If you have created the PodSecurityPolicy resource, you will need to authorize the following service accounts in the ArcGIS Enterprise namespace.
  • arcgis-admin-serviceaccount
  • arcgis-elastic-serviceaccount
  • arcgis-ingress-serviceaccount
  • arcgis-prometheus-serviceaccount
  • arcgis-queue-serviceaccount
  • default

  ArcGIS Enterprise on Kubernetes containers can run without root privileges. However, the control aspect of fsGroup and supplementalGroups of PodSecurityPolicy must have either RunAsAny or a range that includes the value 117932853 as shown in the following examples.

  supplementalGroups:
      rule: 'RunAsAny'
  fsGroup:
      rule: 'RunAsAny'
  

  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 117932853
        max: 117932853
  
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 117932853
        max: 117932853
  

Note:

Pods in the cluster must be allowed to run with a FSGroup and SupplementalGroup ID of 117932853.

Pod Security Standards

Pod security standards are a Kubernetes feature that allows administrators to enforce a set of security rules for pods before they are scheduled within the cluster. There are three levels available by default: privileged, baseline, and restricted. The warning and enforcement of these standards is controlled per-namespace.

When either the baseline or restricted enforcement levels are set, pods that don't meet the security requirements for these levels will not be allowed to run. To learn more about each level, see the following:

• Privileged—Privileged is a unrestricted pod security standard that allows unrestricted access to a wide range of permissions. ArcGIS Enterprise can run under this level without any configuration modification. This level allows for known privilege escalations within the cluster, so running at this level is discouraged.
• Baseline—Baseline is a more limited pod security standard that prevents host filesystem access from the pods and has restrictions on types of other volumes and capabilities. ArcGIS Enterprise can run under the baseline enforcement level by modifying the underlying worker nodes outside of the context of the running application and setting the ALLOW_PRIVILEGED_CONTAINERS property to false in the deployment properties file prior to deployment.

  To match the elasticity of the worker node groups, an administrator must use a scripted process to increase to the vm.max_map_count setting to 262144 on each newly provisioned node by modifying the launch template directly.

• Restricted—The restricted pod security standard offers the most stringent limitations on pod security and requires a majority of securityContext properties to be set explicitly. At this release, ArcGIS Enterprise cannot run at the restricted level.

For more information, see Pod Security Admission and Pod Security Standards.

Metrics Server

To take advantage of horizontal pod autoscaling in ArcGIS Enterprise on Kubernetes, the metrics server is required to scrape pod resource consumption metrics from the kubelet process on each running node. For more information, see Kubernetes Metrics Server.