In Kubernetes, newly created and unscheduled pods are automatically scheduled to nodes that meet their requirements. By using node affinity, taints, and tolerations, you can have more control over the nodes that pods are scheduled to.
Node affinity allows you to specify rules that constrain pods to run on certain labelled nodes. Taints are applied to nodes to repel pods, while tolerations are applied to pods to tolerate taints. To learn more about node affinity, taints, and tolerations generally, refer to the Kubernetes documentation. See Manage pod placement for information on how to apply node affinity and tolerations to pods in ArcGIS Enterprise on Kubernetes.
Combining node affinity, taints, and tolerations helps you achieve granular control over workload placement to enhance isolation, optimize resource allocation, and effectively meet compliance requirements within your Kubernetes cluster:
- Isolate workloads with specialized requirements—Use labels and node affinity rules to ensure that certain pods are scheduled on dedicated nodes. Use taints to mark nodes with specific characteristics, such as high CPU or memory requirements, as dedicated for ArcGIS workloads. Apply tolerations on service pods to ensure they are scheduled on nodes with the required resources.
- Optimize resource allocation—Apply taints on nodes with limited resources to prevent resource overload, and define tolerations on service pods to match the resource constraints on these nodes. Combine node affinity with taints and tolerations to ensure that service pods are only scheduled on nodes that can meet their resource requirements.
- Geolocation-based scheduling—For applications that require data locality or adherence to specific regulations, use node affinity to schedule service pods based on the geographic location of nodes. Taint nodes based on their physical location or data sovereignty regulations, and apply tolerations on service pods to ensure they are scheduled on nodes and compliant with required location constraints.
Autoscaling enhances the use of node affinity and tolerations by dynamically adjusting the number of pods based on workload demands. This dynamic scaling ensures that pods are efficiently scheduled on nodes that meet specific requirements or have the necessary resources available, optimizing resource allocation. By combining autoscaling with node affinity and tolerations, Kubernetes clusters can achieve improved resource utilization, performance, and scalability—adapting to workload fluctuations while adhering to node constraints and preferences. To learn more about autoscaling, see Service scaling.
Scenarios
To better understand how managing pod placement on services can benefit your organization, review the following scenarios.
Scenario 1: Seasonal traffic surge for public mapping services
A public organization experiences a significant increase in traffic during a local festival. Users accessing the web map for event information experience delays due to high demand on the underlying map service. To address this, the organization administrator does the following:
- Configures nodes with high CPU and memory resources with the key-value pair high-performance: true.
- Applies node affinity rules to ensure that the map service pods are scheduled on nodes with high CPU and memory resources:
- Type—Preferred
- Key—high-performance
- Operator—Exists
- Value—true
- Applies tolerations to allow pods to run on nodes tainted for high-performance workloads, ensuring that the map service can handle the surge in traffic:
- Effect—NoSchedule
- Key—workload
- Operator—Equal
- Value—high-performance
- Taints high-performance nodes with workload=high-performance:NoSchedule.
Scenario 2: Data processing for environmental monitoring
An environmental agency is running a series of geospatial analyses to monitor changes in land use. The analysis requires significant computational resources, and the agency has dedicated nodes with GPUs for this purpose. To ensure that the geospatial analysis runs effectively without competing for resources with other services, the organization administrator:
- Configures GPU enabled nodes with the key-value pair gpu: true.
- Applies node affinity rules to schedule the analysis pods on the GPU nodes:
- Type—Required
- Key—gpu
- Operator—In
- Value—true
- Applies tolerations to allow pods to run on the tainted GPU nodes:
- Effect—NoSchedule
- Key—workload
- Operator—Equal
- Value—high-resource
- Taints GPU nodes with workload=high-resource:NoSchedule to prevent less resource-intensive pods from being scheduled there.
Scenario 3: Resource optimization for shared feature services
A city's GIS department has numerous feature services that are not heavily used but collectively burden a single service deployment. To allow the department to maintain service availability without overloading the system, the organization administrator:
- Configures nodes with the key resource-constrained.
- Applies node affinity rules to prioritize scheduling on nodes with lower resource availability:
- Type—Preferred
- Key—resource-constrained
- Operator—DoesNotExist
- Applies tolerations on feature service pods to ensure they can be scheduled on tainted nodes despite constraints:
- Effect—PreferNoSchedule
- Key—resource-constrained
- Operator—Exists
- Taints resource-constrained nodes with resource-constrained:PreferNoSchedule.
Scenario 4: Prevent data store disruption during cluster scaling
A national government has a service usage pattern where services are heavily used during daytime hours. This pattern requires a large number of cluster nodes to support the all the pod replicas needed for these services. Because the services are not used at night, the organization would like to scale down the number of nodes to save on their cloud compute costs. Terminating nodes where system managed data store pods are running, however, creates risk of disrupting those pods. To prevent this potential disruption, the organization administrator:
- Creates a separate node group for data store pods with each node labeled with the key-value pair data-store: true.
- Applies node affinity rules to ensure data store pods are scheduled on nodes in this group
- Type—Required
- Key—data-store
- Operator—In
- Value—true
- Applies tolerations to allow data store pods to run on the tainted data store nodes:
- Effect—NoSchedule
- Key—workload
- Operator—Equal
- Value—data-store
- Taints data store nodes with workload=data-store:NoSchedule.
- Does not scale down the data store node group when scaling down cluster nodes at night.