Raster analytics provides built-in tools and functions for preprocessing, orthorectification and mosaicking, remote sensing analysis, and an extensive range of math and trigonometry operators. Your custom functions can extend your organization's analytical capabilities even further.
Raster analytics is also designed to streamline and simplify collaboration and sharing. Users across your organization can contribute data, processing models, and expertise to an imagery project and share results with individuals, departments, and organizations within your enterprise.
Introduction to raster analytics
In ArcGIS Enterprise on Kubernetes, raster analytics is a flexible raster processing, storage, and sharing system that employs distributed computing and storage technology. Use raster analytics to apply raster analysis tools and raster functions provided in ArcGIS, build custom functions and tools, or combine multiple tools and functions into raster processing chains to run custom algorithms on large collections of raster data. Source data and processed results are stored, published, and shared across your enterprise according to your needs and priorities.
This extensive capability can be further expanded by using cloud computing capabilities and resources. The result is that image processing and analysis jobs that used to take days or weeks can now be done in minutes or hours, and jobs that were too large or extensive can now be completed.
Tip:
Cloud data storage is a requirement for on-premises and cloud deployments. It is used to store raster analytics outputs.Confirm that your administrator has allocated sufficient resource quota and worker nodes to support this premium capability.
Add raster stores and enable raster analytics
The following configuration steps may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding. Before using raster analytics, you must add supporting raster stores and enable it as a capability in ArcGIS Enterprise Manager.
Add raster stores
You must add two raster stores to support raster analytics: one cloud store and one relational store. The cloud store is used to store the raster output of the analysis. The relational store is used for storing mosaic datasets while creating a hosted imagery layer or the raster analysis generates a collection output.
To add raster stores, complete the following steps:
- In ArcGIS Enterprise Manager, click the Data stores button in the sidebar.
The data stores page appears.
- In the Raster stores section, click Add store to add the first raster data store.
Note:
If more than one raster store with the same type was added, the tool will randomly select the raster store to use. You cannot specify which raster store is used.
This raster store will contain the imagery that is added to the organization and the results of any raster analysis tools. When your organization is deployed in a cloud environment, it is recommended to use a cloud storage service from the same cloud provider.
If you have not already added stores, this section will be empty.
- Provide a name for the raster store.
- Select Storage service, and choose one of the following options:
- Amazon S3
- Azure Blob
- Google Cloud Storage
- For specifications for each cloud storage service, see the steps to add a raster store.
- When you're finished, click Add store.
The raster cloud store is registered and appears in the Raster stores section.
- On the Data stores page, click Add store to add the second store: a relational store.
This storage is required for workflows that generate a mosaic dataset for processing rasters.
- Provide a name for the relational store.
- Select Storage service and choose a relational store.
- Click Choose File to add the database connection file.
The relational database will store the mosaic datasets for hosting imagery layers.
- Click Add store.
The raster relational store is registered and appears in the Raster stores section.
Enable raster analytics
To enable raster analytics as a capability for the organization, complete the following steps:
- In ArcGIS Enterprise Manager, click the Capabilities button in the sidebar.
The capabilities page appears.
- Turn on the Raster analytics toggle button.
A message appears indicating that the process to enable may take some time.
- Click Enable.
A request to enable raster analytics is submitted. This process will validate prerequisites and activate supporting resources. The following system services will be automatically started:
- RasterAnalysisTools
- RasterProcessing
- RasterProcessingGPU
- RasterRendering
If the capability fails to enable, repeat the steps above to ensure that the raster stores have been added, the raster analytics license is valid and available, and the system services have been started. Review the logs to identify the requirements for this capability.
Raster analytics is now configured. You can begin to use raster analysis tools and host imagery in your organization. Learn how to use raster analysis and deep learning.
Additionally, learn how to tune raster analytics workflows.
Enable GPU resources for raster analytics
Raster analysis, especially deep learning analysis and AI tools can use graphics processing unit (GPU) resources in your cluster. At 11.5, you can configure GPU resources raster analysis tools to enhance the performance for deep learning training and inferencing tools. For deep learning, GPUs provide processing speed, resource efficiency, and scalability for your organization. Since they can handle massive parallel computations, GPUs are indispensable for raster analysis and deep learning. Individual services function better for both tasks when GPUs are configured for each service.
For training deep learning model jobs, you only need to configure the RasterAnalysisTools service. For inferencing tools, you need to configure RasterProcessingGPU services with GPUs. Certain models may also need the RasterAnalysisTools service to be configured with the GPU to be initialized. In these cases, both services need to be configured with GPUs.
Prerequisite
In ArcGIS Enterprise on Kubernetes, you can implement a device plug-in to enable GPU nodes in your cluster. To enable GPU nodes, a NVIDIA device plug-in for Kubernetes is required. Fore details, see GPU enabled nodes.
Note:
At the 11.5 release, ArcGIS Enterprise on Kubernetes is only supported with NVIDIA GPUs.To enable GPU resources for raster analytics for your organization, complete the following steps, which are specific to your environment and preferences:
- Complete the steps to configure raster analytics or another capability for which you want to use GPU-enabled nodes.
- Verify that your instance has the device plug-in installed.
Many cloud environments are preconfigured with GPU nodes. If the device plug-in is not installed, see the NVIDIA device plug-in for Kubernetes documentation for details and installation steps. If you've deployed on-premises, your administrator must enable GPU resources on each node in a cluster.
- To use GPU-enabled nodes for your organization's GIS workflows, set requests and limits for GPU resources.
- Optionally, to run GPU workloads exclusively on GPU nodes, configure node affinity and tolerations.
Tune raster analytics
For optimal performance and scalability with raster analytics, consider the following recommendations:
- When creating your organization, use a cloud storage service for the object store.
- Increase worker nodes. Premium capabilities such as raster analytics require a minimum of one additional worker node for each architecture profile to support the added capabilities. Work with your administrator to determine which architecture profile was selected when creating the organization and increase worker nodes accordingly. Depending on the analysis that you will perform, you may need more than one additional node.
- Use GPU-enabled nodes when possible. Optionally, configure node affinity and tolerations to run GPU workloads exclusively on GPU nodes. This is increasingly important when conducting deep learning and AI workflows.
- By modifying service deployments with additional pods and resources, you can increase overall availability and throughput for your workflows as follows:
- For workflows that do not include deep learning and AI, scale the RasterProcessing service deployment.
- When conducting workflows to train models for deep learning workflows, scale the RasterAnalysisTools service deployment.
- When conducting inferencing workflows, scale the RasterProcessingGPU service deployment.
- Some raster analysis tools distribute computations across multiple worker pods and write temporary data while performing analysis. Use ephemeral volumes when performing distributed raster analysis on large jobs that need to load data to a temporary space for processing. See configure ephemeral volumes for details.
- When running raster analytics processes, it is recommended that you decrease the log level from Debug. If you're not actively troubleshooting an issue, use the Warning level. If you require finer-grained logging detail, use an alternative log level instead.
- Consider setting a parallel processing factor to control the number of raster processing service instances that can be used for processing the data.
- Memory requirements may need to be monitored and increased based on the type of analysis workflow and size of the data processed. For example, for deep learning and AI workflows, larger batch sizes require more memory.
- Raster analytics tools have preconfigured limits. Some jobs may require additional capacity for the tools to function properly. You can edit the scaling properties to increase memory for different microservices.