Raster analytics provides built-in tools and functions for preprocessing, orthorectification and mosaicking, remote sensing analysis, and an extensive range of math and trigonometry operators. Your custom functions can extend the organization's analytical capabilities even further.
Tip:
Custom python raster functions are currently not supported in ArcGIS Enterprise on Kubernetes.Raster analytics is also designed to streamline and simplify collaboration and sharing. Users across your organization can contribute data, processing models, and expertise to your imagery project and share results with individuals, departments, and organizations within your enterprise.
License:
To configure raster analytics, you must have an ArcGIS Raster Analytics on Kubernetes license. This license includes the functionality available from the image hosting premium capability.
Introduction to raster analytics
Raster analytics is a flexible raster processing, storage, and sharing system that employs distributed computing and storage technology. Use raster analytics to apply raster analysis tools and raster functions offered in ArcGIS, build custom functions and tools, or combine multiple tools and functions into raster processing chains to run your custom algorithms on large collections of raster data. Source data and processed results are stored, published, and shared across your enterprise according to your needs and priorities.
This extensive capability can be further expanded by leveraging cloud computing capabilities and resources. The net result is that image processing and analysis jobs that used to take days or weeks can now be done in minutes or hours, and jobs that were too large or extensive can now be handled.
Tip:
Cloud data storage is a requirement for on premises and cloud deployments. It is used to store raster analytics outputs.Additionally, verify that your administrator has allocated sufficient resource quota and worker nodes to support this premium capability.
Configure raster analytics
The following configuration steps may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding. Before using raster analytics, you must add supporting raster stores and enable it as a capability in ArcGIS Enterprise Manager.
Add raster stores
You must add two raster stores to support raster analytics: one cloud store and one relational store. The cloud store is used to store the raster output of the analysis. The relational store is utilized for storing mosaic datasets while creating a hosted imagery layer or the raster analysis generates a collection output.
- In ArcGIS Enterprise Manager, click the Data stores button in the sidebar.
The data stores page appears.
- In the Raster stores section, click Add store to add the first raster data store.
Note:
If more than one raster store with the same type was added, then the tool will randomly select which raster store is used. You cannot specify which raster store is used.
This raster store will contain the imagery that is added to the organization and the results of any raster analysis tools. When your organization is deployed in a cloud environment, it is recommended to use a cloud-native storage service.
If you have not already added stores, this section will be empty.
- Enter a name for the raster store.
- Select Storage service. Select one of the following options:
- Amazon S3
- Azure Blob
- Google Cloud Storage
- For specifications for each cloud storage service, see steps to add a raster store.
- When finished, click Add store.
The raster store is registered and will appear in the Raster stores section.
- From the Data stores page, click Add store to add the second store; a relational store.
This storage is required for workflows that generate a mosaic dataset for processing rasters.
- Enter a name for the relational store.
- Select Storage service and choose a relational store.
- Click Choose File to add your database connection file.
The relational database will store the mosaic datasets for hosting imagery layers.
- Click Add store.
The raster store is registered and will appear in the Raster stores section.
Enable raster analytics
Next, enable raster analytics as a capability for the organization.
- In ArcGIS Enterprise Manager, click the Capabilities button in the sidebar.
The capabilities page appears.
- Turn on the Raster analytics toggle button.
You are prompted with a message that indicates the process to enable may take some time.
- Click Enable.
A request to enable raster analytics is submitted. This process will validate prerequisites and activate supporting resources. The following system services will be automatically started:
- RasterAnalysisTools
- RasterProcessing
- RasterProcessingGPU
- RasterRendering
If the capability fails to enable, repeat the above steps to ensure that the raster data store has been added, the raster analytics license is valid and available, and the system services have been started. Review the logs to identify the requirements for this capability.
Raster analytics is now configured. You can begin to use raster analysis tools and host imagery in your organization. Additionally, see how to tune raster analytics workflows.
Tune raster analytics
To realize optimal performance and scalability with raster analytics, consider these recommendations.
- When creating your organization, use a cloud-native object storage service.
- Increase worker nodes. Premium capabilities such as raster analytics require a minimum of one additional worker node for each architecture profile to support the added capabilities. Work with your administrator to determine which architecture profile was selected when creating the organization and increase worker nodes accordingly. Depending on the analysis that you will perform, you may need more than one additional node.
- Leverage GPU enabled nodes when possible. Optionally, configure node affinity and tolerations to run GPU workloads exclusively on GPU nodes. This is increasingly important when conducting deep learning and AI workflows.
- By modifying service deployments with additional pods and resources, you can increase overall availability and throughput for your workflows. You may also consider enabling autoscaling on these services.
- For workflows that do not include deep learning and AI, scale the RasterProcessing service deployment.
- For deep learning workflows, when conducting workflows to train models, scale the RasterAnalysisTools service deployment.
- When conducting inferencing workflows, scale the RasterProcessingGPU service deployment.
- Some raster analysis tools distribute computation across multiple worker pods and write temporary data while performing analysis. Use ephemeral volumes when performing distributed raster analysis on large jobs that need to load data to a temporary space for processing. See configure ephemeral volumes for additional details.
- When running raster analytics processes, it is recommended to decrease the log level from Debug. If you're not actively troubleshooting an issue, use the Warning level. If you require finer grained logging detail, use an alternative log level instead.
- Consider setting a parallel processing factor to control the number of raster processing service instances that can be used for processing your data.
- Memory requirements may need to be monitored and increased based on the type of analysis workflow and size of the data processed. For example, for deep learning and AI workflows, larger batch sizes require more memory.
- After enabling raster analytics, the tools have preconfigured limits. Some jobs may require additional capacity in order for the tools to function properly. You can edit the scaling properties to increase memory for different microservices.