Cloud Considerations for Survey Processing

Chesapeake Times, Vol 3 | October 2020

Recently we’ve seen a good deal of interest in cloud platform services and related inquiries into how SonarWiz would run in their environments. Companies stand to see a lot of benefits from these relatively low-cost, scalable, externally-managed resources, and we’ve been investigating the various opportunities and challenges involved in utilizing them.

Originating mainly as internal services used by large tech companies, cloud platforms started to be offered to the public in the early 2000s. Initially utilized by customers primarily as hosting and web software delivery platforms, cloud services have seen increasing use as hosts for traditional desktop software since then. Their availability from any networked location and flexibility in machine configuration are beneficialto many kinds of software and related workflows.

Cloud instances can be tailored extensively – many options are offered, allowing companies to choose machines that are focused toward most reasonable blends of CPU and GPU capability, memory, disk size and speed, and network bandwidth.  Allowing for a mix of machine configurations in the pipeline can help with maintaining a good cost to performance ratio, especially for large projects.

Early in a project, only moderate computing resources are necessary in planning out lines to cover a given survey area and estimating hours required to achieve a desired coverage. A fast network connection can help if using pannable online maps to orient within the survey area, but otherwise, a low-powered machine will suffice.
After collection and upload of raw files to storage, higher disk bandwidth is useful for import; all of navigation, bottom tracking, bathymetry merging and gain processing can benefit from this as well, with the addition of strong CPU capability.

Interpretation benefits from moderate-to-strong disk capability, and may benefit from GPU in some instances. For deliverable creation and export, disk throughput and latency come to the forefront again, with strong CPU helping somewhat, just as with pre-interpretation processing.

After delivery and acceptance, project data can be archived; cloud platforms offer low-cost cold storage options for infrequently-accessed data. The data won’t be available to work with quickly when stored in this manner, but the cost savings over faster storage is substantial. 

Workflows with “spiky” resource needs over a weeks- or months-long timescale can realize a related benefit to the configurability of cloud platforms due to their on-demand nature. Instances with moderate resources can be maintained for the lower baseline activity level – line planning, contact picking, and so forth, with heavier instances brought to bear during more intensive processes such as import, gain application, and geoimage export.

In some instances, cloud platforms can also offer increased options for collaboration.  The model of centralized storage coupled with distributed access offers the potential for different numbers of people to work on a project at different times, depending on need. Currently SonarWiz stores the project as a folder on a disk rather than as, say, centralized in a concurrently-accessible database, making it unfeasible to have multiple people working on exactly the same project.

That said, splitting a survey by geographical area, sonar type or frequency, time or vessel can be used to achieve many of the same benefits. Splitting projects in this way can be done with centralized on-premises storage, but that comes at the cost of maintaining the infrastructure required to allow for off-site access, something that is built into the cloud model.

We are excited about the possibilities offered by cloud services, and are making steady progress toward ensuring that SonarWiz and related tools work well in these environments. There are some challenges to overcome, but the benefits that can be realized are worth some investment.

– Jonathan Fleetwood, Engineer