Running an HPC Simulation Workflow in AWS
Oil & Gas HPC Reservoir Simulation
The customer is a large oil & gas exploration and production company.
The customer’s existing on-premises HPC cluster was a limited set of aging Intel-based rack hosts serving as a processing resource for petroleum engineers. The engineers use modern desktop systems to create and review oil-field reservoir simulations with variety of industry standard SW packages, but rely on the cluster to analyze the raw data. The compute capacity of the HPC cluster was insufficient to support the engineer’s concurrent workloads, causing frequent manual load balancing and schedule coordination to meet business requirements.
The working solution adds an AWS-based queue to the existing on-premises queue and dispatch simulation jobs to a defined set of EC2 instances for analysis. This software monitors the queue for waiting jobs, and matches requested CPU cores to AWS instance profiles and starts cloud analysis as needed. It also checks for idle EC2 instances and stops them until needed again. Use of AWS resources to augment the existing cluster significantly reduced analysis time and eliminated scheduling bottlenecks.
Simulation Times Cut by 40%
Testing with reference data sets showed that by taking advantage of modern processor architecture and high core count instances available on AWS, the simulation time was reduced by 40% over the on-premises cluster.
Zero Wait Time for New Simulations
With the effectively infinite compute resources available in AWS, the number of concurrent simulation runs were limited only by the number of user licenses the customer owns of the simulation SW and not by the available resources on the HPC cluster.
Using AWS Ensures Optimal Compute Resources
Simulation jobs benefit greatly from both the continuous evolution of processor hardware and the availability of new compute technologies such as GPU and FPGAs, both readily available in AWS.
p1 Technologies & AWS
Established in 2009, P1 Technologies is a team of enterprise IT architects with deep cloud expertise. It is our mission to provide unsurpassed value to our clients through the alignment of their business goals and IT infrastructure. We employ proven methodologies to determine the right balance of on-premises and cloud infrastructure for a solution that is seamlessly integrated, simple to manage, scalable and reliable. p1 is proud to serve more than 150 clients in Southern California and beyond, including industry leaders in Media & Entertainment, Public Sector, Higher Education, Energy, Life Sciences, Engineering, Semiconductor and Retail.
Amazon Web Services provides tools and fully managed services that are designed to collect, process, and analyze streaming data in real-time, so you can improve time-to-insight and react quickly.