Running large-scale CFD fire simulations on AWS for Amazon.com | #linux | #linuxsecurity | #hacking | #aihp

Running large-scale CFD fire simulations on AWS for Amazon.com

This post was contributed by Matt Broadfoot, Senior Fire Strategy Manager at Amazon Design and Construction, and Antonio Cennamo ProServe Customer Practice Manager, Colin Bridger Principal HPC GTM Specialist, Grigorios Pikoulas ProServe Strategic Program Leader, Neil Ashton Principal, Computational Engineering Product Strategy, Roberto Medar, ProServe HPC Consultant, Taiwo Abioye ProServe Security Consultant, Talib Mahouari ProServe Engagement Manager at AWS.

Historically, advances in Computational Fluid Dynamics (CFD) were achieved through large investments in on-premises computing infrastructure. Upfront capital investment and operational complexity have been the accepted norm of large-scale HPC research but this model presents challenges for companies wishing to run large-scale CFD workloads in the shortest possible time whilst also minimizing capital investment.

In recent work, Amazon EU Design and Construction (Amazon) utilized AWS to facilitate additional project oversight and faster CFD modeling for Amazon construction projects. CFD is used for fire strategy approvals for all core buildings, and minimizing the time to run CFD change variations to meet permit time lines is critical to meeting business milestones. Previously, alterations to building configurations created delays of up to 14-21 days. By leveraging AWS for CFD simulations, Amazon shortened the runtimes of these models to less than 1 day.

In this blog post, we discuss the architecture deployed by Amazon on AWS to conduct large-scale CFD fire simulations of Amazon construction projects as part of their Fire Strategy solutions to demonstrate both Associate Life Safety and of the Fire Service arriving to site. This is a real world application based on a previous blog, and a step-by-step workshop, that we recommend you reference after reading this post.

The deployed system provides a simple and consistent replicable process for multiple CFD applications, such as FDS, Pyrosim and Ansys Fluent. It allows internal and external consultants to remain in control of the overall process with zero intervention from Amazon. Thanks to this approach, each individual project can meet strict governance of design requirements.

Overview of solution

A cloud CFD solution was required to provide an efficient adaptation of the previous process running on on-premises HPC, optimizing model criteria and providing efficient interfaces with fire simulation CFD applications such as FDS, Pyrosim and ANSYS Fluent that are used by internal and external consultants. Due to type, size and configuration, Amazon building construction does not fit into ‘standard’ regulatory criteria and requires a custom Fire Strategy to be prepared for each building in each geography requiring a consistent replicable process/architecture with fastest possible simulation complete time combined with lowest possible cost.

System capabilities requirement and operators guide were created for Amazon to supply to internal and third-party consultants, along with optional consultant onboarding, problem solving, technical support, and monthly output/ monitoring reports. A break-down of the costs e.g. per model run, fixed overheads are built in to AWS CFD Cloud that may be accessed independently and according to security requirements by Amazon as the host/customer and by internal.

Two variations of the solution are possible using the described architecture:

1. A standardized solution that supports the most popular open-source code, Fire Dynamics Simulation (FDS) developed by National Institute of Standards and Technology (NIST), as a default CFD application. Users access a portal to run only FDS simulation on Linux with the portal not customizable or expandable to other application software apart than FDS. This has the advantage of a simpler interface applicable to more users.
2. An enhanced multi-ISV application solution can be created in which users will have access to a portal where they can run FDS simulations on Linux, with the portal is also customizable to expand it to other fire modelling CFD software such as Pyrosim, and Ansys Fluent.

Solution

Amazon’s CFD fire simulation has several components in addition to the HPC Cluster services used such as user and administrator interfaces, storage, authentication and authorization, and monitoring for security and operational costs. A high-level description of the architecture diagram of the solution is as below:

:

Key Architecture Elements

The six key elements of the architecture are described below with references to the key HPC services used following this description.

Security

At AWS, security is the top priority and AWS has strict security requirements for every solution built that rely on the shared responsibility model. Best practices for security, identity, and compliance can be found here. The following steps were followed to secure the solution:

Data Classification

Data is classified into one of the available data classes specified by Amazon. The data class of an application determines the level of security controls that would be applied to that application and the infrastructure it is running on, hence, this is the first step performed from a security perspective.

Security Controls and Assessment

Once the data has been classified, the next step is to identify the AWS services that the application would be running on e.g., RDS, DynamoDB, etc. and then create a hardening hierarchy – The hardening hierarchy defines a set of security controls that must be applied to every AWS service that the application may be on. This guide will be used to ensure that the services the application would be running on meet the minimum specified security controls defined by Amazon for the data class of the application.

Security Control Services

In this phase, the architecture was assessed and the security services that were required were implemented. Some of the AWS services implemented here include AWS Web Application Firewall (AWS WAF) access lists, logging and monitoring, and encryption at rest and in transit.

AWS WAF provides the option of using AWS WAF managed rule sets and/or creating your own rules. The option you choose would depend on the services and application types you are running. For this use case, some of the managed rules implemented are listed below:

1. Bot Control
2. Core Rule Set
3. Known Bad Inputs
4. Linux Operating System
5. SQL Database

Logging and Monitoring: CloudTrail and CloudWatch were used to ensure that the appropriate level of logging and monitoring were implemented. CloudTrail records all API calls made within the AWS accounts in use while CloudWatch Logs were used to monitor, store, and access log files from certain AWS services in use. Also, a log retention period was set for all the stored logs. some of the logs stored on CloudWatch Logs include:

1. VPC Flow Logs
2. Application Load Balancer access logs
3. S3 Bucket Access Logs
4. RDS Database Logs

CloudWatch Logs can be used with CloudWatch Alarms to create security alarms when certain security thresh holds are exceeded.

Encryption at Rest and in Transit: Encryption is a critical requirement for data protection, hence, every data transiting the AWS services used for this application was encrypted in transit. Also, every AWS service storing data at rest e.g., S3, EBS volumes, RDS, etc. was configured to encrypt the data.

Threat Modelling

Threat modelling was done on all the AWS services used to build this application. This is a step-by-step process which allows us to check if all the required controls were accurately applied on the individual AWS services used to build the application. Some of the controls that are checked here include authentication, authorization, encryption of data at rest and in transit, logging, etc. This process helps us to detect missing controls that may have not been implemented due to oversight.

Code Reviews

Code reviews are an important part of ensuring security for Amazon CDOs. For this application Automated Code Reviews were done with several code scanning tools. All the codes used to build this application and the ones used to automate the AWS services that were set up were scanned and reviewed over several iterations at every stage of the building of the application and all identified vulnerabilities were fixed.

Web Browser Based Access and GUI

A key requirement was to abstract the HPC cluster access and management into a simple GUI front end for fire simulation engineers. To implement this fundamental aspect, we used NICE EnginFrame HPC portal…

Read the full blog to learn more. Reminder: You can learn a lot from AWS HPC engineers by subscribing to the, and following the AWS HPC Blog channel.

Read the full blog to learn more. Reminder: You can learn a lot from AWS HPC engineers by subscribing to the HPC Tech Short YouTube channel, and following the AWS HPC Blog channel.