Utilizing Local Storage on a GPU Compute Node

The default GPU instance types include fast local NVMe storage. This storage is available on the compute node at:

/scratch

For data-intensive workflows, writing input and temporary files to this location can significantly improve performance. After the final output file is generated, it should be moved to the persistent storage location:

/mnt/lustre/lab/{path}

Utilizing Local Storage on CPU Compute Nodes (Upcoming)

AWS EC2 instances can include local NVMe storage at an additional cost. The available NVMe capacity ranges from 100 GB to 22.8 TB, depending on the instance type.

Instances with local NVMe storage are particularly useful for workflows that require high disk I/O during intermediate processing, not just during initial data reads and final output writes.

Example Scenario

Suppose a project needs to process 5,000 files, each 20 GB, and runs 200 jobs in parallel across 200 cpu2mem8a nodes.

For each job:

• Reads a 20 GB input file
• Runs 10 processing steps
• Each step generates a 20 GB temporary file
• Produces a 10 GB final output

Per node:

• 220 GB read
• 210 GB written

Across 200 nodes:

• 4+ TB reads
• 4+ TB writes

This level of I/O would likely saturate the shared Lustre filesystem, causing:

• Slower job execution due to I/O wait times
• Increased compute costs
• Reduced performance for other labs using the same storage system

Solution: Instances with Local NVMe Storage

To reduce load on Lustre, a new compute resource (e.g., cpu2mem8nvme) can be created using EC2 instance types such as:

• m6id.large
• m8id.large
• m6idn.large

These instances provide:

• 2 CPUs
• 8 GB RAM
• 118 GB local NVMe storage

Cost comparison:

Recommended Workflow Adjustment

To fully utilize the local NVMe storage:

• Write temporary and intermediate files to:

/scratch

• Only store final output files on Lustre:

/mnt/lustre/...

Result

Using local NVMe storage reduces Lustre usage significantly.

Instead of:

• 4+ TB reads
• 4+ TB writes

The workload would generate approximately:

• 400 GB reads
• 200 GB writes

This reduces filesystem congestion, improves job performance, and lowers overall compute costs.

NIH STRIDE rates

We operate under special pricing rates for S3 storage under the STRIDES program. These rates are the same for all AWS US commercial regions. STRIDE AWS rates are much cheaper than the standard AWS price (18% to 25% less, depending on which region and type of S3 category; see https://calculator.aws/#/ ) For the latest information regarding NIH STRIDES pricing, please see https://cumcprod.service-now.com/kb?id=kb_article_view&sys_kb_id=44f5b109973e2d5089c5f576f053afc4 (Requires authentication) Below is the pricing as of January 2026

In addition to S3 discounts, CUIMC AWS accounts also receive:

• A flat 9% discount on ALL AWS services except for AWS Marketplace items and Glacier Deep Archive.
• At least a 17% discount for S3 AWS Storage Services – Standard, Infrequent Access, Intelligent Tiering, and Glacier (Specific discounts vary based on region and the specific S3 service).
• No charge for AWS Enterprise Support, and up to 15% in savings for researchers.
• Access to a data egress discount program. The max offset is 15% of total monthly spending on AWS services, several times the usage typically used among our research customers.
• Special pricing for AWS CloudFront.
• Free access to +12PB of diversified biomedical datasets and workload management systems.
• AWS Cloud Credits for Researchers.
• Discounted, Subsidized, and Free Training options (for example, Free Cloud Education for students using AWS Educate.)
• Discounts on Professional Services.
• Support for Sole Source Justification letter by AWS.

Additional benefits/offerings provided by AWS

Support for Researchers, Principal Investigators, and Academic Institutions

Customized reporting and dashboard with usage of cloud resources and billing. Health checks focused on cloud instances, and saving plans. Help with optimal account setup and provisioning.

Amazon Research Awards

The Amazon Research Awards (ARA) program offers awards of up to $80,000 to faculty members at academic institutions (see: https://ara.amazon-ml.com/)

AWS Machine Learning Awards

Funds are eligible for universities, faculty, PhD students, and post-docs under the supervision of faculty advancing machine learning (ML) using NIH STRIDES datasets. (apply with sponsor tag “NIH STRIDES” at: https://aws.amazon.com/aws-ml-research-awards/)

Amazon Internship Program

Amazon offers internships year-round. Projects will depend on a student’s area of research and interest in the biomedical field and the team to which they are being placed (see: https://www.amazon.jobs/en/teams/internships-for-students.)

Application: Amazon Scholar Program

The Amazon Scholars program is designed for academics who want to apply research methods. The program provides opportunities to join Amazon in a flexible capacity, in particular part-time arrangements and sabbaticals.