Atipa Solutions for Lustre
Atipa Solutions for Lustre
What Does A Lustre Solution Look Like?
Intel® Solutions for Lustre*
Data-intensive HPC applications pose strong demands on storage systems to feed large volumes of data into large-scale simulations, multi-dimensional modeling, and computational data analyses. Long-standing file systems used to share data across a network, such as the Network File System (NFS), were not designed with large numbers of clients running I/O-intensive applications in mind. In large HPC environments, NFS servers therefore quickly become a bottleneck that can grind an otherwise optimally functioning cluster to a halt. Distributed file systems such as Lustre, on the other hand, spread the I/O workload over many servers and are therefore perfectly suited for large-scale computational environments.
The Atipa LZ-60v2 and LZE-576v2 High Performance Storage Solutions provide easily usable, scalable Lustre storage solutions for use in academia, government, and industry. Our storage solutions are designed from the ground up to harness the power of the Lustre parallel file system with the data integrity of the Zettabyte File System (ZFS). The Atipa LZ-60v2 is targeted towards users who need a fast affordable storage solution without all the bells and whistles of an enterprise appliance. The Atipa LZE-576v2, on the other hand, is our flagship enterprise appliance equipped with high-availability features and 24/7 support. In addition to our reference architectures, we also offer flexible custom storage solutions designed to fit your needs.
Entry-Level Lustre Solutions
The LZ-60v2 is Atipa’s entry level Lustre appliance for users in need of fast scratch space with small to medium capacity or for users looking for a proof-of-concept that Lustre will accelerate their workload. The solution consists of one combined metadata/management server (MDS/MGS) and three dedicated object storage servers (OSS) with a total storage capacity of up to 60TB* in only 4U of rack space. Storage capacity can be expanded with the LZ60v2-ext expansion module, which increases the available storage up to 140TB* and adds an additional 4U of rack space to the solution. Performance can be increased nearly linearly by adding additional LZ-60v2 modules.
LZ-60v2 Scalable Lustre System Modules
Enterprise-Level Lustre Solutions
The LZE-576v2 is Atipa’s 2nd Generation flagship Lustre appliance for enterprises. The starting configuration consists of one Metadata and Management Module and one Object Storage Module. The Metadata and Management Module contain one metadata server, one management server, one metadata target, and one management target. The Object Storage Module contains two object storage servers and eight object storage targets (four for each object storage server). The entire solution provides 4.4TB of metadata storage and 576TB of usable Lustre storage in 12U of rack space. The modular architecture provides an easy way to scale both capacity and performance by adding additional Metadata and Management Modules and/or Object Storage Modules.
Atipa partners with Intel to offer world class 24/7 Lustre support around the world for enterprise appliances. Atipa engineers are the central point of contact for all support tickets and take the lead in Level 0, Level 1, and Level 2 support issues. This includes, but is not limited to, assisting in collecting diagnostics information, reproducing the problem, eliminating hardware faults. If needed, support tickets are escalated to Intel for Level 3 support (code fixes) with a severity representing the operational impact of the reported issue. Severity 1 (file system is down) issues trigger a broadcast to the entire Intel High Performance Data Division (HPDD) team and have a response time of less than 4 hours.
LZE-576v2 Scalable Enterprise-Level Lustre System Modules
Intel® Manager for Lustre* Server (optional)
Contact email@example.com for more information or customized configurations.
Performance, Density, Availability
What Does a Lustre Solution Look Like?
A Lustre file system is typically comprised of multiple components, each of which has a well-defined role in the operation of the file system. The three main components are the Management Server (MGS), the Metadata Servers (MDS), and the Object Storage Servers (OSS). The Management Server and Metadata Servers are the brains of the file system, keeping track of file system configuration and file locations, while the Object Storage Servers are the workhorses moving bulk data between the backend storage and the clients. Intel® Enterprise Edition for Lustre* supplements these components with Intel® Manager for Lustre*, an easy-to-use web-based central access point to manage and monitor the Lustre file system. A detailed overview of Intel® Manager for Lustre* can be found on the Intel® Solutions for Lustre* tab.
The Management Server, Metadata Servers, and Object Storage Servers each have their own backend block storage, called targets, designed to match their specific duties. The Management Target (MGT) stores file system information, the Metadata Targets (MDT) store file attributes, and the Object Storage Targets (OST) store actual file data. When a client needs to access a file, it requests the location of the file on the OSTs from the MDS. Once the file is opened, data is transferred directly between the client and the OSSs. The MDS itself does not participate in any I/O to and from the file.
The components of a Lustre file system are typically connected by two networks: a management network and a high-speed data-transfer network. The management network is used to deploy and monitor the different Lustre servers, while the high-speed data-transfer network is used to move data between the backend storage and the clients. The following image illustrates how the different components of a Lustre system are tied together by the two networks:
The roles of the Lustre file system components are summarized in the following table:
As an Intel® Solutions for Lustre* – Elite partner, Atipa designs, builds, deploys, and supports HPC storage solutions based on Intel® Foundation Edition for Lustre* and Intel® EE for Lustre* Software. We offer both entry-level and enterprise-level Lustre solutions backed by solid support from Atipa and Intel. A detailed description of our Lustre solutions can be found on the Atipa Solutions for Lustre tab at the top of this page.
Intel® Solutions for Lustre* provides a fast, massively scalable, parallel file system, which is needed to accelerate performance on the most complex HPC and data-intensive business workloads. Lustre is currently running in production at hundreds of sites worldwide and is used by more than 60% of the top 100 fastest supercomputers in the world. Lustre architecture provides a POSIX compliant file system interface and scales to thousands of clients, tens of petabytes of storage, and hundreds of gigabytes per second of sustained I/O bandwidth.
Why Choose Intel® Solutions for Lustre* Software?
• Intel is the leading contributor and investor in the Lustre open source community
• Intel® Solutions for Lustre* software is tightly controlled for quality
• Intel High Performance Data Division (HPDD) constitutes the largest team of Lustre developers and engineers in the world
• Intel® Solutions for Lustre* software is a core pillar of Intel® Scalable System Framework
• Intel® Solutions for Lustre* adaptation continues to expand at an accelerating rate
Why Choose Atipa Solutions for Lustre Software?
• Atipa is the first Intel® Solutions for Lustre* reseller to achieve Elite status in training and knowledge to support Lustre
• Atipa has been deploying quality Hard Disk Drive and Solid State Drive Lustre systems on CentOS Linux, Red Hat Enterprise
Linux, and SUSE Linux Enterprise Server since 2010
• Atipa offers entry to enterprise level stock and custom configurations
• Atipa uniquely compiles Lustre from source for each deployment to match the customer’s preferred linux kernel and high-
speed interconnect software stack
What are the Intel® Solutions for Lustre* software?
• Intel® Enterprise Edition for Lustre* software. The most trusted, reliable and easy to use version of Lustre anywhere. Intel®
EE for Lustre* software couples the performance and scalability of the fastest file system in the world, with
advanced storage management tools for enterprises needing scale-out storage to fuel their technical applications or
Hadoop* MapReduce* workloads.
• Intel® Foundation Edition for Lustre* software. For those who must have the very latest cutting edge version of Lustre, you
now can have it with the confidence of Intel support and a maintenance release stream to ensure the highest reliability
possible. It is ideal for organizations who prefer to design and deploy their own community configurations, and want to
augment in-house management and support capabilities with Intel expertise.
|Compare Editions||Intel® Enterprise Edition for Lustre* Software||Intel® Foundation Edition for Lustre* Software|
|Enormous storage capacity and I/O||√||√|
|Rigorously tested stable software||√||√|
|Resilient, highly available storage configurations||√||√|
|Global 24/7 technical support||√||√|
|Developed and supported by Intel®, the largest contributor to Lustre community releases||√||√|
|Intel backed support for debugging and patches||√||√|
|Stable enterprise platform, built for reliability and performance – with long term support||√||√|
|Intel® Manager for Lustre software for simplified installation and configuration||√|
|Centralized, GUI-based administration for easy management||√|
|Integrated support for Hadoop* MapReduce* applications with Lustre storage||√|
|Open, documented application program interfaces for deep integration||√|
What is Intel® Manager for Lustre* Software?
Intel® Manager for Lustre* software includes simple, but powerful, management tools that provide a unified, consistent view of Lustre storage systems and simplify the installation, configuration, monitoring, and overall management of Lustre. The manager consolidates all Lustre information in a central, browser-accessible location for ease of use. The latest Intel® Manager for Lustre* can further exploit the data resiliency and volume management capabilities available with OpenZFS, the open source successor to the Zettabyte File System (ZFS).
|Atipa added value||Intel® Enterprise Edition for Lustre* Software||Intel® Foundation Edition for Lustre* Software|
|Stock solutions and custom configurations from Atipa||√||√|
|Robust hardware tested and configured by Atipa||√||√|
|Lustre pre-installed and configured by Atipa||√||√|
|Performance validated by Atipa||√||√|
|On-site installation and training by Atipa||√||√|
|Level 0, 1, and 2 from Atipa||√||√|
|Single-point of contact for all support tickets||√||√|
|Silver or Gold 3-year warranty and optional 4th and 5th year warranty from Atipa||√||√|
|Optional firmware and software upgrade services by Atipa||√||√|
4921 Legends Dr
Lawrence, KS 66049
"Congrats on being our first Reseller – Elite participant! "
Intel Lustre Team
"Atipa Technologies, one of the Intel® Cluster Ready partners has been demonstrating early adoption of the new processor family with solutions for Intel® Xeon® processor E5-2600 v4 based clusters ready to go now, assuring compliance with a standardized cluster platform validated by Intel® Cluster Checker 3."
Intel Cluster Ready Team
"Everything has been really solid so far. You are currently at 99.79% uptime while HPL is running."
University of Chicago
"This relationship and interaction with Atipa has allowed us to do our own clustering here while focusing on our research. Basically, we just plugged it in, and exchanged a few emails here and there to solve problems."
Center for Interdisciplinary Research on Complex Systems at Northeastern University
"This is going so smooth compared to our previous cluster; we were in a meeting with our business office yesterday and were discussing the value Atipa provided throughout this purchase."
University of Chicago
"Thank you so much. I was lost without your help."
"Thanks for the great communication."
"Thanks so much! Great work!"
The University of Kansas
"Thanks a lot for all the help. This personal help is the main reason that I keep buying from you guys!"
"Thank you for having these ready so quickly!"
City of Lawrence
"Thanks for taking care of the cluster with in the past year."
Carnegie Institution for Science
"By the way, the cluster is working fantastically well. I was not involved in the installation but I also heard that it went very smoothly. Needless to say that I will always recommend Atipa to my colleagues."
"I knew you would get to the bottom of this quickly! I am so glad! Thanks so much! Made my weekend!"
Carnegie Institution for Science
Copyright© 2001-2017 Microtech Computers Inc.
All rights reserved.
Atipa Technologies, a division of Microtech Computers, Inc., is not responsible for typographical or photographic errors. Designated trademarks and brands are the property of their respective owners.