Disruptive Storage Workshop 3.0

Organizers: Johns Hopkins School of Public Health and BioTeam
Date: July 13 -14, 2017
Venue: Johns Hopkins Bloomberg School of Public Health, Baltimore, MD

Introduction

Johns Hopkins School of Public Health and BioTeam are pleased to announce the 3rd “Disruptive Storage Workshop” to be held in Baltimore MD on July 13-14. This 2-day workshop provides attendees with a deep dive into the mechanics of Petabyte-scale storage systems. The focus is on Biomedical and LifeScience use cases in academic and non-profit research institutions.

Investigators in life-science and Biomedical research institutions routinely generate, process and share tens, and hundreds, of Terabytes of data from high-throughput instruments. For institutions to remain competitive, their IT organizations must provision multi-Petabyte NAS and scale-out storage systems in a cost-effective and sustainable manner. “Enterprise-grade” turnkey storage solutions are expensive and divert an increasingly greater proportion of grant funds and institutional capital away from hypothesis driven research. Are there alternatives? We think so!

At the Johns Hopkins Bloomberg School of Public Health we employ an open-source, white-box strategy to build our own low-cost, low-power file systems. Few research-IT organizations currently possess the skills or expertise to exploit, let alone assess, an open-source/white-box scale-out storage strategy. Yet, our experience is that storage technology is no more mysterious or difficult than linux cluster technology. It is just different.

The workshop begins with a broad perspective on storage technology and trends in the Life Science domain. This is followed by an intensive hands-on bootcamp that teaches attendees how to build and administer Petabyte-scale storage systems based on ZFS-on-linux and Lustre-over-ZFS. We will present reference architectures, best practices, and have discussions specific to Petabyte-scale open-source/commodity storage systems with a focus on Biomedical and life science research environments. We will describe production systems (including our own) as well as solutions with varying price points and levels of availability.

In addition, we will describe business-models and financing strategies including: 1) Convincing PIs to collectively invest in an open-source/white-box storage system and 2) Applying for NIH-S10 and NSF-MRI grant funding. We expect that systems based on documented reference architectures with published performance and reliability data, together with best practices and administration tools, will help to establish credibility in the eyes of individual PIs or review panels from funding agencies.

This will be a deep and intensive dive into the devilish details.

The primary instructors and organizers are from BioTeam and the Johns Hopkins Bloomberg School of Public Health (BSPH). BioTeam’s core expertise is infrastructure for scientific computing. BioTeam brings broad and deep technical expertise in the area of large-scale storage systems for the Life Sciences. The BSPH instructors operate a department/small-institution scale HPC facility that has doubled its storage capacity every 12-18 months for nearly a decade. BSPH brings extensive knowledge and experience in building, administering and financing ZFS-on-linux and Lustre-over-ZFS storage systems. In addition to the primary instructors we will have lectures from experts from Globus and Silicon Mechanics.

The workshop targets attendees who are technically adept with Linux and who are either thinking of, or already dealing with, HPC storage systems at their home institutions. Previous attendees report that the workshop was eye-opening, allowed them to better match storage solutions to the needs of their investigators, and allowed them to interact more productively with vendors.

Previous Workshops

Disruptive Storage Workshop 1.0
Disruptive Storage Workshop 2.0

Teaching Objectives

Workshop participants will take away the following:

  1. A broad perspective on the storage landscape in the life-sciences
  2. A perspective on open-source/white-box scale-out storage
  3. Hands-on experience with installation and basic administration of ZFS-on-Linux and Lustre-over-ZFS.
  4. An understanding of disk-drive failure and tradeoffs between consumer and enterprise drives for Petabyte scale storage.
  5. An overview of reference designs (including parts lists and approximate costs) for two types of storage systems: 1)  ZFS-on-linux and 2) Lustre-over-ZFS.
  6. An understanding of  “where the bodies are buried” and how to recover from failures
  7. Encryption at rest at the Petabyte scale for sensitive and regulated data
  8. Securely sharing/exchanging 10’s of TB of data with collaborators via Globus
  9. Funding strategies for storage systems
  10. An introduction to experts in hardware, ZFS, Lustre and Globus.
  11. An introduction to a community of like-minded research-IT organizations.

Agenda

The 2-day workshop will have four sessions. Session I (Morning of July 13th) will provide a broad-brush overview of technologies and trends. There will follow an intensive three-session bootcamp suitable for technical experts with interest in developing their own solutions. A detailed agenda and tutorial material will be found on the schedule page.

Venue

The workshop will take place in l (room W5030) at the Johns Hopkins Bloomberg School of public Health (615 N. Wolfe St, Baltimore MD, 21205). For directions to the Wolfe St. building please click on either of the following links

  1. The “official” directions
  2. Google Maps

If you are driving, we recommend that you park in the Washington St. Parking lot on the NE corner of the intersection of Wolfe and Washington streets. The school is on the SW corner of the same intersection.

We do not have a hotel with a block of reserved rooms, however, the links below should be useful.  Airport hotels are inexpensive, but you would need to rent a car. Downtown or Fells Point hotels are a short taxi or uber ride to the medical campus and they are in “fun” neighborhoods.

  1. List of Hotels that provide discounts if you mention “Johns Hopkins.”

Registration

Please register here.