USC Launches Powerful Cloud Archive

USC News

USC is offering cloud archiving and digital archive technology pioneered by the USC Shoah Foundation Institute to scholars and archivists everywhere.

Researchers, galleries, museums and archives will be able to keep their collections safe and within reach of their keyboards through the state-of-the-art cataloging, digitization, search and indexing technology originally created to preserve and distribute the testimonies of 52,000 survivors and witnesses to the Holocaust.

The USC Digital Repository can digitize any holdings not already digital, store all holdings in the cloud and provide a patented search and indexing interface. This solution not only safeguards the holdings, but also allows researchers and institutions to provide easy access to colleagues, students and other audiences around the world.

“The USC Digital Repository was born out of the need to preserve something as important as Holocaust testimonials,” said Sam Gustman, the executive director of USC Digital Repository and associate dean of the USC Libraries. “Now this technology, and the thoughtful systems behind it, is a resource that others can use.”

USC takes cloud archiving to the next level, as it is the first to combine academic-quality archival abilities and keyword-driven database search with the digitization power and speed of a supercomputer, Gustman said.

“USC is the best place to bring digital collections – and if they are not digital yet, we will make them digital,” said Gustman, also the chief technology officer for the USC Shoah Foundation Institute who led the effort to protect, store and share these vital pieces of history for the last 18 years.

The system was developed by the Survivors of the Shoah Visual History Foundation, established in 1994 by Steven Spielberg to gather video testimonies from survivors and other witnesses of the Holocaust.

Creating the repository took hundreds of hours of planning and implementation, thousands of hours of manual labor and millions of dollars.

In January 2006, the Survivors of the Shoah Visual History Foundation became part of the USC Dana and David Dornsife College of Letters, Arts and Sciences, where the testimonies will be preserved in perpetuity.

The USC Digital Repository unites the expertise and resources of the university: the USC Libraries, where the USC DR is based, provides imaging service and archival experience; The USC Shoah Foundation provides mass conversion, cataloging and media preservation; and USC Information Technology Services provides the processing power of one of the nation’s most powerful academic supercomputers.

“Preserving and making accessible primary research materials is fundamental to our role as an academic research library,” says Catherine Quinlan, dean of the USC Libraries. “The USC Digital Repository allows us to leverage the expertise of library faculty and staff as well as that of our partners across campus.”

The archiving system allows users to perform keyword searches that navigate directly to a segment of video clip. The integrity of the files is evaluated through industry standard algorithms and other state of the art preservation techniques.

“The USC Digital Repository is uniquely positioned to provide professional expertise and technological resources to allow customers to cost-effectively create, manage and preserve sophisticated digital collections,” Gustman said. “We make things searchable and safe.”

The USC Digital Repository has a current capacity of 40 Petabytes for digital storage,  8 Petabytes of which has been filled to date.

The repository digitizes collections through the university’s Center for High-Performance Computing and Communications. The center’s supercomputer, the seventh fastest academic supercomputer in the country, has two Linux clusters with more than 43 terabytes of combined memory and more than 1.3 petabytes of temporary disk storage.

This network can be accessed via multiple high-capacity interfaces, allowing for fast transfer of massive amounts of data.

The USC Digital Repository is connected to the Los Nettos regional network, headquartered at USC. Los Nettos provides redundant, reliable, and high-speed  network connectivity.  Offsite backup storage is provided as well through cloud archiving.