We got a unique opportunity yesterday to get a tour of San Diego's Supercomputer Center located on the campus of the University of California, San Diego. We met with Ross Walker, NVIDIA CUDA Fellow and Assistant Adjunct Professor at UCSD who gave us a tour of the San Diego Supercomputer Center.
If you don't remember, the San Diego Supercomputer Center is the place where the largest supercomputer in the world exists that utilizes the largest flash array in the world. This supercomputer is known as Gordon (building on the name Flash Gordon, but they obviously couldn't use that name). Gordon's specs are as follows, it features 1,024 dual-socket Intel Sandy Bridge nodes, each with 64 GB DDR3–1333 memory with over 300 TB of high performance Intel flash memory SSDs via 64 dual-socket Intel Westmere I/O nodes. It also has large supernodes of memory capable of delivering over 2 TB of cache coherent memory. Data Oasis high performance parallel file system with over 4 PB capacity and sustained rates of 100 GB/s (the most impressive spec of this supercomputer).
We learned a lot of things about how these supercomputers are funded, set up, used and continually used by researchers in order to solve today's biggest problems. Ross' team, however, is specifically focused on GPU computation and utilizing GPUs to accelerate their biological simulations to help solve the biggest problems in today's world. His team are working on deploying GPUs inside of supercomputers and already have utilized some desktop gaming graphics cards in their own systems. They have a cluster of computers that they have put together into one rack with each computer utilizing four GTX 680s per computer and by joining these computers together with their GPUs, Ross and his team are able to gain incredible acceleration with very little power and heat.
Here are some pictures of our tour...
First, we saw Gordon.
A plaque installed on Gordon to commemorate the sudden and unfortunate passing of it's father, Dr. Allan Snavely who suddenly died of a heart attack shortly after the supercomputers completion. Many in the San Diego Supercomputer Center still hurt from their loss.
Below is one of the many racks of Gordon, below, you can see one of the racks filled with disks that include lights to show drive activity.
This is what the backside of each rack of the Gordon supercomputer looks like. Note how small each system is and how many systems fit inside of a single rack. As someone who is slightly OCD, the neatness in this photo is not only comforting but beautiful.
Looking down at the Gordon supercomputer from the opposite end, looking at and appreciating its sheer scale and impressiveness.
After we saw the famous Gordon supercomputer and checked out it's internals, Ross took us over to his own team's racks to show us what kind of hardware they were working with. As we stated earlier, they were working with GPU systems. Originally, the codename for the cluster was to be Dante in anticipation of the amount of heat generated by running dozens of GTX 680s at full utilization. However, once they had actually turned the system on and broken it in, it was significantly cooler than they had anticipated. The systems that they had built by EXXACT were well below what they were expecting thermally.
And here is the back side of those EXXACT systems, with the 32 GPUs clearly visible.
Here are some more assorted shots from the remainder of our visit.
One of the old terminals.
An older system with terabytes of RAM.
The Infiniband switches that interconnect the different nodes of the supercomputers together.
Uhoh, It looks like I found a loose cable. (Don't worry, we didn't break anything it wasn't even connected to anything)
They still use mainframes (IBM) at the San Diego Supercomputer Center because some systems still will only work with mainframes and some data can only be stored or accessed from it. So, there is a small bastion of old technology that still remains as many of the tape drives have been removed as they simply couldn't keep up.
Last but not least, we have the Halon fire extinguishing system which is designed to stop any fires from burning down the entire place. The problem, though, is that Halon is illegal to use due to environmental laws that passed after its installation. Any system that already has it installed can only use it once. Once the system is discharged, it will all have to be ripped out, moved, and upgraded to be compliant with environmental laws.
As a bonus, Ross took us to his hardware grave lab where some of his old (some still operational) hardware goes to die. He even showed us some of the ghetto mods that he and his team were able to create using Nvidia GPUs and some interesting cooling solutions.
The GPU below worked for quite some time until the heat from the graphics card burned out the bearings on the fan. Nevertheless, it still worked for quite a while.
Then came our favorite mod, a frankenstein Tesla GPU with a Corsair self-contained watercoolling system that is generally intended for CPUs, ziptied to the GPU on the board.
© 2009 - 2013 Bright Side Of News*, All rights reserved.