A rack fully ethernet cable'd up with IPMI switches at the top. No PDUs or power cables yet, and the boxes are just empty chassis.
Mike installing the servers
This is the area where our cage is going, Our cage is going to be about 560 square feet and contain 18 cabinets.
Some underfloor work going on to the right of our cabinet. The building doesn't have any live customers i it yet, so it's still under pretty heavy construction.
The first shipment of servers on the trucks.
Starting to stack the servers up to stage for racking (the racks still needed to arrive and be bolted down)
More progress
All 202 of the first shipment of servers.
The first 6 racks came upside down on pallets. We had to flip them over to wheel them in.
Photography is hard work. Andrew pushes while Ming and Tom let 'er down easy. Drew gets the perfect angle for the action shot.
Another customer had 15000lbs of EMC gear arrive the same morning. I guess it was 9 cabinets, mostly populated. Probably around 10 million dollars worth of gear.
More of the other guy's equipment.
Unfortunately our tall, dense 47U racks (90 inches tall with the wheels on) required us to tilt them at a 45 degree angle to get them in.
Kind of like moving a big couch through an apartment building..
except that the couch is padded and this thing is completely rigid.
Halfway done..
A view of all 6 cabinets lined up. This will probably be pretty close to their final position.
Another angle. There will be 3 more cabinets to the right, and then another row of 9 a few feet closer to the cabinet. You can see the perforated tiles on the floor, that will be part of the "cold row" blowing cold air into the fronts of the servers.
The cabling work done by AMAX looks really good. We're going to plug the hanging cables into the Force10 switches when we get them.
Some more of the construction going on in the building. A lot of wiring work is being done.
Ming and I removing the wheels so we can bolt the cabinets down
We got a "system" going so it went pretty quickly
This will probably be pretty close to the final layout of the first 6 racks. The other 12 will likely extend towards the observer.
building codes state that you need a certain amount of clearance from sprinklers, lights in the aisles, etc. All this stuff affects the layout of your cabinets.
The hot aisle and FM200 inert gas fire suppression tank. Some of the ceiling tiles will be removed to allow hot air to escape.
The view from outside. We're on the ground floor which is actually "floor 2". Seattleites call the basement "the first floor." Seattle was originally built below sea level so it flooded whenever the tide came in, so in the early 1900s when they rebuilt the city, they raised the streets to the second floor of all the buildings (Only in downtown Seattle -- our building is 10 miles away!). The convention seems to have stuck even for new buildings.
There are 3 roughly identical buildings, our colo company Fortress has a whole floor of this building and is building another floor in this building, as well as the single floor in the adjacent building (the one we're going into)
A view of the entrance from the street. The campus is called "Sabey Intergate West" for the Sabey company, who owns a lot of real estate and development in the Seattle area.
Another view from the road.
Our Force10 gear arrived. The C300 core switch is on the right; the front plate is to prevent the backplane from bending.
One of the S50n cabinet switches. 48 gig ports, 2 expansion slots for the XFP cards. We have 16 of these.
One of the XFP cards. We got 32 of these.
The documentation for the OS. Thanks to Sarbanes-Oxley, this CD (and the right to run their OS) cost us $4000.
One of the 8-port line cards. 8 XFPs. We have 8 of these
This is the component that sets the Force10 system apart from the alternative. It allows us to run full routing protocols (including inter-VLAN routing) on the core switch. There are 2 and they run in active-active redundancy.
a box of 10Gbase-SR XFP adapters. We have 128 of them.
An up-close view of an XFP.
We should figure out a more environmentally friendly way to ship our next cluster of servers. This is only about 40 servers worth of styrofoam.
80 servers ready to be racked.
Our panels. We will run about 80kVA from each one.
A stack of Force10 S50n cabinet switches, with the XFP expanders and SR XFP's installed
S50ns before the expansion cards are installed
Bringing in the circuits from the ground up
Anchoring the cabinets. These brackets go down to the concrete subfloor below the plenum.
Each junction box is a single 30 amp 208V (single phase) circuit, which will power 22 servers and a switch (half a rack)
Victory as Mike racks the first server.
I've been waiting for this for 2 weeks..
Testing the power circuits to make sure we have the correct circuits mapped to the correct breakers, and to make sure all circuits are delivering the right kind of power.
They're pretty imposing, about 7 feet tall..
The cabling in the back is mostly done, we'll probably tie up the power cables a little for tidyness but other than that these racks are completely done.
A closer shot of the wiring. It's all very clean, AMAX did a great job. The cables are the perfect length too.
The first 3 racks powered on! They're pretty loud..
The install setup..
The machines above the gap hold our log data. Damian copied it over from the storage servers and we shipped it up to Seattle with the rest of the order.
Isaac and Neal breaking down boxes
Ming landing the next set of cabinets
Isaac where he belongs.. a 30 yard (or “100 body”) dumpster.. We filled the whole thing with styrafoam and cardboard, and that was just from about 10 pallets (120 servers)
Neal and Isaac in front of some of their handiwork
These cabinets will be bolted down by the time we receive our third shipment..
The Force10 C300 core switch, fully populated. (well, not all of the XFP's are plugged in yet)
Our colo is starting to get imposing!
Datacenters are not the most ergonomically friendly environment..
We had to manually boot all of the boxes from memtest back into Solaris..
This is Isaac before his daily 8-shot Americano, 5 sugars..
Makin' Copies (of labels)
omg what are we gonna do with all this trash.. it took a while for them to get the dumpsters (we filled 6 30-yard dumpsters)
superclusters ROCK!
We did it. All of the servers are racked, cabled, and labeled.
Back of the "even side." You can see the Force10 and Cisco racks (the ones with no servers)
A plain view of all of the servers (there's a hole in the upper left where RMA machines go)
View of the back of the "odd rack," 9 cabinets full of servers in a row.
Cutting the fiber raceways.. if you're careful you use a miter box and a hand saw, but if you're in a hurry you can use a clamp bench and a band saw..
Some of the raw materials
A section of the raceways
You can see some of the downspouts on the middle section.. there is 1 per cabinet.
The main downspout in front of the core switch, about 1/4 of the way populated.
All of the raceways have covers to protect the fiber and neaten everything up.
