Update: I now have a newer 4th generation EXPTA server build using the latest components, including the new Intel I5-4570S Haswell quad-core processor and faster SSD drives. Check it out here!Over a year ago I wrote an article detailing how to build a Blistering Fast Windows Server for about $1,000 USD. At that time "Windows Server 8" hadn't even been released yet, but I wanted to build a server that would work with "future generations" of Hyper-V. The article proved to be extremely popular and paved the way for many fellow technologists to build their own lab servers.
Now that Windows Server 2012 has been out for a while I wanted to update that article to incorporate newer technologies, like 3rd generation Intel processors and faster DDR3 RAM. I also made some tweaks to my initial server over the year, adding another SSD drive for active VMs and enabling sleep mode on my physical storage hard drive to save more power. I'm including those items in this build, while maintaining the same price point as over a year ago.
Lessons Learned
I modified a few things since I built the original lab server I documented in January 2012. Here are the lessons I learned:
- If RAM is king, IO is queen. The two most important things for a Hyper-V 2012 server are RAM (VM capacity) and IO (VM performance). IO becomes even more important as you add more concurrently running VMs, which you can easily do with 32GB of RAM!
- SSD = IO. My original design used a single SSD for the operating system and binaries. I soon learned that VM performance was pretty poor running off a traditional mechanical hard drive, even though I was using a fast SATA III 6Gbps drive. I ended up buying another 250GB SSD drive to host my active VMs.
- CPU isn't as important as I thought. It's important to have enough cores to share with your VMs, but most of the time my CPU is idling at 10% utilization even with 8 VMs running simultaneously.
- Deduplication is amazing! You can increase the VM density on an SSD drive using Windows Server 2012'd built-in deduplication feature.
- You can never have enough SATA III ports. My first build used an Intel motherboard with two SATA III 6Gbps and two SATA II 3Gbps ports. I ended up having to buy another SATA III controller when I added the other SSD drive. Better to have at least 4 SATA III ports to begin with.
My Design Requirements
This build has an emphasis on cost. Even though my budget is the same as the earlier build, I have to make it work with two SSD drives instead of one.
- Minimum of 4 cores
- Windows Server 2012 capable. Hyper-V for Windows 8 requires hypervisor-ready processors with Second Level Address Translation (SLAT).
- 32GB of fast DDR3 RAM
- Must support SATA III 6Gb/s drives
- Must have USB 3.0 ports for future portable devices
- Low power requirements
- Small form factor
- Budget: Under $1,000 USD
As before, the RAM requirements drove most of this design. Interestingly, I found that the newer technologies (3rd generation Intel Core I5 Ivy Bridge and DDR3 1600 RAM) actually cost less than the 2nd gen I5 and DDR3 1066 RAM in my last build.
Unlike last year's build, I discovered that Amazon usually has the lowest price for everything. This makes it a lot easier to order and receive since all the components come from one place. This should also make it easier for my European friends since they can source it all from Amazon, as well. Another big bonus is that I have Amazon Prime which gives me free 2-day shipping on all the components. I could even choose to spend $3.99 more to get it next day! I love this service!
Here's the entire parts list for this server:
Unlike last year's build, I discovered that Amazon usually has the lowest price for everything. This makes it a lot easier to order and receive since all the components come from one place. This should also make it easier for my European friends since they can source it all from Amazon, as well. Another big bonus is that I have Amazon Prime which gives me free 2-day shipping on all the components. I could even choose to spend $3.99 more to get it next day! I love this service!
Here's the entire parts list for this server:
Quantity | Item | Description |
1 |
Intel Core i5-3470S Quad-Core Processor 2.9 GHz 6 MB Cache LGA 1155 -
BX80637I53470S This is a 3rd generation Ivy Bridge Intel processor. It includes Intel HD 2500 graphics and runs at a low 77W. 3 year limited warranty. |
|
1 |
AS Rock PRO4-M LGA1155 Intel H77 Quad CrossFireX SATA3 USB3.0 A V GbE
MATX Motherboard H77 I chose this LGA 1155 Micro ATX motherboard over Intel because it has 4x SATA3 and 2x SATA2 connectors. It also uses the Intel H77 chipset, supports RAID 1, 5 and 10, has 4 PCI-Express slots, USB 3.0, and has a great BIOS. See the video below. 3 year limited warranty. |
|
2 |
Corsair Vengeance 16GB (2x8GB) DDR3 1600 MHz (PC3 12800) Desktop Memory
(CMZ16GX3M2A1600C10) 240 pin dual channel RAM with built-in heat spreaders. Lifetime warranty. Latency is 10-10-10-27. Each package contains 2x 8GB sticks (16GB). Be sure to buy two packages. |
|
1 |
Kingston SSDNow V200 128GB Bundle SV200S3B7A/128G SATA3 SSD used for the Windows Server 2012 operating system. The package includes the drive and SATA3 cable, an external enclosure, and cables. 3 year warranty. |
|
1 |
Samsung MZ-7TD250BW 840 Series Solid State Drive (SSD) 250 GB Sata
2.5-Inch SATA3 SSD used for active VMs (the VMs I normally have running, like a domain controller, Exchange servers, Lync servers, etc.). Super-fast drive. 3 year limited warranty. |
|
1 |
Kingwin 2.5 Inch to 3.5 Inch Internal Hard Disk Drive Mounting Kit Metal mounting kit for 2.5" SSD drives. Holds two SSD drives, stacked on top of each other. |
|
1 |
WD Green 2 TB Desktop Hard Drive: 3.5 Inch, SATA III, 64 MB Cache -
WD20EARX 2TB Western Digital Green (low power) SATA3 drive. Used for storing ISOs, seldom used VMs, base images, etc. I usually configure this drive to sleep after one hour to save even more power. 2 year warranty. |
|
1 |
Lite-On Super AllWrite 24X SATA DVD+/-RW Dual Layer Drive - Bulk -
IHAS124-04 (Black) Great quality DVD burner. It's cheap, too. I connect this to one of the SATA2 ports on the motherboard. 1 year limited warranty. |
|
1 |
SATA Data Cable (2pk.) I need 4x SATA3 cables for this build. The ASRock motherboard comes with a black one and the Kingston 128GB SSD comes with another read one. |
|
1 |
Rosewill 40-In-1 USB 2.0 3.5-Inch Internal Card Reader with USB Port /
Extra Silver Face Plate (RCR-IC001) This is just a handy cheap addition. It slides into the floppy drive tray of the case and adds another USB 2.0 connector, SD card reader, and lots of other reader slots to the front of the computer. |
|
1 | APEX TX-381-C Black Steel Micro ATX Tower Computer Case USB/Audio/Fan Mini ATX tower case for Micro ATX motherboards, like the ASRock. It includes a carrying handle and 2x USB 2.0 ports and audio jacks under a small door on top of the case. It comes with a fairly quiet 80mm rear case fan and clear instructions. |
|
1 |
Rosewill Stallion Series 400W ATX 12V v2.2 Power Supply RD400-2-SB Dual 12V rails. Nearly silent 120mm fan and mesh cable sleeving. Includes 4x SATA power connectors and 1x PCI-Express. 1 year limited warranty |
Click the video below to hear a description of the parts I ordered for this beast:
It took about 90 minutes to assemble everything and take these pictures. The following slideshow shows how I put it all together:
Once assembled, I updated the BIOS online (very cool - see the video below) and installed Windows Server 2012 Datacenter Edition. Installation took only 4 minutes, 50 seconds! Amazing.
Windows Server 2012 recognized all but two of the computer's components, but some required updating so Windows Server can use their advanced capabilities. Do NOT install the drivers using the setup program on the included ASRock H77 Pro-4M DVD. The ASRock setup programs will BSOD the server since they are written for a different OS. Instead, open Device Manager, right-click the following devices, and update the driver software using the ASRock DVD.
Here are the devices that need to be updated, in this order:
System devices
- Xeon(R) processor E3-1200 v2/3rd Gen Core processor DRAM Controller - 0150
- PCI Express Root Complex (Becomes "PCI bus". Requires a restart)
- Intel(R) H77 Express Chipset LPC Controller - 1E4A (Requires a restart)
- Intel(R) 7 Series/C216 Chipset Family SMBus Host Controller - 1E22
- Intel(R) 7 Series/C216 Chipset Family PCI Express Root Port 8 - 1E1E (Requires a restart)
- Intel(R) 7 Series/C216 Chipset Family PCI Express Root Port 6 - 1E1A
- Intel(R) 7 Series/C216 Chipset Family PCI Express Root Port 1 - 1E10
Universal Serial Bus controllers
- Standard Enhanced PCI to USB Host Controller (Becomes "Intel(R) 7 Series/C216 Chipset Family USB Enhanced Host Controller - 1E26")
- Standard Enhanced PCI to USB Host Controller (Becomes "Intel(R) 7 Series/C216 Chipset Family USB Enhanced Host Controller - 1E2D")
Other devices
- Unknown device (Becomes "Intel(R) Smart Connect Technology Service")
Sound controllers
- High Definition Audio Device (Becomes "Realtek High Definition Audio")
- High Definition Audio Device (Becomes "Intel(R) Display Audio")
Network adapters
- Realtek PCIe GBE Family Controller
IDE ATA/ATAPI controllers
- Standard SATA AHCI Controller (Becomes "Intel(R) 7 Series/C216 Chipset Family SATA AHCI Controller". The DVD drive will probably change drive letters after this update.)
- Standard SATA AHCI Controller (Becomes "Asmedia 106x SATA Controller". This one is tricky. Restart and press F8 to boot in Safe Mode. Restart again into normal mode. You will now see new "ATA Channel 0" and "ATA Channel 1" controllers.)
Display adapters
- Microsoft Basic Display Adapter (Becomes "Intel(R) HD Graphics". The screen flashes during installation.)
Install Intel Management Engine Components from the ASRock DVD
- Run <DVD Drive>:\Drivers\ME\Intel\(v8.1.2.1318_1.5M)\Setup.exe
- Accept the Intel Manageability Engine Firmware Recovery Agent license agreement
- Check for updates. This takes a few minutes.
- This will fix the unknown PCI Simple Communications Controller device.
I also recommend that you update the Samsung SSD 840 firmware, which includes better TRIM support:
- Download and install the Samsung Magician 4 software.
- Click Firmware Update and Update. Reboot to finish the firmware upgrade.
Finally, run Windows Disk Management to initialize, format and label your Samsung 250GB SSD and Western Digital 2TB drives.
Here's a video of the Windows Server 2012 Hyper-V server in action, showing its sub-10 second boot time and VM performance:
I hope this article, slideshow and videos are helpful to you in your quest to build the perfect Hyper-V lab server. This is a great investment in your IT career!
Special thanks to my ExtraTeam colleague, Aman Ayaz. It was his need for a new Hyper-V lab server (and his Visa card) that made this article possible. :)
awesome! I am SO doing this.
ReplyDeleteI would like to thank Jeff for scoping out this project. I think this is a very good investment for technologists seeking ROI without compromising on reliability. There were industry leaders who pointed me towards a quote of more than $2,500 from amazon for my requirements, well in the end it was Jeff's experience in (reliable and cost effective) components and their specification that lead me to get my solution well under $1,000.
ReplyDeleteThank you very much once again Jeff!
I solved the I/O problem by putting four 1TB SATA drives in a RAID10 in my lab host at home.
ReplyDeleteI've already got my asrock m/b ordered before you cause a sellout. How is it that you can run 6 VMs plus the host (I also read your dedup blog and will try that too) with only 4 threads on your processor? It this a new feature of 2012 HyperV
ReplyDeleteYou don't need to dedicate cores to each VM - they can share cores as long as they're not maxed out. This has been the case since the beginning of Hyper-V.
DeleteCongratulations on getting started! Let me know if you have any questions.
Very Nice! Thank you for demoing the 10 second boot of Server 2012 and a total of 22 second to start using Server Manager. Yes, this really gets the point across when doing a side by side comparison of the same OS loading from a mechanical drive. WOW, what a difference to Solid State Drives makes!!!
ReplyDeleteat the spec you hve provided, how many VM's do you think you can run?
ReplyDeleteLooking at upgrading my MCITP Exchange to the MCSE, but i like to play with multiple scenarios and even branching out to Lync 13 aswell...
and i will be the first to admit that if the wife see another computer in the house i think she will kick me out :-P so for me its all about consolodation!
I normally run a Windows Server 2012 Domain Controller, 1x Exchange 2010 multi-role server, 1x Exchange 2013 multi-role server, 1x Exchange 2013 CAS, 1x Exchange 2013 Mailbox server, and two Kemp virtual load balancers 24x7 off SSD. the performance is incredible. Be sure to read and implement my WS2012 deduplication article to improve VM density on the SSD.
DeleteThanks Jeff! As always, your timing is impeccable! I need a new lab server.
ReplyDeleteUnfortunately, many of the items are "Currently, item can be shipped only within the U.S. ", so no joy for European colleagues :(
ReplyDeleteWhat online stores do folks use in Europe for computer parts? I may work up a parts list for European technologists.
DeleteTime to report back. I got the case, m/b and ram as you spec'd and used other components I had in stock for my build. Updated the drivers per your list but find that if I install the ASMEDIA driver, the computer will not boot. The version it installs is 1.3.8.0 dated 7/18/12. Everything else went fine. Installed 2012, 2012e, 2008r2, Win7 & Win8 as VMs, and applied updates. I had 92 GB free on the second SSD. Installed the dedup feature you mentioned elsewhere, ran it, and now I have 150 GB free. BTW, I configured all VM with 3 Virt Proc and ran two updates at a time and never had more that 20% processor utilization. Thanks for all the training.
ReplyDeleteThanks for the update, Ken! Glad the server is working well for you.
DeleteIt's a nice setup Jeff. Right now I'm studying for another certification and on the same time preparing docs for MCSA Windows2012. Your post helps and points me into right direction how to build the server for cheap. Actually I would use a WD RED instead of Green as it is quiet and fast comparing to Green. I have both Green and Red working 24/7 in the same server and can hear Green working and no sound from Red =)
ReplyDeleteAgain, nice post and waiting to see more details about your home lab. It motivates other followers to build the same ;)
>>What online stores do folks use in Europe for computer parts?
ReplyDeleteebuyer.com is a great source of quality components in the UK.