Apr 252013
 

X58 LogoThere is something that got on my nerves just a tiny little bit. And that’s the memory limit of socket 1366 processors. Not Xeons on server chipset platforms, mind you, but regular Core i7 on X58. According to Intel specification, the IMC (internal memory controller) of the CPU will only support 24GB of memory, and the same is stated in X58 chipset specifications. On the best of desktop mainboards there are 6 DIMM sockets for the triple-channel setup, and 8GB modules are actually possible. Just not the full amount of 6×8=48GB, so you should only be able to plug in either 6×4GB or 3×8GB for a 24GB maximum. Or so Intel specs tell us at least.

Still, I found a very few people on the web who claimed that they tried 48GB and that they had it working just fine. Some others said they tried and got no POST at all (black screen on boot), some tried on my own mainboard (Edit: An ASUS P6t Deluxe) and succeeded while others claimed it’s impossible. And there were also people who said that it did work, but they only got 32GB shown in BIOS and within their operating system, Windows 7 in that case. That was some Gigabyte mainboard if I remember correctly, but don’t quote me on this.

The thing is, these Core i7 desktop processors actually have a 36-bit address bus. With a single byte being the smallest addressable unit, that means we can address 236 bytes, which happens to be a total of 64GB. For comparisons sake, current Xeon processors have a 40-bit address bus, which enables those chips to address 1TB of memory, or 2TB in a 2-processor setup based on a NUMA architecture. So basically, there should be no actual 24GB hard limit.

Since I hate changing platforms (mainboard, resetup etc, it’s just a pain), an 48GB upgrade would mean that I don’t need X79 so badly anymore, so I decided to give it a shot. X79 would be the direct successor with 8×8GB=64GB maximum on a quad channel setup for socket 2011 CPUs. However, interesting CPUs will only come in the form of Ivy Bride E, which may even be a disappointment should Intel choose not to release an octocore, but only another hexcore. So, would it be possible to prolong the life of X58 just once again for memory intensive applications? Well, look for yourself:

In case you’re asking yourself why I picked Crucial DRAM, well, as hardware.fr shows, it’s [RMA rate is pretty low] and it was quite a lot cheaper than the Kingston HyperX alternative that I would actually have preferred. Memory is cheap alright, but at that amount it’s still a lot of cash, and I managed to save a 100€ here when picking Crucial. Besides, I never had Crucial RAM anyway, so I’m giving them a chance. ;)

Now, all that needs to be proven is that this setup is not just possible, but also stable! I’ll keep using this and report instabilities in case I do encounter any. As you can see above, I picked CL10 (10-10-10) latencies for the memory. The two combined 12GB triple kits from Mushkin that I had used before were also CL9 by specification, but when combined to 24GB they would only be stable at CL10. So I left it there, as I have just combined three 16GB dual kits to a whopping 48GB here. I’ll better play it safe with the latencies.

But well, there you go. 48GB on X58 with a regular Core i7 processor! It CAN be done after all!

Edit: After filling the memory for several times, using tsMuxer as well as MKVmerge and my own backup scripts including robocopy for backups, confirming data integrity after flushing using diff, it actually seems as if this was really stable. I continue to stand surprised. Easier than I would’ve thought.

Edit 2: Now, over three weeks later, the setup is still going strong. I have transcoded and multiplexed (to MKV) a lot of Blu-Rays, played lots of games, filled the memory with NTFS write & read caches many times. Current system uptime stands strong at 18 days without any single problem for the time being, having my CPU load at 100% for days now with memory filled up all the way. I’d say this actually works!

Edit 3: As a final confirmation, the machine has now run for almost 5 months straight, 24/7. Dozens of Blu-Rays have been transcoded, many games played, many files copied and modified, lots of images processed. Not a single hiccup!

Edit 4: Finally, it failed, a few months before today, which would be 2016-08-12. That’s not the platforms’ fault however, it was just one of the memory modules failing (I have confirmed that by testing them on several other mainboards). Hm, back to Kingston after all? ;)

CC BY-NC-SA 4.0 Intel X58 chipset and 48GB RAM: Impossible? No! by The GAT at XIN.at is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

  261 Responses to “Intel X58 chipset and 48GB RAM: Impossible? No!”

  1. hoi x58lovers
    as i am on a R3G with 6x8gb with a almost maxed out ram usage
    i stumbled across these RAM modules:

    CT2K204864BD160B
    DDR3 PC3-12800 • CL=11 • Unbuffered • NON-ECC • DDR3-1600 • 1.35V • 2048Meg x 64
    DDR3L 240-pin DIMM
    Desktop modules that operate at speeds up to 1600 MT/s and have a CL11 latency. It is dual voltage and can operate at 1.35V or 1.5V. It is Unbuffered and is non-ECC. It conforms to the industry standard UDIMM layout of 240 pins and is compatible with computers that take DDR3 UDIMM memory.

    is there a chance that this will work on the R3G?

    • Hello Tomster,

      Short answer: No, with a confidence of about 99.99% (and I mean it).

      Long answer: The IC density on those modules exceeds what’s supported by the CPUs integrated memory controller by a factor of four. 2048MiB x64 essentially means eight 2GiB chips. That should be a single-rank module with all 8 chips on just one side.

      So, the issue here is, that the individual chips on that module are 2GiB large, while the maximum supported officially would be 512MiB.

      That’s what I’ve been talking about a lot somewhere below. The maximum according to spec would be 512MiB × 8 bit × 2 ranks, so 512MiB × 8b × 2R = 8192MiB or 8GiB if you want to populate all channels, as the maximum total number of ranks per channel is 4 for unbuffered memory. This would populate all 4 ranks per channel, while also maxing out the IC density.

      Another user here has [already tested] modules organized as 1024MiB × 8b × 2R = 16GiB, so twice the specified IC density, and the system did no longer boot up, no POST even. So exceeding IC density seems to be impossible, hence my high confidence that this won’t work.

      According to the specs, Reg. ECC can be driven at a higher rank count (4 per module instead of 2, 8 total per channel), but so far nobody got Reg. ECC to work on X58 in a stable fashion. And to my knowledge, nobody has even tried the larger 16GiB Reg. ECC modules.

      I only see two remaining chances for getting to 96GiB of RAM; The first is to try and break the rank limit for unbuffered DDR-III by using memory organized as 512MiB × 8b × 4R, so quad-rank RAM. But memory like that seems to be hard to find (does it actually exist?), and it’d still be a game of chance… with a high risk as well. :(

      The second is to find that magical something that would make Reg. modules work on X58 without freezes and crashes. Because with that we could double the ranks (as long as you have a Xeon, not an i7), problem solved! But nobody found any clues yet, and as time progresses, less and less people are interested in X58… :|

      • thank you grand-admiral!
        i agree, the densitiy is the dealbreaker here. i only wanted to clear things up,
        as it seems i need a new setup in the near future. helped a lot!

  2. Hey, I just picked up an old Dell T3500, gonna drop in an X5675 and some workstation graphics cards. So far the entire rig (tower, 2 graphics cards, and cpu) have cost me $375US. My primary purpose is to run gpu passthrough to a VM with modeling software. That is why I’m interested in the RAM upgrade. Perhaps I’ve missed it, and forgive me if so, but when you were installing your 48GB did you bump up the voltage, or did you just plug and go? I’m not interested in overclocking, excluding the RAM everything will be in stock configuration. Interestingly enough, the RAM will end up costing more than the rig itself.

    • Good evening Jerry,

      For the two Crucial BLS8G3D1609DS1S00 3-module DDR-III/1600 kits running at DDR-III/1456 speeds, I had to up the V(DIMM) to 1.65V. At that speed and with those timings set at either 10-10-10 or 11-11-11 (I don’t remember exactly), everything was dead stable even under massive loads for weeks. However, there was a slight possibility of encountering DRAM-related kernel crashes at stock settings. But who knows? One of the modules has died by now, so maybe it was just bad memory after all.

      The individual Kingston HyperX Fury (KHX-1866C10D3/8G) modules I got now (DDR-III/1866) are running at default settings, configured by SPD EEPROM at DDR-III/1333 speed with latencies set to 8-8-8-24-2T (also by SPD). So no overvolting, no overclocking! Rock stable for many months now.

      On a second machine with an ASUS P6T Deluxe V2 instead of P6T Deluxe I’ve had a similar experience with Kingston memory, individual KVR1333D3N9/8G modules in that case. No tweaking needed, they’ve just run at 9-9-9-24-2T in my Linux workstation since almost a year now. No issues whatsoever.

      Main workloads were x264 & x265 video transcoding, sometimes heavily in parallel, virtualization (Windows, Linux and UNIX VMs using VBox and VMware Player) and data compression (7z with LZMA2 and huge dictionaries for data deduplication). Less significant workloads were audio transcoding and games.

      Hope this helps! :)

      • Thanks for the info! I went with the crucial modules as they were over $100 cheaper (per 6 as of march 15th 2017) than the kingston. When they arrive I will post an update on the install and modification of any settings to get them to run reliably.

        • Yeah, I remember my Crucials also being the cheapest option back then, and the RMA rate reported by hardware.fr was at an all-time low for Crucial, but I still got a bad one. Well, it can happen I guess, even if it’s rare.

          Maybe yours will even work without the need to change any settings in the BIOS (I’m still guessing my memory was bad from the very start).

          Looking forward to seeing your results! :)

          • Well, I did it. I am using a Dell T3500. I plugged in 48GB 1600mhz ballistix RAM. It defaulted to 1066mhz in the BIOS which means it would be running at 6-6-6-16 but running i2c-tools it gives the following, and the timings seem to line up with DDR3-1600mhz:

            —=== Memory Characteristics ===—
            Maximum module speed 1600 MHz (PC3-12800)
            Size 8192 MB
            Banks x Rows x Columns x Bits 8 x 16 x 10 x 64
            Ranks 2
            SDRAM Device Width 8 bits
            Bus Width Extension 0 bits
            tCL-tRCD-tRP-tRAS 9-9-9-24
            Supported CAS Latencies (tCL) 11T, 10T, 9T, 8T, 7T, 6T, 5T

            —=== Timing Parameters ===—
            Minimum Cycle Time (tCK) 1.250 ns
            Minimum CAS Latency Time (tAA) 11.250 ns
            Minimum Write Recovery time (tWR) 15.000 ns
            Minimum RAS# to CAS# Delay (tRCD) 11.250 ns
            Minimum Row Active to Row Active Delay (tRRD) 6.000 ns
            Minimum Row Precharge Delay (tRP) 11.250 ns
            Minimum Active to Precharge Delay (tRAS) 30.000 ns
            Minimum Active to Auto-Refresh Delay (tRC) 48.750 ns
            Minimum Recovery Delay (tRFC) 260.000 ns
            Minimum Write to Read CMD Delay (tWTR) 7.500 ns
            Minimum Read to Pre-charge CMD Delay (tRTP) 7.500 ns
            Minimum Four Activate Window Delay (tFAW) 30.000 ns

            For anyone else running a Dell T3500 I am running BIOS revision A10. Later this week I will be receiving an Intel X5675 hexacore xeon. If my BIOS revision supports it then I will plug in and go, otherwise I will update to A17 and cross my fingers I don’t lose my RAM compatibility. This machine blows my M6500 out of the water.

            Screenshotfrom2017-03-2210-16-22

            • Nice one! :)

              However, dmidecode from the i2c-tools is by no means a reliable tool when it comes to detecting CAS-RAS2CAS-RAS etc. I’ve seen all kinds of misdetections with that tool. To make sure, you’ll need to enter your system BIOS and check it out there.

              I still find it weird that we don’t have a sure-fire way to detect active memory latencies like with CPU-Z on Windows, but uh…

              Or did you actually compare the effective latencies/nanoseconds to other boxes?

              • I didn’t compare anything. I googled CAS Latencies went to the wikipedia page and referenced the time in nanoseconds to the latency and it showed nanoseconds that corresponded to 1600mhz DDR3. Of course the BIOS did state I was running 1066mhz which makes sense as my Xeon W3550 doesn’t support anything greater than 1066mhz.

                I upgraded my BIOS from A10 to A17 today and installed the xeon hexacore, X5675. This offers VT-d which I need for pci passthrough and hardware AES decryption which is great for my encrypted HDDs. The X5675 supports 1300mhz RAM and the BIOS reports that I am running 48GB at 1300mhz.

                I ran prime95 on the “CPU stress” test for about 10 minutes and had no errors. I will probably let it run overnight on “blend” to stress the RAM and CPU and if it comes back clear in the morning I will say this experiment was a success. For roughly $600USD I was able to put together a dual GPU, 3ghz hexacore, 48GB machine. I’m happy I didn’t try to buy a new workstation, LOL.

                Screenshotfrom2017-03-2414-11-09

                • Effective latencies depend on CAS (etc.) as well as clock rate though. CL10 at DDR-III/1600 is faster (less ns) than CL10 at DDR-III/1333, because CAS etc. are measured in clock cycles. So higher clock = faster latency.

                  Since your results “look like DDR-III/1600” at 1066MHz data rate, well, let’s see:

                  CL6 / 533333333Hz = 0.00000001125s = 11.25ns
                  CL9 / 800000000Hz = 0.00000001125s = 11.25ns

                  Mystery solved, DDR-III/1600 CL9 and DDR-III/1066 CL6 have the same effective CAS latency. Your memory was indeed running at DDR-III/1066 CL6! The formula is always the same:

                  CL / physical clock rate in Hertz (half the data rate for DDR) = seconds
                  seconds × 109 = nanoseconds

                  I’d love a # demidecode -t memory that actually works. ;)

                  Edit: Meaning, I want it to report not just correct effective nanoseconds, but also correct, currently active CAS etc. values… In any case, I think you’re gonna be really happy with your upgraded X58 workstation! My Linux machine at work… I couldn’t live without those 48GB anymore. Certain software tests require me to fire up different 16GiB VMs (that actually really consume that whole load of memory), FreeBSD, OpenBSD, Solaris, Windows, MacOS X, all kinds of stuff. And more VMs for smaller tests. Wouldn’t be possible to work like I do without those 48GB anymore. :)

                • Ah, I’ve got one question: What Hypervisor are you using for PCI passthrough? QEMU/KVM?

                  • Yep. I had some issues with windows so I upgraded to Win10. I manually installed the updated OVMF UEFI BIOS from the web because the OpenSUSE one was causing win10 to BSOD, which means I had to edit my qemu.conf to show the location of the file. I deleted all unnecessary virtual hardware as well, so no spice display, no QXL video card, no cdroms etc…I was thinking about attaching at least one monitor to the card but I have been using RDP with great success (it’s all local over ethernet) with some minor tweaks. I did have to upgrade my Win10 license as well because it wouldn’t let me use more than 1 core which caused a lot of issues.

                    • Interesting. I have no experience with Qemu/KVM yet, which is why I’m asking. So you can have say 1 graphics card rendering your Linux desktop, and a second one that you pass through to the VM of your choice? Because this is something I’ve wanted for years now. ;) Kind of a wet dream, to have a Linux or UNIX workstation, and just do the gaming on a Windows VM that I can hibernate afterwards.

                      Ah and that thing about the CPU cores: Wasn’t it rather because you passed your cores to the VM as vCPUs instead of as “cores” or “threads”? (-smp cores=1,threads=1,sockets=2 instead of -smp cores=2,threads=1,sockets=1?) Because lower Windows 10 editions are limited to one physical CPU package/socket, higher ones to two physical CPU packages/sockets. But both can do a lot of cores. Even Windows 10 Home should support 32 cores for its 32-bit version and 256 cores for its x64_64 version, as long as its single socket… :-?

                      Or am I making a mistake somewhere?

                      • Exactly that. You end up isolating your PCI(e) card of choice, doesn’t need to be video, as long as your processor supports VT-d, and then you set it up as a PCI device in Qemu/KVM. A lot of stuff on the net talks about using pci-stubs when VFIO is now the standard so it was irritating sifting through that. There’s specific tweaks you have to do if you use nvidia cards since the firmware is set up to detect virtual machines and shut down the card, but since I’m running AMD I didn’t have to worry about it.

                        Another thing I didn’t even realize at the time is you can actually use Steam In Home Streaming on your Windows VM and never even have to look at the system. You can just have Steam autostart then open Steam for linux and pick your game. Best of both worlds I think.

                        As far as the CPUs, it could could have been, I updated my license and pinned cpus at the same time.

                        Here’s a little album to show proof of concept: http://imgur.com/a/asuc1

                        PCI 0000:03:00.0 is the video card and PCI 0000:03:00.1 is the HDMI Audio. You’ll also notice Win10 knows it’s a virtual machine. Been using Gnome Remote Desktop Viewer to RDP into the system with no drawbacks to my workflow so far. Might change my tune once I start doing some 3D modeling but so far so good.

  3. Hello all. I’ve been running 4x8GB corsair dominator gt 1866 on my X58 ASUS R2E for a while now paired with xeon x5674 @ 4,4GHz. I also added two 2GB sticks for a total of 36GB. Thanks to this thread I have decided to stick with my X58 board but will “upgrade” to R3E and 3-way SLI GTX 980 Ti. The rig is all water cooled and I will try 6x8GB Corsair Dominator GT 1866.

    The life of X58 shure is extended and I’m already gaming at 2-4k with my rig and expect to be doing so even better with my new rig.

    ASUS Rampage III Extreme with 48GB RAM, XEON X5675@4,4GHz and 3-way SLI GTX 980 Ti on water.

    Oh yeah….

    • Hey Frode,

      I’m not up to date about it, but does 3-way SLI still work in games for the GTX 980 Ti? I’m asking because it was disabled for the 1070/1080 series cards, or rather, there are no longer any SLI profiles for 3- & 4-way SLI in the driver for games. Or are 980s’ unaffected by the change?

      Anyhow, X58 sure is a long-lived platform, mostly because CPUs didn’t evolve that much in the past few years. I’m also still using my X58 workstations, both the one at home as well as the one at work. Upgraded my home one to USB 3.0 using Startech PEXUSB3S44V card. Only thing that’s a slight bit problematic is the slower SATA/3Gbit, but I’ll survive the bandwidth cap. ;) And PCIe 2.0 is still good enough for graphics cards, so yeah (I’m on a Titan Black, not that I’d use it much…)

      Well, with that Rampage III Extreme you’ll get NEC/Renesas USB 3.0 onboard, so that’s nice. :) If you need it that is. In any case, that’s going to be quite the powerful gaming rig! 8-)

      • Hey thrawn

        That’s a good question; will 3-Way SLI still work with GTX 980 Ti with the new limit for 10xx cards to 2-Way SLI?

        I was running 3-Way SLI with my GTX 680 cards with no problem, untill that setup died on me, and I’m now running 2-Way SLI with GTX 980 Ti. Quite a powerful gaming rig indeed, and futureproof for 4K gaming @ 60Mhz. X58 still rocks!

        However I can’t affort a third card atm, so I can’t comfirm it works but will get back to this thread if I get that in the furure. My guess is that it will work just fine as the limit on 2-Way SLI for 10xx cards are related to newer technology that should not affect older cards running 3-Way SLI..

      • But thrawn

        ASUS Rampage III Extreme have 2 x SATA 6Gb/s port(s) running on a Marvell PCIe 9128 controller.

  4. Hello,

    This is a great news for X58 !
    I’m still using 2 x Asus Rampage Gene rev. 2 with i7-920 (rev. C0 & D0 respectively) as software development.

    For Linux, I have programmed a cpu monitoring software which is able to query the channels DRAM timings through the i7 IMC.
    I will really appreciate if you could give it a try and report the timings of your 48GB kit.

    Available in the GitHub at will show standard and extended timings, QPI speed and DRAM frequency also.

    Feel free to contact me for any assistance with CoreFreq (how to build, how to run, etc)

    Regards
    CyrIng

    • Hello CyrIng,

      I tried to use your CoreFreq tool, but unfortunately, it doesn’t even compile on my system. Here a select part of the error log, there a many more of a similar kind related to the given fields on subsequent lines and in other functions:

      corefreq-cli.c: In function ‘CreateWindow’:
      corefreq-cli.c:1833: error: unknown field ‘fg’ specified in initializer
      corefreq-cli.c:1833: error: unknown field ‘un’ specified in initializer
      corefreq-cli.c:1833: error: unknown field ‘bg’ specified in initializer
      corefreq-cli.c:1833: error: unknown field ‘bf’ specified in initializer

      Build attempt after git clone https://github.com/cyring/CoreFreq.git was just like this:

      $ cd ./CoreFreq/
      $ make

      Kernel and system information:

      $ uname -a
      Linux host.unileoben.ac.at 2.6.32-573.8.1.el6.x86_64 #1 SMP Tue Nov 10 18:01:38 UTC 2015
      x86_64 x86_64 x86_64 GNU/Linux
      
      $ rpm -qa | grep kernel\- | grep 2\.6\.32\-573
      kernel-2.6.32-573.8.1.el6.x86_64
      kernel-headers-2.6.32-573.8.1.el6.x86_64
      kernel-doc-2.6.32-573.8.1.el6.noarch
      kernel-devel-2.6.32-573.8.1.el6.x86_64
      kernel-firmware-2.6.32-573.8.1.el6.noarch
      
      $ cat /etc/redhat-release 
      CentOS release 6.8 (Final)
      
      $ gcc --version | grep GCC
      gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-17)
      
      $ make --version | grep Make
      GNU Make 3.81

      So this is basically RedHat Enterprise Linux 6.8 x86_64, just in its CentOS flavor. Binary compatibility is 100% though. Maybe there is an incompatibility with my more conservative kernel version?

      • Hello,

        Thanks for trying. Indeed 2.6.32 is an “old” kernel. CoreFreq builds and runs successfully with 4.x versions and should be OK with 3.X. I recommend you to change to a modern distribution such as Arch Linux.

        If time is missing, I can also suggest to boot an Ubuntu live cd-rom (not an virtualized iso), then fetch the developer packages. With such environment (network including), you clone the git repo, and build CoreFreq.

        Prerequisites:
        1- Disable the nmi_watchdog in GRUB or SysLinux boot loaders prior booting the kernel.

        2- Enable the experimental features of CoreFreq when inserting its kernel module:

        insmod corefreqk.ko Experimental=1

        As a final result, you will get a screen like the attached image.

        CoreFreq_12GB

        • Hello CyrIng,

          I’m sorry, I cannot just change the distribution, it’s a complex workstation setup at work, that I need to stay binary compatible with our computation machines and servers for testing purposes. Also, even if it wasn’t a professionally used machine, I wouldn’t switch distros on a whim. ;) My systems are usually set up to stay for many, many years without switching the OS. I hate the extensive work involved with re-configuring a fresh environment…

          That aside, what I can do is to try the mainline kernel 4.10.1 from the ELRepo and see whether I can compile/link for this kernel version. Also, if necessary I can use GCC 6.2.0, which I have built for myself from sources, so that’d be available as well.

          I’ll just try that tomorrow if I have time. But then, I can just as well get the timings from the BIOS, because I’d need to reboot anyway. ;) But still, I wanna know whether I can compile this software with a newer kernel…

          In the meantime, here’s the timings from my Windows machine (also X58 with 48GB RAM, just faster modules than the ones at work), this is at DDR-III/1333:

          CPU-Z showing the timings of 48GB of Kingston HyperX Fury DDR-III/1866 kit on X58
          Kingston HyperX Fury (KHX-1866C10D3/8G), auto-configured by SPD EEPROM on an ASUS P6T Deluxe

          Edit: Ah, I guess that’s not very good, SIV gives a more complete report under Windows:

          SIV showing the timings of 48GB of Kingston HyperX Fury DDR-III/1866 kit on X58
          SIV report on the same system (Click to enlarge)

          • Can’t wait to see your results. However be aware that CoreFreq is a development project, consider it if your target computer is in production. Save, close applications, sync file-systems before starting corefreqk.

            • Yeah, good advice of course. Especially since parts of it live in kernel space. I’ll just install the fresh mainline kernel, boot up, try to compile & run if possible, then boot back to the stock kernel. I’ll let you know how it went, in around… 12-14 hours or so I guess. :)

            • Morning CyrIng,

              And… it’s a failure. A big part of it seems not to be my kernel, but my compiler, GCC 4.4.7. I can compile corefreqd, but not corefreq-cli, that’s when a thousand errors like shown above pop up. Since your Makefile isn’t using the CC environment variable, I changed it to use my GCC 6.2.0. With that, I can compile cpufreq-cli as well, no errors.

              The problem is, that the mainline 4.10.1 kernel from ELRepo is built with the CentOS 6.8 platform compiler, GCC 4.4.7, and you can’t just mix compiler versions in kernel space. So I cannot build the kernel module (fails with a ton of errors as well), that’s where I had to stop. No time to test any live systems.

              However, to still fulfill your original request, I got the timings from the P6T Deluxe V2 BIOS, modules are 6 × KVR1333D3N9/8G, all configured automatically by SPD EEPROM again:

              CAS-RAS2CAS-RASPrecharge-RASACT-RAS2RAS-REFCycles-WriteRecovery-ReadToPreTime-FourActWinTime-Back2BackCASDelay:
              
              9-9-9-24-4-174-10-7-20-0
              
              DRAMTimingMode-RoundTripCHA-RoundTripCHB-RoundTripCHC:
              
              2N-57-59-60
              
              WriToReaDD-WritToReaDR-WriToReaSR-ReaToWriDD-ReaToWriDR-ReaToWriSR-ReaToReaDD-ReaToReaDR-ReaToReaSR-WriToWriDD-WriToWriDR-WriToWriSR:
              
              5-5-16-10-10-11-7-6-4-7-7-4
              • I really appreciate all your efforts to make it starts. Indeed the driver software requires a recent kernel with an up-to-date environment.

                Your timings are awesome considering 6 sticks of 8GB and the pressure they should have on the IMC.
                Do you clock them at 1600 MHz ?

                • No OC, it’s all stock by SPD, running at 1333MHz data rate, even the HyperX Fury DDR-III/1866 is.

                  I did overclock a lot in the past, but not anymore. The stability testing is just too much of a hassle these days, as I need really reliable machines with no data corruption. ;) The highest clock rate I’ve ever ran 48GB at was with Crucial BLS8G3D1609DS1S00 DDR-III/1600 modules, I ran them at 1456MHz data rate (BCLK 182MHz × 8). Needed 1.65V(DIMM) and 10-10-10 or maybe even 11-11-11 for that, I don’t remember exactly. I gave up on the Crucials, because one of the modules died.

                  Only weird thing is that the HyperX Fury kit is running 8-8-8-24, when it should be 8-9-8-24 by [spec] at that speed. Not sure why, I don’t even have XMP?! But it’s stable under very high loads 24/7 (both CPU and memory load), so I’m fine with it.

  5. Hello From Florida, USA, Thrawn!
    I read your post about getting 48Gb of Ram to run on an Extreme X58 board. I have an ASRock X58 Extreme in my machine, which I purchased from a company that since went under, as many PC companies have.

    About a year ago, I bought a tirple channel memory kit with some bonus cash I got. It is 2 kits of 12GB Gskill memory at 4GB per stick. Put it all in the motherboard. Should be 24GB. I was running XP Pro when I did this and I think it was only x32 not x64 for the OS. Whatever the case, I found out the OS I had wouldn’t see 24GB. No worries.
    Sometime this year, I took the plunge and went to Windows 10 Home x64. For whatever reason, I just recently took note of how my BIOS and Windows both reported only 16GB. When I went into the BIOS, it showed Banks A and B full and C empty. I moved the RAM around. On reboot, it showed A empty and B and C full.

    I did a little digging, and came across hundreds of posts complaining about the X58 boards, regardless of manufacturer, not just ASRock. One post said remove the RAM, install a single stick, Flash BIOS and so on. It didn’t work for me.
    Another post said reseat the CPU. Not only did I reseat the CPU but I bought a Corsair CPU water cooler. Still only 16GB of RAM.
    I came across something else that said the INTEL i7 930 might not run RAM at certain frequencies; but the mobo should automatically set the RAM defaults to speed that work. Basically, when people started talking about adjusting timings and voltages, I was in over my head.
    I have CPUID and it identifies 24GB of RAM, and each individual stick correctly.
    Reading your post about getting 48GB of RAM to work on the same board makes me think you’re the guy who cna tell me how to fix this issue. Please let me know what you need from me to help me out- provided of course, you’re willing to help.

    • Good morning Jason,

      I took the liberty of moving your comment from the [about:XIN.at] section over here, since it fits better contextually.

      As for your problem, it reminds me of [Jims’ case] further down. For him it was an electrical contact problem, where re-seating the memory solved the issue. Just in case you could also try to clean the contact pads of your memory modules. I typically use pure alcohol (like in cleaning cloth for glasses) for that. Clean the modules’ contacts carefully and re-seat them. Now boot up and see whether you get 24GB.

      If that doesn’t solve the problem, you should have a look at the BIOS settings next. According to your description it might be a faulty pair of modules as well, but testing the BIOS settings is faster, so we’ll do that first. Yes, we’re going to adjust those timings and voltages now, but bear with me, it’s not that hard.

      To make things clear: CPU-Z (I guess that’s what you meant by “CPUID”) detects your 24GB of RAM. That means that the SPD EEPROM on all of the modules is accessible. That’s a small chip on the memory modules which holds important information about each module (like timings at specific clock rates, manufacturer, model number, capacity etc.). Somehow though, the actual DRAM ICs can’t be used in your case. If we can’t solve this mechanically/electrically, please take a look at your mainboards’ [manual], pp. 55. We’ll look at the following settings in your BIOS, one after another:

      1. Advanced / DRAM Frequency
      2. Advanced / XMP Frequency
      3. OC Tweaker / DRAM Timing Control
      4. OC Tweaker / Overclock Mode / DRAM voltage

      Adjusting the DRAM frequency is the easiest way to go. While on the Advanced page of your BIOS, please verify whether it shows any specific clock speed and timings under XMP Frequency, like shown in the manual. If yes, then your memorys’ SPD EEPROM chip also holds a high performance XMP profile specific to Intel mainboards.

      Please adjust your DRAM frequency to the lowest possible value. If you do have an XMP profile as well, lower the XMP frequency to the lowest possible speed as well. Now save & reboot. To make sure, power the machine off entirely and restart it after that.

      Boot into your OS and check out the available memory; Press <Windows Key>+<r>, enter msinfo32 and hit <Enter>. If it shows all your RAM now, your memory was simply operating beyond its capabilities. You might look at the timings and V(DIMM) now (see further below) to stabilize a higher clock speed or you may just keep running your memory at a lower clock speed.

      If this doesn’t do the trick, keep the memory clock low and adjust the next part, the DRAM timing control. Change the timings tCL, tRCD, tRP and tRAS to their highest (=slowest) values respectively. Those are the most significant ones. Also please ensure that Command Rate is set to 2T, not 1T! That last part is especially important when using a full six modules! Save & reboot again and test whether it’s working.

      Still no dice? Then look at the OC Tweaker / Overclock Mode page, and adjust your DRAM Voltage from 1.5V to 1.65V. Make sure not to go higher than that though! Too much V(DIMM) might harm your CPUs’ integrated memory controller. Save, reboot and test again.

      Still nothing?! Now it’s really time to question whether your memory is even working at all. Revert everything in BIOS back to default settings. Remove all but one memory module. See whether the machine boots up with it. If you really wanna be thorough, download the free version of [Memtest86] and burn it onto a CD or create a bootable USB flash drive with it. Boot from it, and test that single module for 1-2 hours.

      That module seems to be working fine? Ok. Repeat the test with the next module, and then the next and the next, one after another. This is time-consuming if you really run Memtest86 on each one of them, but with a bit of luck you’ll find a bad one immediately (=no POST / black screen on boot). I’m suggesting this method because I’ve had it happen to me as well, when some of my Crucial modules suddenly died.

      Even if you do find a faulty module, don’t abort the test! You’ve got 16/24GB working, so that means that two 4GB modules might be bad. So test all of them!

      I understand that this kind of diagnosis is tedious, but at that point it’s what should be done I think.

      If you end up empty-handed after all this, I’m not sure whether I can help. You have already verified that all individual memory channels are up and working (A, B and C), so it shouldn’t be the mainboard or CPU. One last thing you could still do is to remove your CPU, take a magnifying glass and carefully inspect the LGA pins in the open CPU socket. The socket should look somewhat like this:

      LGA socket
      An LGA socket, image is © [RECH COMPUTER] (Click to enlarge)

      Those pins are fragile. I already had a case where I accidentally dropped a Xeon processor into a LGA1366 on a server mainboard. The result was that one memory channel was permanently dead because the corresponding pins were bent downwards in the socket, no longer making any contact with the lands on the CPU. Carefully fixing the pins with a sewing needle did the trick. This shouldn’t be the case given your problem description, but you might still wish to inspect those pins carefully, at a very low angle.

      And that about sums it up, can’t think of anything else to try right now… If you have additional questions – e.g. about the BIOS settings – just ask away!

      • Thank you so much! I’m going to try this approach, even though my BIOS is very “clunky” compared to some I have seen screenshot of. I appreciate you taking the time to help me out and I’ll report back here when I’m done.

        • Hey Jason,

          Please take all the time you need! :) The BIOS options I listed above are taken directly from your mainboards’ manual, so it should be ok. Although it does sound more like a hardware problem in your case, but we’ll see…

    • hello thrawn, armand from malaysia. manage to upgrade 48GB RAM on my rampage 3 extreme, my processor W3680

      and…how to post picture here??

      • Hello armand,

        Thank you for your report! Nice to hear it works for you as well!

        Also: Posting pictures wasn’t allowed here. Actually, I’d have loved to give you the “img” HTML tag to hotlink to off-site images, but it was too hard to do quickly (plugin wouldn’t work). Instead, I gave you a functionality with which you can attach images, it’s below the “Submit Comment” button now. The images will be placed below your text and are clickable for a zoom in (done by JavaScript/AJAX, you can then right-click and “save as…”).

        I do have a request though, please be gentle! ;) This is a 200MHz quad Pentium Pro server with just 8Mbit/s of symmetric bandwidth running software it was never meant to from 22 years after it was built. Uploading (& processing) images is going to be very taxing on the CPUs, and if they’re large, it’ll take longer to process and download. Please try to use proper compression algorithms and settings (e.g. JPEG w. 90% quality / 10% compression for photographs, 8-bit PNG for Windows 10 screenshots etc.) as well as reasonable resolutions to reduce the load for my poor 1995 server. ;) It’s slow enough already! :P

        Thank you!

  6. Hello,

    Thrawn, first, congrats on this awesome blog post!

    I was wondering, if someone did try 16 GB Non-Registered DDR3 modules with X5690 or W3690 CPUs?

    Thanks

    • Morning,

      Not to my knowledge, unfortunately. I’m waiting for that myself. I still consider it highly improbable for this to work (breaking either the density or the rank barriers), but who knows. Thing is, we still don’t have any 16GB DDR-III modules at work and likely never will, because now all new machines are using either regular DDR-IV or LRDIMM DDR-IV.

      The cheapest 16GB module I could buy right now would be the Crucial CT204864BD160B. 155€. Too much of a risk to test even a single one. Now imagine six of them – 930€! It’s just too expensive to do this privately. :(

      • Yikes, those modules are really expensive.
        People rather buy DDR4.

        Well, if someone finds out more about it, let us know.

        • Evening EinZwein,

          Hmm, the X58 may be one of Intels’ chipsets having a stronger following over the years, but it doesn’t look like anyone’s gonna try 16GB modules anymore. Would be quite a bit of luck for anyone to have the opportunity these days.

          The best course of action might be to ask a X79 user with 16GB modules, and have them try it on an X58 or borrow you some of his memory to try it. Unfortunately I don’t personally know anyone who has an X79 board with 16GB sticks (for a maximum of 128GB). :( I could even send them a X58 board just for testing…

          Of course, if I do get the opportunity to try it, or if I do learn of anyone having tried 16GB modules on the X58, I will surely post about it here!

          • I have a pair of 16GB DDR3 knocking about and an X58 which isn’t powered up too often. If I get a moment over the Christmas break I’ll see whether they’re recognised.

            • Hey marctxk,

              Now that would be awesome! :cool: Ah, and if you can find the time to do that, would you mind telling us what exact part number those memory modules have? Just so I can look up the organization of the modules (ranks, chip density etc.).

              Thanks a lot!

              • So, I just tried it and no go.

                RAM is Crucial CT204864BD160B.C16FA. These are DDR3L-1600. Each has 16 chips, marked 5DA47 C9BDV.
                It came as a 32G dual channel kit, (and I hoped to use it in a mini ITX LGA 1150 but to no avail).

                I have a Asus P^T-WS Pro BIOS 1205 with W3690 CPU. I got no screen sync at all the error code cycled through four values: b2 00 e0 01.

                HTH

                • Ah, well, I thought as much.

                  In any case, thanks a lot for taking the time and effort to try this! It’s a valuable piece of information! Knowing what won’t work is almost as important as knowing what will work after all.

                  So now we know that exceeding the maximum IC density limit won’t work (16 chips means 16×1GiB instead of the maximum density specified, 512MiB). No 1GiB ICs means no 16GiB 2R modules will work. Now the last question that remains is, whether 4R modules with 32 chips would play nice with X58 in a 6-modules configuration… I seriously doubt it though.

                  Thank you very much, marktxk!

                • “(and I hoped to use it in a mini ITX LGA 1150 but to no avail)”
                  OT, but if you are still there have you tried it with a i5-5675C or i7-5775C? I read that some Broadwell chips will support them but want to know which ones.

                • What CPU did you test this with? i can understand a nahalem failing but a westmere should work.

                  • He Danial,

                    He mentioned it above, he used a [Xeon W3690] processor, which is a Gulftown/Westmere hexcore.

                    Officially, besides the wider address bus, Xeon IMCs only differ in the amount of buffered memory ranks they support, but not in supported IC density. So that result was to be expected.

                    I think the big question remaining is: Can we exceed the limit of unbuffered memory ranks?

            • Uh, uh, uuuuh, please, please do!
              I’m just about to upgrade memory!

              Thanks in advance!

          • For 16GB ram sticks on the x58 platform maybe try contacting Linus Sebastian from LinusTechTips on youtube. He likes doing videos about interesting tests like this that most people wouldn’t even consider doing or sometimes would just be curious about.

            You may be able to reach him at the Linus Tech forums if you can’t reach him on youtube. So definitely give that a try and see where it goes. Might be old, but it would still be an insanely interesting video.

            • Hello Stefanos,

              First of all, my apologies for deleting one of your comments (and moving this one around for context). It’s just that they were pretty redundant.

              Do you know Linus personally? So I can tell him “Stefanos sent me”, or something? I’m a bit reluctant to just write him a PM out of the blue, given he’s kind of a big shot, right?

              If you think it’s fine though, I can try of course. I just need to concisely show him what we have so far, since I can hardly make him read the entire comment section here. :)

  7. help I cant get past 16gb with Xeon L5640 + P6tdeluxe

    I’m trying to run 4x8gb hyperx hx318c10f/8
    ive also tried 3x8gb in triple channel – both only show 16gb in bios and windows ( cpu-z shows 24 or 32gb tho?)
    asus chat state 24gb max 4gb max mods.

    is it just bios settings to achieve this?

    • Hey jimnz…

      Don’t scare me, man… That very memory is sitting on my desk right now, 6×8GB, ready to replace my Crucial set, where one memory module had failed a few months ago. If that shit doesn’t work, it’d be serious trouble…

      Hm, I’ve been running Kingston ValueRAM with no issues however, so I’m still feeling confident! Problem is, I can’t test it right now, as I do not wish to reboot right now. Got 66 days of uptime and the machine still has a lot of work to accomplish. My plan is to not just exchange the memory, but clone the box to a new 1.6TB SSD as well and install a quad-controller USB 3.0 card while I’m at it.

      So I do have the very same modules right here before me, but it’ll take a bit of time before I can test them and tell you more about if/how they work on the ASUS P6T Deluxe. I’ll have to strain your patience there a little bit. Maybe next week, but seriously, I can’t promise anything

      • same board and ram.. spooky!
        Yea I can wait, thanks for the reply.

        What your saying is if I had gone cheap and got K.value I wouldn’t have this issue? ha.
        also, did you have to make any bios changes to get 24gb+ or default?

        thanks

        • Hey,

          Hm, all I did was to up the VDIMM to 1.65V to stabilize my Crucial memory, otherwise it wouldn’t work properly at 1333MHz data rates. As for the Kingston ValueRAM, I didn’t have to do anything at all. It’s all default settings in the BIOS, and see here:

          $ uptime
          10:16:01 up 32 days,  1:30, 83 users,  load average: 8.06, 9.42, 9.59

          Since I upgraded that second machine with 6×8=48GB of Kingston KVR1333D3N9/8G, it’s been happily crunching away at every job I’ve been throwing at it, whether it was running multiple virtual machines, building Knoppix images or transcoding large batches of H.264/AVC video. Sometimes the memory load would sit in the 20-30GB range with the rest filled up by the ext4 file system cache. No problems whatsoever.

          As soon as I get to test the HyperX modules, I’ll report back here!

          • thnx . ill drag my arse with a mere 16gb til then.. :-)

            • Hey jimnz,

              I just wanted to reply with my apologies, I still haven’t taken the time to test the memory. There were to main reasons for that:

              1. A drive was kicked out of my primary RAID-6 disk array, and a rebuild was running, that I didn’t want to interrupt with reboots. Since this is my main data storage, it was kinda important to keep everything online.
              2. Laziness. Like, massive laziness. And lots of Anime. And Visual Novels. And Manga. There’s no excuse for that, I was just too damn lazy during my vacation, reboots/downtimes just suck. ;)

              I should probably fix a date for this, but… ah… dammit.

              • BAHA – na you just chill

                Ive almost come to the conclusion ill need to swap out to a 2nd hand x79 and E5 2670 if I cant get the 32gb

                thanks for the update

                • Good Evening Jim,

                  Well, actually, I had to do a reboot yesterday, or rather today in the middle of the night, because of some other problem. I used that opportunity to try the new RAM.

                  Well, bad news. Or good news… Depends on how you look at it: The memory is working just fine with a full 48GB capacity being detected in my system on default BIOS settings:

                  48GB Kingston HyperX Fury being detected on an ASUS P6T Deluxe
                  Seems to work fine…

                  The only odd part is, that it should be running 8-9-8-24 timings at DDR-III/1333 speed according to the [specifications]. However, something with the SPD EEPROM doesn’t seem to be right, because it’s actually running at faster 8-8-8-24 timings instead. I hope it’s stable that way…

                  In any case, I was unable to reproduce your issues. :(

                  • Well that’s odd as fu$k?
                    the only diff I see is theres NO “B” in my model number: HX318C10F/8
                    I think that’s just color?

                    also I’m loading 4x mods in 2x dual or trying 3x in triple so not the full 6x mods like you, perhaps it will only read 2x triple channel?

                    ill try loading 4x mods and do a bios reset, see if that helps.
                    I’d appreciate you sending 2x sticks in the for me to test, but I don’t suppose you have one of those..

                    • Morning,

                      Yeah, “B” is the color, I got the black ones, yours should be the blue ones. I think blue has been the standard HyperX color since when they started producing the series…

                      And then the memory channels. Hm. But if you’re loading two channels with 2×8GB each, it should work just fine. I haven’t been running the board in dual channel before, but I know people who have used X58 boards like that. Heck, there are even dual channel processors like the Xeon LC3528 (it’s dual core, and it’s IMC only supports two channels of DDR-III/800).

                      Unfortunately, no Tardis’ available here. :D I do have 3×8GB Crucial DDR-III/1600 free now, but I guess international 2-way shipping would just be too expensive (including customs and all) to make this feasible.

                      Hm, it probably doesn’t matter, but just to make sure: Do you have the [latest BIOS version] installed for your P6T Deluxe? Maybe that could be of some help. I mean, probably not, it’s just that that’s the version I’m running, only with Intels latest µCodes patched in for the Xeon X5690. The CPU and RAM did work without the µCode patch as well however, so I think my BIOS mod doesn’t have anything to do with this.

                      • thanks for your help, but yes I’m running 2209. I might update to that bios anyway – never know

                        below are the screens of Cpuz for reference, note cpuz states 24gb in triple (as installed) but task man states 16gb ( 2 of 6 slots.)




                          • Morning,

                            CPU-Z only states what a memory modules’ SPD EEPROM is telling it. But that doesn’t say much.

                            Could it be that an entire memory channel is offline/buggy? I had such a problem once due to contact problems in the CPU socket with a dual Xeon box. Some of the LGA lands in the socket were bent down, so some of the memory controller contacts on the CPU didn’t connect, resulting in a missing channel. I fixed the socket and everything is working fine ever since.

                            Have you tested all channels individually?

                            • 32GB impossible? NO!

                              i took your slot testing advice:
                              So i started slot swapping.
                              A1B1A2B2-16gb
                              B1B2- 16gb
                              A1B1C1- 16gb
                              C1C2- NO BOOT ? faulty slot??
                              A1A2- 16gb
                              B1C1 – 8GB – both C slots dead??
                              A1B1 -16gb

                              last tried true 4 slot config:
                              A1A2B1C1 32GB!!

                              So im guessing ive just major kooked it and not seated ram correctly ( a few times)

                              Anyhoo – i now have the full 32 running – thanks alot for your guidance. else i would have given up or at least unnecessarily replaced board and chip – right on..

                              now its back to OC’ing the CPU ;-)
                              im gonna try to get 3mgh+ outta my 2.27..

                              • Hmm, interesting. I have seen bad contacts before as well, even though they looked clean. Could be fat from touching them or whatever, no idea. I’d just clean them with alcohol and retry. But if the sockets are affected… Could be some other issue as well of course, but if it’s working, it’s working! :)

                              • Just to help someone else…
                                Every ASUS motherboard have its own config for inserting 1 – 6 slot modules.
                                You can’t just plugin slots that you want. You must follow the ASUS P6T Deluxe manual recommendations for installing RAM modules.
                                For your motherboard with 4 modules this is the only config that WILL work with 4 modules: A2A1B1C1.

                                If you want to add another module it must be: A2A1B2B1C1.

                                Read more the manual for your motherboard: http://theor.jinr.ru/guide/i7/e4398_P6T%20Deluxe%20V2.pdf

  8. Hey thrawn, which motherboard did you use? Thanks in advance

    • Hey serpico,

      Ah crap, I just noticed that I didn’t even mention the mainboard in the initial post! My bad. So far I’ve run this on an ASUS P6T Deluxe (old version with SAS controller) for several years and an ASUS P6T Deluxe V2 for about two weeks. Latest BIOS in both cases. The latter is my Linux workstation at work and was upgraded only recently, more info [down here]. If you browse through the comments, you’ll find a few more mainboards that were tested successfully by other users. :)

  9. I recently purchased a kit of KLEVV 2800mhz CL12 1.65v memory but my motherboard is being a pain trying to get these to run at higher speeds. I have to adjust the advanced timings for them to even post past 1333mhz. At the lowly 1066mhz these work so I’m assuming they are compatible and it’s just my motherboard not setting the correct timings automatically.

    Any feedback on whether there’s a reason this memory would give me any issues on an X58 motherboard would be amazing! I was trying these on an ASUS P6T (not deluxe, not SE, original P6T) with the latest BIOS. The X5670 I have works great it’s just the memory. I have an ASUS P6X58D-E board on the way to replace my P6T since I’m selling that to someone else and I’ll try on that board but still want to use this RAM since I’ve read amazing reviews for it and got it at such a good price!

    Here are some more details on the 2400mhz variant of the same modules;

    http://hw-db.com/wp-content/uploads/2015/10/KM3G4GX2Y-2400-11-13-13-31-0_Thaiphoon.png

    • Hello xenkw0n,

      These may be some stupid questions, but just to make sure; Are you running them at an actual V(DIMM) of 1.65V, and have you activated the Intel XMP SPD programming in your BIOS as well?

      • thrawn,

        I am not trying to run these at 2800mhz. That would require a minimum of 4200mhz uncore/northbridge frequency and based on that alone I am not trying to run these with the XMP profile. I have set the V(DIMM) to 1.64 (P6T steps up in .02v at a time) just to make sure I have plenty of voltage for the frequencies and timings I’m attempting.

        You can see some of the impressive overclocks this one reviewer had for this memory so even at 1.64 V(DIMM) and frequency of 1400mhz should be a non-issue with CL9;

        http://hw-db.com/memory/2833/klevv-urbane-and-genuine-8gb-ddr3-2400-review/2

        I’m thinking the board is not recognizing all of the advanced timings correctly because these are much newer memory sticks (released early 2015) and the X58 platform just isn’t handling them correctly. My main concern was that the physical specs (shown in the first message) are what would cause some issues, i.e. 1 rank of high density vs. 2-rank lower-density. With the different replies I’ve read you give to other people I can tell you seem pretty well versed in the type of memory X58 can handle.

        • Oh I can’t believe I haven’t stated which model I have… I have the KLEVV Genuine series, not the Urbane, which is supposed to be better binned. This was one of the speeds/timings the above-mentioned reviewer was able to hit;

          1600mhz 8-8-7-10@1.44V

          These except I got a 4x4gb kit and am only trying to run with 3 at the moment;

          http://www.newegg.com/Product/Product.aspx?Item=N82E16820101046

          • Hi again,

            […] With the different replies I’ve read you give to other people I can tell you seem pretty well versed in the type of memory X58 can handle.

            That’s not exactly that true. ;) My own hands-on experience is rather limited (3 mainboards, 3 types of memory), so a lot of my knowledge comes from either datasheets or other peoples’ experiences.

            As for the ranks… As long as you don’t exceed the absolute maximum number of ranks allowed, I don’t see the problem. And when it comes to IC density, we’re equal, as I’m running 2-rank with maximum density as well, whereas yours are 1-rank. But both your and my modules use 4Gbit / 512MiB chips. I just have double the amount of chips, and that’s it. So the capacitive load can’t get any higher than it is in my systems.

            If anything, my setups should be less stable than yours, not more… :? So I’m at a loss here, I’m afraid. Maybe it’s a fundamental problem of the P6T or maybe there really is some minor issue with the timings, I just can’t say. It wouldn’t be the first weird memory incompatibility issue either. It’ll be interesting to see whether it can work on the P6X58D-E you’re getting…

            • Still more experienced than I for this socket!

              Thank you for confirming the hardware setup on your memory. I’m just trying to cover all of my bases before putting more effort into getting this memory to run. The limited amount of data on advanced memory timings is becoming a hassle since I would like to find examples of different advanced memory timings for memory at different speeds and try to adjust around them. I’ll probably have to dive into some research and figure out what the different names for the advanced memory timing settings in my BIOS are referencing compared to the more widely used acronyms and make sure I have everything entered correctly.

              I am also interested to see how the new motherboard handles these. Maybe it’s the processor itself but I highly doubt that since the IMC is built into the motherboards on the first gen chips.

              I actually have a couple X5670’s I can use since I’m building some machines for friends who want to jump on the X58 bandwagon since the hexacore Xeon’s are so cheap now! You can pick up a X5670 on ebay in the US for around $65 and if you wait for the right deal a motherboard for ~$100. Crazy how 7 years later these are actually still relevant. I scored 36k CPU score on vantage with my X5670 overclocked to 4.1ghz and 47k on my brand new Broadwell-E 6800k overclocked to 4.1ghz (~30% increase for same amount of cores/threads). And if DirectX12 truly starts to handle CPU threading for video games these systems could have new life breathed into them.

              • So I was able to get this RAM working well. Turns out you can’t go past x10 memory multiplier with the Xeon X56xx series processors. I increased my BCLK up to 200 and got these running at 2000mhz 9-9-9-27. One thing I’ve learned while continuing to read up on memory though is that the density of the chips plays a vital role in the tRFC timing. So since these are single-rank 4gb modules my tRFC can’t be manually set at these speeds because it needs to be above 160 which is the maximum my board offers for manual tuning. Leaving it on auto sets the memory to 232 tRFC, which is kind of high for my taste…

                So that leaves me to ask – do you have a good way of finding out if memory is single or dual rank? Most online storefronts don’t mention this and I’m having a hard time finding out (i.e. Crucial Ballistix Tactical Tracer 4gb modules have some really good overclocking reviews – if they are dual-rank then I could tighten tRFC considerably with them).

                • Hey xenkw0n,

                  Unfortunately, I do not know of an easy way. In my country we have a price comparison site called [geizhals.at] which lists the ranks for a lot of modules, but not for all. So in the end, I’d always stay with “boring” Kingston and Crucial standard memory, because you can get decent specifications on those. The stuff made for gamers often comes without any in-depth information about memory organization. :( But [here is a Geizhals list of 2R 4GB modules with data rates of 1866MHz or higher] (english version of the website, no guarantee that the specs are correct btw.!). Not a lot, but better than nothing?!

                  As for that Ballistix Tactical Tracer, I can only make an educated guess based on [this Crucial page]. It says the memory is 512MiB × 64-bit. Since each module features 4GiB, it means that they have 4096 / 512 = 8 chips. While the rank and chip width specifications are missing, my guess for a module with 8 chips would be “single rank” with 512MB×8-bit wide chips. It’s mostly the modules with 16 chips that are dual-ranked after all.

                  Oh, by the way, I checked out my current tRFC out of curiosity. It’s autoconfigured by SPD at 200 clock cycles. But that’s at a lowly data rate of 1333MHz. I guess it might go higher at higher clock speeds.

  10. Hi,

    As of today, which CPU (W3690 or X5690) and RAM you would recommend? I’m planning on buying full set of memory for my P6T7 WS SuperComputer. The max motherboard RAM speed is listed at 2000 MHz max.
    Right now I’m running 12 GB (6 x 2GB) with i7-920.

    Also did someone figure out the 16 GB DIMMS?

    Thanks!

    • Good evening Masterdev,

      If there isn’t too big a price difference, I’d always go for the X5690. It does have a slightly lower VID on average, meaning the chances of a CPU running a slightly lower V(Core) are higher. Also, it gives you VT-d, which the W3690 doesn’t appear to have. Not that it matters for most users.

      As for the RAM, nobody I know has tested Reg. ECC 16GB sticks as of yet. The unbuffered ones are extremely unlikely to work (due to maximum rank / chip size specification), and I wasn’t able to confirm any X58 board to be able to run Reg. memory without causing stability issues. My P6T Deluxe and P6T Deluxe V2 both run into trouble with Reg. ECC. So for the moment, I’d say the chances for X58 to run 96GB of memory are slim. But hey, I’m welcoming anyone who can prove me wrong there! :)

      As for the memory I’d recommend, well. It’s not like anything I can say would have serious statistical relevance, but I’m currently running 6 × Kingston KVR1333D3N9/8G just fine, as mentioned two posts below. Also available as a 3-module kit under the name KVR13N9K3/24. Default SPD timings, default voltage, no issues. The Crucials mentioned in the original post have shown trouble with one module failing after ~2 years, part number BLS8G3D1609DS1S00. Quite surprising, given the low return rate of Crucial memory. But this can happen for any piece of hardware after all.

      And memory speed… Do you intend to overclock? Officially, the X58 does DDR-III/1066 max. I’m using DDR-III/1600 and DDR-III/1333, but effectively they’re running at stock speeds here. If I were to aim for high DRAM clock speeds, I’d probably go for Kingston HyperX Predator or HyperX Savage right now. But that’s more a gut feeling than anything else, as I haven’t personally tested any high speed modules.

      • Thrawn, thank you so much for replying!

        I agree with you about the CPU, I will get X5690. Just wanted to hear another opinion.

        I tried on my motherboard Registered ECC, it was a no go. Not even a BIOS boot!

        I’m not planning to overclock. But I’m willing to buy faster memory 1866 or 2133 even though my motherboard runs 1866 and 2133 only overclocked. Beyond 1866, I don’t see any benefit with my motherboard.

        Would it be possible to achieve the memory speed of 1866 with the X5690, if the memory is rated 1866?

        Here are the modules that I’m considering: http://www.pand.li/newegg-1866-ddr3-hyperx

        • Hi,

          Well, I don’t personally know your mainboard, but from the CPU side I don’t see any problem with DDR-III/1866. If ASUS says it can do DDR-III/1866, then it should be able to do it no matter the IMC/CPU you plug in. Not that I’ve ever tried high memory clocks. I think the highest I went was just a bit over 1400MHz data rate.

          • Thank you thrawn!

            I will keep you posted on my progress.

          • Hi, just to update…
            I bought (4GB x 1) MICRON MT36JSZF51272PZ 2Rx4 DDR3 PC3-10600 1333MHz ECC REG MEMORY RAM, to test with my old W3690.
            It DIDN’T boot. So ECC Reg. memory and this CPU are no go.
            My motherboard says “ECC, Non-ECC, Un-buffered Memory”, so it might not work at all with registered memory.

            I will test again with the X5690 when I get one soon.

            Tested config:
            Motherboard: ASUS P6T7 WS SuperComputer
            CPU: Intel Xeon W3690
            ECC Registered DDR3

            • Morning,

              I’ve been playing around with Reg. ECC for a while now, and the longest period of successful operation was maybe 2 days, on a P6T Deluxe V2. Then it would freeze.

              It’ll behave in a really erratic fashion as well, sometimes no POST, then it would lock up the machine etc. I’m starting to think that Reg. ECC is a no go on Desktop mainboards, even if the electrical traces (like the Reg. parity lanes and so on) are present on the board, and they are on mine.

              At least I haven’t seen any X58 board that can run memory with register buffers in a stable fashion yet… And short tests might not be enough to find out whether it really works or not either.

              Still looking forward to your results though. For me it produced an unstable setup with both the Core i7 980X and the Xeon X5690.

  11. I just wanted to confirm that ASUS Rampage II Gene has no issues with 8GiB DDR3-1866 memory.

    http://valid.x86.fr/b9yv62

    • Hey ike,

      Thanks! Actually, I’ve very recently confirmed the ASUS P6T Deluxe V2 to work with six individual Kingston KVR1333D3N9/8G sticks (8GB unbuffered DDR-III/1333). I didn’t get the triple-channel kits, because our supplier at work couldn’t deliver them anymore. But even with six individual modules, it still works nicely with a full 48GB RAM. This is what the machines’ typical workload looks like:

      48GB RAM on an ASUS P6T Deluxe V2
      (Click to enlarge)

      Currently, that CentOS 6.8 Linux machine is running a create_compressed_fs worker building a system image for our Linux distribution (based on Knoppix) using a 7zip compressor, about 16 terminal windows for work, two chat clients, two Vivaldi 1.3 web browsers with 4-8 tabs each, Thunderbird, several gedit instances, several LibreOffice instances for documentation work, one x265 video encoder, one x264 video encoder and 2 of 3 typically used virtual machines using VMware Player 7 and VirtualBox 5.1.2, all across eight virtual desktops.

      The most memory actively used was close to 20GiB, the rest being filled with file buffers, so with ext4 file system cache. So far no problems and no data corruption, running for exactly one week, 24/7. CPU load is not always that high, but there are runs for 10-20 hours at loads like this, sometimes even higher.

      So I guess those Kingstons are looking good as well! :)

  12. Hello Thrawn,
    Thank you for your informative post. I found it while searching for a solution to my problem. I too have an X58 based system, and have dealt with some “ghost POST” issues. After replacing lots of hardware and nearly ruling out “noisy electricity”, I believe I’ve narrowed it down to RAM configuration. As mentioned in your “About” page, can I email you? Wanted to get your opinion on my current configuration.

    Thank you!

    • Hello IlChengis,

      Of course! Although I’m not sure if I can really help, but I guess I could at least look at the problem and see whether I have any ideas… Hm, the email address you provided looks a bit weird, is this a real address/alias of yours? In any case, I’ll just try and send you an email. :)

  13. Hi!
    Can I mount 3x 8GB 1600MHz DDR3 on asus p6t se with intel i7 920?
    I don’t know if the CPU would support 1600MHz of frequency and single 8GB modules.
    I’d like to use modules like the Kingston KVR16N11/8.

    Thanks!

    • Hello Smashemcee,

      While I haven’t tested the P6T SE myself, this should work just fine. The Core i7 920 can handle it, I have verified that myself (Core i7 930 on P6T Deluxe and Core i7 950 P6T Deluxe V2, same CPU tech). The mainboard won’t be clocking the memory at a full 1600MHz data rate without overclocking, but that’s not an issue. It’s going to be running at 1066MHz without problems.

      The specific memory you meantioned ([KVR16N11/8]) has a 512MB × 8 ICs × 2 ranks organization, so 16 chips in total over 2 ranks. That’s within the specifications, and exactly the kind of module you should buy for this!

      • Thanks a lot.
        I’ve read the 1600MHz frequency can be easily set from the bios settings (AI Tweaker > DRAM Frequency), leaving everything else to the “Auto” value. Is it right or should I manually adjust something to the voltage or else?
        Would be better to buy KVR1333D3N9/8G? I already use KVR1333D3N9/2G modules, so I’m sure 1333MHz works fine.

        • Hey again,

          You can always buy the faster ones, it doesn’t really matter. 1600MHz DDR-III will have settings in their EEPROM chips for automatic configuration for lower data rates as well (1066MHz and 1333MHz). So you really don’t need to do much. Since you won’t be overclocking the memory itself, there’s no need for raising voltages either, unless you’re trying to push it with 6 modules for the full 48GB. In my experience, Uncore Voltage can stay on default as well when raising DRAM speed.

          Also, setting your memory to 1600MHz (if easily doable from your BIOS) should work just fine as well. The memory should be configured automatically (timings and all) due to the 1600MHz settings stored in its EEPROM chip.

          Personally, I don’t think it matters a lot really. You got triple channel anyway, so the clock speed difference doesn’t do that much, unless you have exceptionally memory-transfer intensive applications. I played around with gaming and video/audio transcoding, but couldn’t really measure much of a difference.

          My recommendation: Get the 1600MHz ones, set them at 1600MHz. If it works, nice! If it’s unstable, just go down to 1333MHz data rate. Not much of an issue anyway. :) I got 1600MHz modules myself, running them at 1066MHz because I stopped caring about overclocking (I got lazy you see, even running a Xeon at stock speeds now). No problem. :)

          • Thanks a lot!

            Greetings from Italy!

            • Hey!

              Did you end up trying 3x8GB with your P6T-SE? I’ve got the same motherboard and I was about to pull the trigger on going to 3x8GB as well.

              Also planning on getting a X5670 to replace my i7 920.

  14. Hi. I have the Asus Rampage III Gene motherboard with the i7 950 and was wondering if I need to to purchase 2 triple memory kits, or will 3 dual channel kits work for 48GB (3x2x8GB)? Will this setup operate in dual channel mode?

    Thanks.

    • Hello Wei,

      Combining dual channel kits works just fine, as long as they’re identical. The Crucials you can see above are actually from three such dual channel kits, and that just worked out of the box. And yes, in triple channel mode. If the whole memory gets detected at bootup and in your operating system, the memory will operate in triple channel mode! There is actually no other way on X58 boards, if it comes up in dual channel mode, you’d notice that immediately, because you’d only get 32GB RAM instead of 48GB.

      • Hi Thrawn,

        Thanks for your reply!

        I am now considering between:
        Kingston HyperX FURY 16GB Kit (2x8GB) 1866MHz DDR3 CL10 DIMM – Red (HX318C10FRK2/16) and
        Crucial Ballistix Sport 16GB Kit (8GBx2) 1600 MHz Clock Speed DDR3 PC3-12800 240-Pin UDIMM Memory Module (BLS2KIT8G3D1609DS1S00)

        As they are both pretty close in price now. Do you think the Kingston would be better value and would also work?

        On another note, would a lower power module (1.35V) make any difference in terms of probability of success?

        My PC has been stuck on 6GB of RAM since 2011, so should appreciate the upgrade!

        Thanks again for your help.

        • Hello Wei,

          Lower voltage might make a small difference, as almost all X58 boards drive the memory at 1.5V anyway. Running 1.35V modules at 1.5V should make them more stable at high clock rates and capacities, but it might not matter if you’re not overclocking anything.

          You might also consider to just upgrade to 24GB instead however. That’d be within the specifications no matter if it’s 3×8GB or 6×4GB modules.

          As for value… I always considered Kingston the “safe choice” in all things memory-related, but Crucial’s been pretty reliable too. I guess it doesn’t matter much and both are compatible according to the specs. I was standing before the same choice actually, and went for Crucial – but just because of the lower price at that time.

  15. Nice seeing so much people getting the computer to run with 48gb ddr3 it will be my next upgrade for my Rampage 3 extreme and sell my 3 pairs of different memories that i run now using 2 different kingston ram modules and one cant remember the name that is an 2133mhz memmory while the otherones is 1600mhz but its rocking on like hell no problem running an highly overclocked 290x will mostly get another one when they are getting abit cheaper used. But my biggest problem for my x5660 @ 4.5ghz system is im kind of low on diskspace. Does anyone know what Asus ramapge 3 extreme have as an limit at how big hdd you can use. Can i buy an 3tb or an 4terabytes drive and everything will be okay im using the latest official bios. Knows there are alot of modded bioses so if 3tb or 4tb disks wont work maybe an modded bios could do it? But dont want to wreck my lovely x58 system that is the best computer ive ever had got the motherboard for free after i got an breakin an they took everything later i could from small money 18months ago build this lovely thing. So if 3 or more tb wont work then i well must stack up with alot of 2tb drives then and loose areflow in the fractal defign f4 chassi :< I hope someone here has an happy news and says it can support 3tb or higher drives need for my 4k videos and my big pictures :(

    Best regard from Marcus

    • Hey Marcus,

      There have been reports of addressing issues with >2TiB drives on ICH10R, but those are all software problems. To make sure you don’t run into any issues, you should not only be running the latest BIOS, but also the latest possible Intel chipset / RST drivers! There were some serious issues with Intel AHCI/RAID drivers regarding >2TiB disks in the past. Please see what ASUS themselves has to say about this regarding BIOS version 1502 for the Rampage III Extreme:

      The ASUS Rampage III Extreme support page, Drivers & Tools section, BIOS files, says for BIOS 1502:
      Rampage III Extreme 1502 BIOS
      Supports over 2.2TB huge HDDs and RAID.
      *Please install IRST 10.5.0.1026 when using this Bios version.

      But, please do keep in mind, that your machine is BIOS- and not UEFI-based. As such, you can only use MBR-based partition layouts to boot from, no GPT. What this means in essence is, that you cannot boot from hard drives larger than 2TiB. While it should in theory work with 4Kn drives >2TiB (4kiB physical and logical sectors), I wouldn’t count on it. Really, I wouldn’t. And those are still rare and expensive anyway.

      Bottom line: 3-4TB drives should work just fine as long as your software and firmware is solid and new enough, but only as data disks, not as bootable volumes. So your boot drive needs to remain at <2TiB!

      Plus, even if you do run into trouble on the Intel ICH10R (which you shouldn’t), your mainboard still also features JMicron JMB363 (1 port internal, 1 eSATA) and a Marvell 88SE9128 (2 ports) controllers. It should work on the Marvell at least, if it’s running with latest drivers. I’m using 3TB and 4TB drives on my older Marvel 88SE6121, and it works ok, as long as the drivers are new enough.

      So there you go! :)

      PS.: Some better text formatting (paragraphs etc.) and punctuation would make your post much more enjoyable to read, just saying…

  16. For all of you using a Xeon 5500 or 5600 series (Nehalem-EP or Westmere-EP), you might try Registered DIMMs as well. I have a setup with a Xeon L5520 and 12 GB PC3-8500R (ECC Registered!) RAM running nicely.
    The CPU brings along its own memory controller, so the chipset *should* not have anything to do with that. Except BIOS limitations maybe, but that seems to differ from manufacturer to manufacturer. My Supermicro X8SAX board works fine with the above combo.
    The ECC Reg. seem to cost less then ECC only DDR3 sticks, at least on eBay.

    HTH,
    D2k

    • Hello Duke2k,

      While it’s true that the integrated memory controllers have no issues with Reg. ECC, a lot of mainboards do. My own has all the necessary traces connected on the board (DIMM sockets to CPU socket), but the board just won’t bring up ECC. Also, I know of no way to verify whether the register buffers are actually working correctly, but I have run into stability issues on my P6T Deluxe just as well as on the P6T Deluxe V2. System freezes, black screens on POST, etc. Other people have reported similar issues. For some Desktop boards it works, for some it does not.

      It appears the ECC chips aren’t the issue. Pure ECC memory always works, it’s just inactive if the BIOS doesn’t activate the feature and/or if the ECC traces aren’t even there on the board. But as soon as you add the register buffers, a lot of Desktop mainboards not officially supporting Reg. ECC seem to start having problems using the memory in a stable fashion.

      So I guess one should try with some small, very cheap Reg. ECC first, to see whether it’s fundamentally working. But when going for 96GB (if anyone’s ever bold enough to attempt that on X58), the buffers probably really need to operate properly to ensure stable operation…

      As to why some boards have issues with register buffers despite having the copper traces connected – no idea. Maybe a layouting problem or some BIOS crap.

    • Hello,

      could somebody more experienced maybe help me with my P6T7 WS board.

      I purchased 6 sticks of 4GB Corsair Ripjaws F3-12800CL9S-4GBRL memory for a total of 24GB but I can’t get it to boot to save my life.

      Tried one stick in any blue slot, and then in other combinations as instruced in the manual.

      All I get is a black screen.

      Works just fine when I switch back to Corsair Vengeance 4 x 2GB.

      Thank you in advance!

      • Hi Dlajo,

        You mean G.SKill Ripjaws… Those aren’t Corsairs. Not that it matters, that memory should work on your mainboard just fine. You’re within specs as well with those 24GB of yours. Off the top of my head I couldn’t think of any reason as to why that wouldn’t work. All other people who’re using 24GB @ P6T7 WS seem to do just fine with that, although I couldn’t find anyone with the exact same memory.

        The very similar [triple channel kit] based on the same modules even supports your mainboard specifically, see QVL tab. Strange.

        I know this isn’t very helpful, but maybe you could test the modules in another machine, if possible. It’s unlikely that all six are dead, but I just don’t have any better idea at this moment. :(

  17. hi thrawn, i test on my motherboard Rampage 3 extreme for 30G of ram and its ok, using xeon w3680, will try on the 48G later, thank you for the info

  18. Just installed 36GB Corsair Dominator GT (4 x 8GB + 2 x 2GB) 1866Mhz on ASUS Rampage II Extreme with i7 940 and it works fine. Have not tested it for more than a few boots but it looks good.

  19. Hi,

    I have a Asus P6X58D-E that should be limited to 24 Go.
    I tried 48Go of memory:
    – G.Skill Ripjaws X F3-12800CL10D-16GBXL => Only 32 Go detected.
    – Crucial Ballistix Sport DIMM Kit 16GB, DDR3-1600, CL9-9-9-24 (Model BLS2CP8G3D1609DS1S00CEU) => Only 32 Go detected

    So the Crucial memory is not working with every X58 motherboard

    • Hello Snow,

      It’s a pity that it didn’t work for you, but still, thanks for reporting this, it’s valuable information for other people after all. I did expect that there’d be trouble with at least some boards though, we’re fooling around with completely unsupported stuff after all.

      Out of curiosity: Did you try to fool around with your chipsets’ settings a little bit? Like relaxing memory latencies and/or raising DIMM voltage a little? Not saying it’s necessarily gonna help or anything, but maybe…

  20. Well, I have the Asus Rampage Formula III X58 and since my CPU is the Xeon X5650, I am hoping I will get luck as well. :D
    I currently have 12GB so I will try the 24GB of Kingston HyperX Savage first and if that works, will then add the 12GB of Crucial Tracer LED ram and test again. My goal would be all 48GB of course.
    Wish me luck!

  21. hi

    this is a rather long thread and i couldn’t read every single post and replies etc …

    but sounds like you have perfectly been able to do what i have been thinking and dreaming about for quite a long time now! ;-)

    my mobo is Asus x58 Sabertooth … not sure what its chipset is but i can check anyway … my CPU is also i7 but the 950 one, which is not exactly the same as yours although it sounds close enough …

    recently, after checking it out with a salesperson where i bought the components to build my workstation four years ago, he told me it’s impossible to upgrade my system to more than its present 24GB configuration … but searching around online and asking more someone sent me this page’s link and now it seems the salesperson was wrong! :) (or wasn’t?)

    anyway, after reading more here, now i realize there are 16GB (as well as 32GB) modules available out there too but apparently, the DDR4 16GB modules are 288pin instead of 240 … and if i got it right, the x58 mobo’s memory sockets are 240pin …

    right?

    also, would it be worth it to try 16GB modules on this mobo although i’m finding them to be rather too costly at this moment and beyond my present budget …

    any feedback would be greatly appreciated, thanks!

    • Hello Shawk,

      Your chipset is the “X58 Tylersburg”. And the memory you need is DDR-III. DDR4 won’t work. And the salesperson you talked to told you what the official specifications say. 48GB tend to work well on a lot of X58 mainboards, but it’s not officially supported.

      As for 96GB: While 16GB modules exist, only Reg. ECC modules have the necessary memory organization for this stunt. Regular (unbuffered) 16GB memory modules won’t do (Reasons somewhere below). Also, Reg. ECC modules don’t work on all X58 boards, and if they do they may or may not cause stability problems. Also, 32GB modules won’t work at all, or may be detected as 16GB or smaller.

      On top of that, nobody has tested Reg. ECC 16GB modules on X58 yet, so to just buy those blindly is a big risk! Even if they’re detected properly, you NEED the register buffers working, or this will not run in a stable fashion. I’m not sure if Reg. memory will work properly on your Sabertooth. So there are a lot of big question marks with those 16GB modules.

      You’d best go for normal, unbuffered DDR-III modules (any speed equal to or above 1066MHz data rate will do) at 8GB each. That way you can reach 6×8=48GB, which is what works on almost every X58 mainboard. A much safer bet!

  22. Hell yeah,
    great thread. Tomorrow I am going to go and buy Gene II with i7 920 CPU! Looking forward to this “vintage” (1st high caliber intel of newer generations) locomotive! When I have read info about 12 GB Ram being max for X58 I started to be a bit nervous and though what a crap it must be. And now I see it surely is. So propably getting Crucials Tacticals 3x8GB for my 24GB gaming, media rig. Had a luck with previous Gryphon board and Crucial ram, hopefully will be compatible now too. And as a pinnacle here I ll swap 920 for X5650 for better performance and lower heat and power consumption.

    i just wonder… Does X58 accept 1,35V modules ok? Or do they have to be 1,5+? Thanks :)

    Thanks for this test of 48GBs. Made me happy.

    • Hello Waki,

      The DRAM voltage doesn’t really have to do anything with your chipset, it’s a matter of onboard voltage regulators and your BIOS. If you go ahead and look at the [manual] of ASUS’ Rampage II Gene, go to section 3-15 (page 85). There you can see, that the DRAM voltage range you can set goes from 1.51106V to 2.50481V. So in essence: No, it doesn’t officially support 1.35V modules.

      Interestingly though, Crucial themselves list some 1.35V modules as being compatible to the Rampage II Gene, see [here]. I guess they’re considering it perfectly safe to run 1.35V modules at 1.5V and to just go with that. It’s not like they’re going to go up in flames considering they’re likely based on the same 30nm manufacturing process anyway.

      But hey, if you want to be on the safe side, there are a lot of 1.5V 8GB modules made by Crucial as well. Ballistix series and such. :)

      In any case, I hope you’ll have fun with your X58 platform! To me it feels a bit like the ancient Intel 440BX, legendary quality stuff. It’s just rock solid, doesn’t die, and performs nicely for much longer than anticipated. :)

      I mean, I wrote that post in April 2013. At that time, I’ve been running that system for 4 years already, as I got it in February 2009, with a Core i7 920 as well. Then it was a Core i7 980X, and finally a Xeon X5690. And now it’s 2016 and I’m still using it, and it’s still going strong! Talk about some long-living hardware, a rarity these days? With a Titan Black, a SAS/12Gbps and USB 3.0 upgrade, everything’s still nice and shiny. ;)

      • Thanks for the message,

        I have read about being dangerous to use 1,65V+ rams for X58 but the newer one also, for long period of time, so, if there will be 1,35V in bios, I guess it does not hurt to order some low vol ram to the shop I work at and try them. Got the board and cpu at home now, but have not swapped it yet. I have tested it in man’s home tho. All slots filled with 4GBs and listed 24 as those should. Thanks once more. :)

        • Hi Waki,

          1.65V are still fine, even according to Intels’ own recommendations. It has just been said that you shouldn’t go above that, as the voltage seems to be applied to a few parts of the CPU itself as well, and 32nm Westmere cores weren’t made for voltages that high.

          In any case, 1.5V are completely safe of course, and 1.35V should be fine as well I presume. Note however, that you will be driving 1.35V RAMs out of spec on your board (because you can’t go below 1.5V in your BIOS). Not a problem in my experience, but if you absolutely want to stay within specs, 1.5V are the way to go.

          • Howdy,
            yep. You are very correct, Sir. Assebled now and waiting for X5650 now, so the 920 could be sold for some more juice. The ram in bios is truly 1,51 minimal. I am sure 1,35 are fine. Even better, those can be raised to 1,5 and with higher clocks as I did with a nice and quite cheap Ballistix Tacticals I belive those were 1600 8-8-8-20 with 1,35. I had those with 2133 with 1,5 and 9 or 10 clocks. Cannot remember now 100%. I will probbaly run it stock now, as the power won’ be needed that much and hopefully the pc will be strong for another bunch of several years as previous owner did not overclock the thing at all and used it mostly for casual stuff and sometimes a mild gaming. :)

  23. I feel like I’m bringing this back from the past but had a few questions after reading all the comments.

    I currently have an Asus P6T7 WS Supercomputer motherboard and am getting a Xeon X5675 for it.
    I am not looking to overclock etc but to build a large NAS.

    Have you been able to test if 96gb of ram (or even 16gb modules) works on another mobo? Also how did you get Registered ECC ram to work when most X58 mobos only support unbuffered ECC (if they support ECC at all)? Could i run registered ECC ram on my P6T7 and still have the ECC functional? I am planning a FreeNAS build with this system and need as much ram as possible, and would love to go for 96gb if you think its possible.

    Thanks

    • Hello Gutray,

      I’m afraid I still cannot say for sure. I’ve never tested 16GB modules myself (too expensive just for a test, and we don’t have any at work either). You can see the theoretical details regarding this [two posts further down].

      Also, some mainboards don’t officially support register buffers, but the modules still work somehow. There might or might not be stability issues however. I’ve seen it work nicely on some board/RAM combinations, and I’ve seen it behave very weirdly on others, like giving black screens on POST sometimes on my P6T V2.

      FreeNAS, so ZFS, eh? Probably going for data deduplication? If yes, I can understand your hunger for more RAM, but instead of shelling out a ton of cash for a risky memory upgrade, you might want to consider a fast HET-MLC SSD as a ZFS’ L2ARC device instead. Maybe PCIe-based or so. Would be slower than DRAM of course, but it’d be a safer bet, and probably cheaper as well.

      It seems nobody is willing and/or able to try 6 × 16GB of Reg. ECC on X58…

      • I am more than willing to give it a shot with 16gb modules if I can find a cheaper source, or maybe one that would sponsor me trying for the sake of the community. I will probably start with 48gb of ram and then do a 16 x 3tb array, with 4 parity disks and 2 hot spares so I do have some left over memory to run other applications like plex, couch potato and a few others. Also 48gb of ram lets me use unbuffered ECC which I know my mobo can support. I would still love to explore the 16gb module idea just because it would be fun to figure it out. Thanks for all your great information. I will keep you posted on the build, i have to sell off my 2 Netapp Storevault S500s first before I can start the next build.

        • Hey Gutray,

          You could try just two 16GB modules first to see whether they even work at all. If only I could talk my boss into upgrading our Proliants with more memory and hexcore processors, I could do the tests. But no such luck. ;)

          In any case, I’m looking forward to seeing your build come to life!

          Ah, and once more, my apologies for this blog being so slow upon commenting. Heavy blogging software from 2016 takes its toll on my 200MHz server. ;) I hope it’s at least somewhat tolerable.

          • Thrawn,

            No need to apologize for the site, it runs great in my opinion. Right now I think Iam going to start with 48gb, get that running stable, and then look at playing with 96gb or testing out the 16gb stick in the future. DO you know if I need any special BIOS for the P6T7 to run the Xeon X5675?

            Thanks,

            Gutray

            • Hey Gutray,

              According to ASUS’ [CPU support list], the Xeon X5675 is officially supported by your mainboard since BIOS version [0811], with the latest BIOS being version [1001]. So you should be good to go, no mods necessary. You have a workstation mainboard after all. ;)

              If you decide to do this and if you make any progress (or even if you fail to get 16GB modules working), please let me know, so other people can read about it here as well. Thanks! :)

              • Well, today, my new Xeon x5650 for GENE II arrived and working wonders. Temps, power, a very nice bonus, the temp of NB and SB is also bit lowered, due lesser heat from CPU. Recommending one to everybody, compared to ordinary 130W beasts :3

                • Hey Waki,

                  I had a similar experience with my X5690 vs. i7 980X. Due to the lower VID and probably a higher quality core with less leakage current – just assuming there – the Xeon simply runs a bit cooler at the same or even higher clock rates. My Noctua NH-D15 is kind of a waste for such a cool CPU, given I’m too lazy to even try and overclock it this time around. ;) Growing old… :roll:

                  • Well, I though I will totally ignore the option as I had decent middle class tower cooling, but for some reason I am damn looking too much on NH C14 for better cooling and mostly for blowing the cold air on motherboard VRMs, NB,SB for better temps… So tempting… Someone is selling the smaller cheaper version (has 14cm fan as well i think) but not sure if it is mountable inder the heatsing… C14… C14 damn…So sexy…

                  • Hello again, so I made myself “happy” and got 24GB of 3×8 Crucial ballistix tacticals. The thing is, it wont boot or post when set to 1600MHz and that makes me really unhappy. tried declared 1,5 with 888 24 timings and no go on Rampage II gene. It will post, boot fine on 1333 any tips? I do not want to be limited by ram when OC. Thanks for tips. Running with X5650, 2SSDs, 3 HDDs.

  24. Hello Thrawn,

    I can confirm too 48gb definitely works on x58 platform.

    My system:

    Motherboard : Gigabyte GAEX58-UD5 ver1. 2
    CPU : Xeon X5690
    Memory : 48 GB corsair vengeance pro DDR3 24000mhz CL11

    @now the aim to shoot for would be 96GB (for me important as I do use multiple VM’s often).
    Tried already with a kit of server RAM (16GB x module) , Hynix part, but the memory was not recognized , the board did not booted…suppose registered RAM will not work on my board.
    Is there any non ECC/registered parts 16GB x module to be found ??

    Best Regards
    D

    • Hey Dani,

      16GB unbuffered modules do exist, for instance the Transcend [TS2GLK64W6Q] or the Crucial [CT2K204864BD160B] DDR-III/1600 CL11 1.35V parts (1.35V is supported by the Xeon 5600 series).

      The Transcend is a 2-Rank module, which is an absolute must when attempting this, quad rank won’t work at full population. So, given that the Xeon 5600 series can do at most 8 ranks per channel, this would mean 2R × 3 2 DIMMS = 6 4 ranks per channel, so that’d be fine as per the [Intel® Xeon® Processor 5600 Series Datasheet, Volume 2], table 3-1, page 57 ff.

      Now as for the DRAM IC density and maximum width, this is where the crap starts: Maximum density sits at 4Gbit per chip, that’s 512MB. However, the maximum ranks are 2, maximum data link width is x8. So you can get 512MB × 8 ICs × 2 ranks = 8192MB or 8GB per stone. The Transcend module has 2 ranks and x8, but it needs 8Gbit / 1GB DRAM ICs to achieve its 16GB capacity at those specs. This’ll likely not work.

      As far as I can understand it from the specs, you’d need to break through either the IC density wall, the link width wall or the rank wall to get to 96GB. I’m not sure which one is the most probable to work, but in my experience, using more ranks than what’s specified is truly impossible.

      Registered DDR-III on the other hand can be driven at 4 ranks with a data link width of x8 ICs. And that’s how they reach 16GB/module. According to the data sheet, this is not supported for unbuffered memory.

      Bottom line: One would need to test whether either of the limiting factors may be ignored. This will very likely be extremely shaky though, running 96GB of unbuffered memory on an X58… Also, you need to know the exact specifications of the memory you’re about to test (density, width and ranks) so you can understand what it is precisely that you’re attempting.

      Unfortunately, I can’t really test this with any 16GB unbuffered memory sticks myself at the moment… Don’t wanna buy them for just that to be honest.

      • Intel is mostly not officially supporting 8Gbit DDR3 at all and I think in pre-Broadwell systems they will not work at all.

        There is actually a good way to determine the maximum DDR3 DRAM capacity of a Intel memory controller (with the exception of 8Gbit DDR3), and that is to look at the number of memory address lines. If the address lines only goes to A14 for example like with most of the Core 2 chipsets, than only up to 2Gbit DDR3 can be supported (4Gbit DDR3 would require an extra A15 address line).

        Note: This post has been edited / merged with another one by the editor.

  25. Hello,

    Sorry for my bad english, i’m french :)

    I have got 48gb with my x58 too and i confirm it’s work perfectly. I have buying these memory 1 week ago.

    Motherboard : Asus rampage 2 extreme
    CPU : i7 980x
    Memory : G.Skill Kit Extreme3 2×8 Go 2133 MHz Trident X CAS 9
    G.Skill Kit Extreme3 4×8 Go 2133 MHz Trident X CAS 9
    GPU : Evga GTX 770 4gb superclocked with ACX cooler

    CPU OC : BCLK 133 x ratio 30 = 3990 mhz @ 1.30v
    Memory OC : 1866 mhz cas 8-9-9-26 @ 1.60v

    • Hello Thunder77,

      And that’s the second report of 48GB of RAM working on the Rampage II Extreme, thanks for reporting back, and for stating the memory modules you used! ;)

  26. Needing to buy a new motherboard to replace my Asus P6X58-e Pro and your article has given me hope of being able to replace it without having to buy all new components or decrease the GB of memory! Do you know if the increased memory can be used on any of the LGA 1366 boards?

    • Hi tktucker,

      Unfortunately the answer is no, I do not know. While chances are high that it’ll just work, you can never be 100% sure with anything that wasn’t part of any original specifications. There are just too many things that might get in the way, be it BIOS code or PCB design or whatever…

      All you can do is to look up the mainboard you’re interested in on the web and check whether people have been successfully running it with 48GB of RAM. Or check the comments here, there are a few mainboards reported to work fine.

      Out of curiosity: Are you replacing it because it’s dead? If so, you could just get the same model off eBay or something, that’s what I did with my P6T Deluxe as well (although it turned out the board was fine after all, what has broken was just my power supply, weird symptoms, which I had misinterpreted at the time).

      • Thank you so much for the response! It is definitely the Mobo that is dead. We have been searching the internet for the last few weeks trying to find another P6X58-e Pro but have only found some that are comparable (Except for the max ram). We are probably going to purchase an ASUS P6X58D Premium and hope we can get our Ram to work on it, as well

  27. Success with 48G of Kingston HyperX in a Rampage II Gene with a Xeon x5680.

    CPU > 4.7GHz
    RAM > 1.9GHz CAS 8

    • I have Crucial ECC and can’t get it above 1648 Mhz, 1.9Ghz, that gets awfully close to DDR4 speeds, WOW! Not only is the speed super close to 2133 however you benefit from the lower CAS of 8, INCREDIBLE! Are you using this for gaming??? At 4.7Ghz, what is your passmark score??? 4.2 gets roughly 12,000, so at 4.7 with nothing else running you should achieve a 14,000 passmark score, putting your six-core in the league of the brand-new 5930k, NICE! With the kind of results your hardware has accomplished there is no need to upgrade, even if you’ve got the money, SERIOUSLY! I wonder what the power consumption is going to be with 4.7Ghz compared to the stock consumption of the 5930k, which uses 10 more watts stock compared to your x5680’s 130 watts. Now that AMD is basically on the verge of death these Xeon’s will become EVEN MORE VALUABLE, HANG ON TO THESE FOR DEAR LIFE!

      • I’m running Ubuntu 14.04 and using the machine as a workstation. I’ve been running the Isabelle interactive theorem prover. I haven’t run any windows benchmarks or played games.

        I made it mainly from old parts and happened to use a HTPC case. The OC was limited by VRM cooling as I had replaced the thermal pad with arctic ceramique which dried out over the 6 years I had the MB. I’ve now redone all the thermal compound and crafted a new VRM heatsink and cut a hole in the case to cool the back of the MB so the VRM temps are now good. I want to have another go at getting 5GHz stable.

      • Just to be clear, the RAM speed I got was > DDR3-1900 so > 850MHz not the > 1.9GHz of my previous post unfortunately.

        With the new cooling for the VRMs and a little more VTT I managed to run my tests at 200×25 for 5GHz CPU and DDR3-2000 RAM @ 8-10-11-24. CPU is at 1.66V, VTT at 1.36V. Room temps were very cold and the CPU cores were hitting 85C. Hyperthreading was disabled.

        209×24 with DDR-1650 RAM @ 7-9-9-10 is getting there but not quite stable at the same voltages. Maybe need a bit more VTT (and a spare processor just in case this one goes pop!).

        • The 850MHz above should read 950MHz obviously. No preview or edit?

          • Hello Harry,

            Quite the machine you have there, I’m amazed that you can even hit 5GHz at all, and wow, your VCore is completely insane (you may wish to consider electromigration effects in the long run). This is still on aircooling?! If yes, then you’re a brave man! :) Well, those S1366 Xeons are relatively cheap by now anyway.

            As for your critizism regarding the commenting system here: I hear you, but as it is now, I can’t do much about it. To allow users to edit their posts, I would need to let them register here. This poses two significant problems:

            1. An additional security/spamming risk. This page is already under attack all the time and had to be hardened against break-in attempts. There are tons of anonymous spamming attempts every day as well, bogging down the server, which brings me to…
            2. Severe performance issues. This is a heavy weblog software from 2015 running on a [quad Pentium PRO 1MB 200MHz server]German flag from 1995 (with “fast” SCSI RAID and 2GB 66MHz ECC+P FPM-DRAM in 4-way interleave). If I let you register, you’d be served dynamic content all the time instead of server-side precached static HTML. Let me tell you, you don’t want that, because the user experience is abysmal, performance-wise. Like 20-40 seconds of loading time per page.

            So there you go. I’m already caching pre-compiled PHP byte code on the server side, but it’s just not enough, so I’m also precaching the individual pages in their final HTML form to serve static content to readers wherever possible. You might have noticed, that posting comments here is extremely slow? That’s because it needs to update/serve dynamic content at that stage, by calling PHP code.

            It’s horrible! But the server is one of my projects I just can’t give up, and I’ve done all I could to speed it up. I just picked the wrong software after all, but now it’s too late.

            I might be able to do something about the preview idea though, need to check whether there’s a plugin for that. It’ll be slow, but maybe better than having to double-post.

    • After LGA 1366 Xeon’s the generations of Xeons afterward SUUUUCK since Intel locked down the multiplier settings, and removed BCLK. Just look at the generation immediately after 1366>namely LGA 2011, ALL of the Xeon’s SUCK. Intel took advantage of people’s stupidity and sold them (mainly businesses with stupid owners) knowing people would be unable to extract any additional performance via overclocking. For PROOF, just try overclocking an e5-2670…the highest clockspeed achieved was only 1 ghz faster, for 3511 mhz, not enough to compete with the 5960x, let alone an x5680. Gee I wonder why that is? Intel knew from a while ago what they could make an 8-core do, this is why they make so much money, they release a sub-par CPU, jack up the price, sell it out, then years later release another one that can do more, only to charge even more because of course the motherboard must be replaced and memory must ALSO be replaced, ridiculous! They only care about profits, it should be obvious, unless people really are THAT stupid. I know Intel is the best, they’ve proven it 80% of the time (except for the period between 1999-2004 when AMD actually took them to task), however, their tactics are low and encourage corruption in an industry that should encourage innovation instead. AGAIN, ANYONE STILL USING LGA1366, STAY RIGHT WHERE YOU ARE, DON’T BE FOOLED BY THIS CORRUPT INDUSTRY!

  28. Hey Thrawn,

    TQ for this article – currently on 48GB of memory with my 980X on my MSI Eclipse SLI.
    Managed to utilize all my 6 DIMM slots with a 8GB sticks of Kingston ValueRAM 1600Mhz.

    Just to share some information – since the IMC is built into the processor, you may need to loosen the Command Rate from 1N/1T to 2N/2T as using anything with 4 or more DIMMs would not be possible.
    *Tried to increase the DRAM voltages & loosen the other DRAM timings but none worked except to run it at 2N*
    *Previously on 1N with 24GB (8GBx3) & was y-cruncher stable*

    • Hi cOMMANDER,

      So you used Kingston sticks, that would’ve been my other choice if the Crucials wouldn’t have been cheaper, since both have extremely low return rates. :)

      I never actually ran into the timing issues because I was too lazy to fine-tune the latencies anyway after going from 24GB to 48GB, leaving them at auto, only setting the clock speed and VDIMM to 1.65V manually. My P6T Deluxe switched over to 2T by itself after going from 3 sticks to 6 sticks for the first time. Since my switch to the Xeon X5690, my latencies are very relaxed at 11-11-11-30 2T, and that’s at a low 1333MHz data rate even.

      I’m just too lazy to overclock right now. ;) Plus, the machine has been doing video transcoding work for 90-100 days 24/7 now, and it still has quite a lot of work to be done, don’t even want to reboot at the moment, so it’s all stock speed right now. Boring, I know. :)

  29. I wonder what about the Xeon 3400 platform. It seems that quad rank x8 16GB registered DDR3 are rare at this point. I think that CPU is still limited to 36-bit addressing.

    • Same goes for the 3600 Westmere series, those have 36 bit address buses as well. You’ll definitely need to go for the 5500 or 5600 series if you want the wider 40-bit address bus to break the 64GB RAM limit.

  30. I’ve been using a Gigabyte X58A-UD5 with ECC unbuffered ram, and I can turn on the ECC option in memtest. I guess this means the ECC traces are there? I also found that registered ECC works if they are 2r x8, unfortunately most cheap server pulls are 2r x4.

    • Hi JH,

      I’m not sure about how the latest versions of Memtest86+ work, but the older version 4.20 is supposed to offer proper detection for ECC on Nehalems/Westmeres and older memory controllers. What you’re saying sounds like you got ECC and like everything’s fine, but I can’t say for sure without at least taking a peek at a photograph of your screen or a detailed description of what is being displayed by Memtest86+.

      One thing I can say is this: If you ARE able to run the ECC injection test, then you can be sure that you got ECC up and running as it should!

  31. Hello and very very thanks to dear Thrawn.
    Your comment are very usefull. Thank you again.
    I have P6T Deluxe V2 M.B. and I7-960 CPU. Can i apply the v2209 bios whit my cpu?

    • Hi PanoGraphic,

      Hell no!! The v2209 is strictly for the ASUS P6T Deluxe, but not for P6T Deluxe V2. Please do not attempt to flash v2209! If you’re referring to the Xeon support discussion below, I can provide you with the latest BIOS for your mainboard inlcuding all the Xeon µCodes, just in case you too wanna go for a Xeon processor. Here you go: [ASUS P6T Deluxe V2 ModBIOS v1202 with Xeon µCodes].

      If you do not require Xeon support, you’ll be perfectly fine with just the latest v1202 BIOS that you can obtain from ASUS directly, you can get it [here].

  32. I have another question, i have X5650, working on 4000 Mhz, question is – 32Gb DDR3 ECC REG will work on my rampage ii extreme with that processor or not ?

    • Hello Reyhs,

      Unfortunately, it’s impossible to say for sure. I only found [this guy]German flag here, who tried the Kingston KVR1333D3D4R9SK3/24G kit on your board, which consists of 3×8GB Reg. ECC modules, and he couldn’t even get the system to POST using it.

      Some X58 boards will simply refuse to boot when using Reg. memory, no matter what CPU you might have. To find out whether the Rampage II Extreme really is one of those, all you can do is buy cheap memory and try it out. Also, stay with as few ranks as possible (do not exceed 2R!) just to make sure! Problem is, that it’s impossible to make any reliable statement about any motherboard I do not own myself. :(

      On top of that, 32GB sounds like you’d be aiming for 4×8GB? If yes, you might want to reconsider, as you’d only be using two out of the processors’ three memory channels, losing some bandwidth in the process.

    • Hi, I have a Rampage2Extreme with 6gb of RAM laying around and since my HP xw6600 was sold to cover the rent I’m thinking to buy a cheap x5660 and boot it up, does it need anything special to boot like a specific bios or setup? It doesn’t have any cpu in it and I don’t have any available cpus to boot it and update. I’m currently using a HP dx2400 with a modded E5450. I found a xeon w3503 for 5 euros so I can buy it, update and run the x5660 if need be but theres a chance it might not work as well as it’s a xeon itself… Sooo any help is appreciated….

      • Hello Giorgos,

        The Rampage II Extreme doesn’t support any Xeons officially, not even with the latest BIOS version 2101 flashed in. I inspected the BIOS ROM, and I can confirm this, there are only Core i7 µCodes to be found.

        Now that alone doesn’t have to mean much. The fact that the BIOS doesn’t have the matching µCodes only means that detection may fail and the CPU will be using its stock µCodes. The BIOS may show something like “Unknown CPU”. That CAN be problematic for certain, very restrictive BIOSes. But given that Reyhs is running a Xeon on his Rampage II Extreme – likely without a mod BIOS – I think it should just work out of the box. ASUS BIOSes are not known to be particularly picky anyway.

        There might be errata though, and if any affect the systems boot process before it can reach the operating system (which will by itself patch in the latest µCodes it has available), there might be issues. This is extremely rare though. In any case, I’d say you’d get far enough to at least update the BIOS. And when you do, you might be interested in a version that actually has full support for your Xeon, featuring the latest µCodes for it. Which is why I prepared one for you right here:

        If all else fails and you’re left with a system that can’t boot that processor, let me know. I might be able to lend you a Core i7 930. It may turn out to be costly though, if you’re not within the EU (shipping, custom tax)… Just saying, as an emergency measure..

        • You’re a legend, downloading it immediately!! Thank you for offering your 930, it’s a gesture that means a lot,

          I didn’t want to spend 100+ pounds to test if it works and I bought the xeon w3503 without a heatsink for 9 quid (3+6 shipping to Cyprus), it hasn’t arrived yet and it’s a week late but if it works I’ll wait till after the summer and save up to buy a watercooling kit and a X5660/70/80 since I’ve got no coolers for it anyway :-p, I’m trying to built a good system for the lowest price possible since there’s no ‘payments’ system in Cyprus unless you actually take a proper bank loan with really high interest and I haven’t heard back from my latest interview, anyhow…

          @Mantus84 I tried booting up without a CPU and the LCD poster says CPUINIT so I guess the mobo is OK, waiting on the w3503 and if that works I’ll start bidding on either a X5660 or a 70, maybe even an 80 if I save up enough, I need a good power supply but I found a CX750M for 70 euros used, I’ll probably go for it since I need a better PSU even on the shitty dx2400 stock PSU case I currently have, thanks for the info mate :-)

          PS. I see we’re the same age too… ’84

          • Hey Giorgos,

            Just trying to help! :) Since Mantus84 confirmed a Xeon X5660 to work properly, you may not actually need the i7 930 or even my ModBIOS. If you get to the operating system (and if it’s fully updated), it’ll patch the latest µCodes all by itself at runtime anyway, based on the CPUID of the processor. All wide-spread OS kernels I know a bit better do this (Windows NT 5+, Linux 2.6+, FreeBSD 9+ at the very least). If it does truly NOT POST at all because of it’s older BIOS, we can still talk about the i7.

            Awesome to have a guest from Cyprus by the way, and Sweden too. :) I know virtually nothing about Cyprus really, other than its ethnic/political separation of course. It seems private persons are really in a sea of trouble if you can’t even use your bank account in a way I’d consider normal. That rules out credit cards and systems like PayPal just as much. :( Damn…

            Anyhow, the CX750M should be a good choice, the electronics are made by Channel Well Technologies, they’re well-known for their high quality server power supplies, so that PSU sure ain’t no crap. Got a Corsair for my new S1366 system as well, a unit built by Flextronics, a company which has a good name in the field of industrial power supplies, the AX1200i. Should be a nice box you’re building there! :)

            • I’m Greek originally, been living here the last 7 years, I can use my Paypal and everything else as intended it’s the payments/installments system that is f.ed up, Since until the crisis hit Cyprus every single person here literally had enough money to burn if it was too cold and not give a damn, That gave us the lower tier working force (I’m a barista) enough money to live nice and afford our hobbies without the need for anything under a BMW 320 full extra M Pack(roughly 45.000) to be paid in installments(which of course I don’t own but more than half the population under 30 does, the older ones go for Mercs and Jags and Astons), all that changed of course and as I left Greece right before the recession hit and never got actually affected by it till last year that it crossed the narrow sea (see what I did there? any GOT fans?) and since the status quo hasn’t had the need to incorporate such a system and our salaries dropped by half, plus an unexpected hernia operation right after quitting an area manager position at a local coffee chain and finding out that they haven’t been paying medical insurance on my benefits I am left here, trying to revive an ancient mobo I found at a friend’s house together with 6gb triple channel corsair dominator RAM and a GTX9800gx2 which I gave him 60 euros for the lot but he only asked for 40. I guess I was feeling guilty that it was too little :-p

              Enough sob stories, I’m really excited of accidentally finding a blog/ger that is so well informed and goes beyond the norm of tech forum “Gurus” being shitty or not sharing their knowledge, that little piece of info you gave me on the Corsair PSU is something that more people know about but won’t just tell you until you specifically ask “IS IT ANY GOOD?” and even then they will say “Google is your friend” or just “GIYF” .

              TL;DR thanks guys

              • Morning,

                Ah I think there are enough nice “gurus” out there who are willing to help, you may just need to dig through a pile of crap first, to actually locate them. ;)

                Quite the story you have there about your life though, especially the medical insurance part, that’s completely unthinkable here. Farther up north where I live, most people are still somewhat ignorant about the crisis, or… not really “ignorant”, they just can’t truly imagine it, never having experienced anything of the sort. I really pray to all nine hells that it won’t hit us too in the near future (I’m from Austria, I’m just blogging in English to keep my language skills sharp and reach more people).

                If my salary would drop by half, I’d be dead in the water! And if I’d lose my health insurance, I could basically hang myself, literally. But besides medical supplies, I’d have my expensive internet and a high electricity bill because I’m running my own, free home server (including this blog!), and it’s based on an [ancient Quad Pentium Pro 200MHz machine] from the mid-90ies. I’d need to give up the server, which would break my heart and leave several users of my free services dead in the water. I would probably be able to keep my flat, but I’m less sure about the car, my crappy old 1997 VW Passat… Sometimes I wonder about how blind (or alternatively: how malicious) the world leaders truly are.

                In any case, I believe I am the single last person left alive in the West who hasn’t seen a single episode of GoT. Yes yes, quite the blasphemy, I know. ;)

                But please do keep us updated on your progress with that Xeon, would be interesting to see whether it works (and with which BIOS version exactly). Also, it’d be interesting if the BIOS really reports a “Xeon”, or just some “unknown Intel CPU” or something. My P6T Deluxe was clearly missing the µCodes for my X5690, and still reported the CPU as “Xeon X5690”, which shouldn’t have been possible actually. So I wonder…

                • No worries about the Game of Thrones show, there’s always binge watching all seasons at once or not at all depending on personal taste :-)

                  The small HP/Compaq dx2400 I’m using has a Pegatron (it’s a subsidiary or spin-off of ASUS if I recall correct and they actually manufacture the iMac for the last couple of years) 2a73h motherboard with the chipset being this:
                  Northbridge – Intel G33 Express
                  Southbridge – Intel ICH9

                  the original CPU that came with it was a Pentium Dual Core E2220, I ordered a Xeon E5450 with the sticker on it that swapped two pins and just installed it without using the modified BIOS with the Xeon μCodes since when I checked they were already on it from the latest official update, bear in mind that the socket of the motherboard is 775 and the Xeon is 771 is what makes it strange for the μcodes to be on it, first boot black screen/forgot to plug in the HDMI ;-p, next 3-4 boots I got blue screen when entering windows, remembered I reset the BIOS and forgot to change the SATA to AHCI/phew, next boot everything perfect and the CPU is recognized everywhere as a Xeon E5450, my only problem is the the mobo doesn’t support OC and since I didn’t find any good diagrams for BSEL mod specificaly for the 5400 series xeon I just left it as is.

                • So…
                  CPU never arrived so I claimed and got my money back, the full 13.41 euros of it :-p

                  I will try and bid in a x5660 if it doesn’t go too high
                  :fingerscrossed:

        • Hi!
          Does the CPU Xeon 5600 series compatible with motherboard Asus P6T Deluxe v1?

          • Hello Serhio,

            Yes, I’m currently running a Xeon X5690 on the original ASUS P6T Deluxe, BIOS version 2209. The CPU just works even without having to patch updated µCodes into the BIOS.

            • Thank you very much for your researches, which useful for many people and for quickly answer!

              Sincerely, Serhio.

      • Hello mate i have an R2E and bought an x5660 on ebay for trying out and for streaming gta 5 and so on, Had the latest official bios from asus homepage just swap the processor works like an charm remember to do a bios reset. Using it now in 4.5ghz with no problems at all. done all stability checks and everythings working awsome. Best regards //Marcus from sweden

  33. The decentralized Yacy search engine you have running, is this on your X58, or is this also operating on your XIN.at server??? Considering your HTML isn’t 5th generation it’s still surprisingly modern, just saying! So your XIN.at server has a total of 160 GB of storage, since it’s in RAID mode, is all of it still available, and how much of that storage are you allocating to this site, including all the high-res images you have plus software…all in all, really cool! I wonder how much power your XIN.at server actually uses???

    • YaCy is supposed to be running on the X58 platform, cause it can eat a ton of RAM, depending on configuration. The more RAM you give it for its search index, the less disk I/O will happen, and it can be 64-bit, so no practical limits. It’s down at the moment however, because of the stability issues on that platform right now. I am still unsure as to what’s really causing the problems, but the RAM was/is one possible candidate. That’s also why I’m interested in ECC. :)

      If you’re interested in system load, you can check it out [here] (this also takes some time to load, because the graph anti-aliasing is unfortunately single-threaded, please be patient).

      About storage, it’s a 36GB HDD for the system at this moment, with ~17GiB being used. The data is stored on a regularly backed-up RAID-5 with 68.3GiB capacity and 53.8GiB being used, although about 10GiB could just be deleted, scratch data. This is for about 10 users at this moment, including myself. This WordPress weblog here – including all images and files – consumes 1.59GiB right now. Not too much, eh? The SQL database it’s using is 61.5MiB in size. Not slim, but acceptable.

      Some people I know personally (including my relatives) prefer to use my services, because they come (to some extent) as trustworthy, secure, free and with no abuse involved. If I would mess up, they’d know who to blame. So I’m kind of accountable, which serves to build up trust. This is not the case with other opague organizations on the web (think: Google, Amazon, Facebook, blah blah).

      So yeah, I am a big fan of decentralized, user-driven services that do not rely on any of the big players. :)

      Ah, and if by “power” you mean “electricity”, weeell, including all the networking hardware it usually sits between 470 – 525 watts. It sure ain’t cheap. But what hobby is? ;)

      • 475 watts is about what my AMD Opteron consumes since it’s got a 1TB WD Green, AMD Radeon 6850, and a 4GB Gigabyte RAM Disk operating at similar frequency to the 4GB on-board. You could say I actually have 8GB RAM, just in separate areas, all housed in a semi-portable shuttle PC case, oh yea and gigabit ethernet too so it’s definitely capable hardware even today :D. I’m absolutely amazed that your XIN.at server is so energy efficient given it’s age, AWESOME!

        • You’d call that energy-efficient? Most people think I’m completely stupid for running such hungry hardware 24/7. ;)

          Especially the people from Germany next door (I’m from Austria) totally don’t understand that. But you can see why, when looking at their energy prices. Here in Austria, I currently pay my energy provider 6.86¢ per kW/h, which can be considered a highly competitive price. That plus 26€ operational cost a month for the network operator.

          In the end, my entire household costs about 86€ per month in 100% water-generated electricity, with about 800-1000W constant 24/7 load. However, in Germany I couldn’t afford that so easily. Average price per kW/h there is roughly 25¢ with operational costs in the range of 7-10€/M! That’d put the price at roughly 210-220€/M, more than twice as much! And that’s the same for their nuclear-fission-powered electricity, not the eco stuff I’m buying!

          That’s why I had to completely re-evaluate my approach when asking German people to run their own home servers for a bit more independence and privacy. Typically, they’d rather go for an Atom or even Raspberry Pi machine, and understandably so.

          It really depends on where you’re living…

          • Solar Power can eliminate much of the energy cost, IF you can get through the initial investment!

            • That’s not the only issue. I’d agree if I were living in Nevada/Arizona, or maybe in Oman or something. But where I live, solar power is highly unreliable, and mostly used only as an auxiliary source of energy for warming up water or something.

              In Winter, you can’t get much out of photovoltaics here, as the sun won’t even go near it’s Zenith anymore. Plus, lots of clouds, which you may also get for several days in a row in summer. Also, the frequency of thunderstorms in summer has risen in the past 20 years too, further reducing the amount of sunny days per summer.

              The cells are also horribly expensive, and I can’t even mount any, because I don’t live in my own house, but rather in a flat. And our housing cooperative has used up the money we accumulated for necessary renovations (new roof, complete thermal insulation of the building), so things like solar cells are completely out of the question for the next 10-20 years or so. ;)

              In Germany, the situation isn’t much different. People who are building a new house for their family would consider photovoltaic cells, because when compared to the cost of an entire house, the cells are a minor factor, and you can get some subsidies from the government in some areas for that. But retrofits? Not common I believe… Cleaning and pricey battery replacements also need to be considered.

              Currently, battery-backed solar cells are often used for bus stops and street lanterns that have been upgraded to LED lighting. But a ~500W server and a ~300W (when idle) workstation, that’s a bit too much. :)

  34. https://www.youtube.com/watch?v=FJyXSmiWMHg , in the first video I get a good overview of your board’s BIOS and I must say it’s remarkably similar to my own, almost identical even! https://www.youtube.com/watch?v=nYp_tnC5Ho8 , this video gives a nice visual basis for the X5690 to start from, one thing I’ve noticed is that the i-7’s are chopped up from their TOP forms, aka, 920 is severely chopped up form of X5690, whereas the 980x is an X5680 with certain elements disabled, it’s all the same tech just cut into less complex forms or with features disabled as is the case with the 980x. What I don’t understand about your board at first glance though is how your board gives difficulty when producing stable overclocks, and then after looking at images of your board, I see now what it is, your VRM’s! The problem isn’t that your VRM’s are solid, and therefore good, they appear to be similar to my own, the issue is, if you compare your board’s layout to that of my own, the Rampage III Gene, there is one BIG difference in both layouts, and that’s the placement of the grounding plug!!! The spacing between your vrm’s and your io’s are too narrow, which impedes cooling from what I can see, this explains your board’s instability at higher frequencies! There is a way to improve this, and it’s fairly simple, just mod your heatsink that attaches from your sb, nb, and vrm area by removing the crappy thermal pads that ASUS provides, and replacing it instead with a more permanent solution, namely, a special mixture from Arctic Silver called “Allumina”, this is basically two different pastes (part A and B) which combine to form a PERMANENT thermal paste. Keep in mind that although this is a highly effective thermal compound when combined, once the two compounds combine it hardens (and bonds) within 5-8 minutes and cures in about a day. After it cures it cannot be removed, EVER! I applied it to my Rampage III Gene and never have to reapply it and it works extremely well as my NB temps never go past 68c under load! I hope this helps you out and this is something else I’d definitely recommend you do, if possible, before your X5690 arrives, oh and some copper vrm heatsinks might help too. I strategically placed two on my board, one near the memory area, and one towards the back near my io backplate. I’m really excited that you made the jump to the X5690, WOOOT!!! :D

    • Hmm, optimizing the VRM and MCH/ICH cooling? Well, why not, it’ll take months before that machine is really completed anyway (because building the RAID will take more time, given the insane SAS hard drive prices), so why not. I still have several good thermal compounds lying around, and lots of old heat sinks I could cut into proper shape for additional cooling.

    • Hey again, WOOOT!,

      It took a bit of time, because it was sent via a letter from Germany (which seems to be much slower than packages, which are handled by a different company), but here we go:

      Intel Xeon X5690
      (Xeon X5690, click to enlarge)

      Reg. & ECC Testing will commence either today or tomorrow on my secondary P6T Deluxe using two Hynix HMT125R7BFR8C-H9 modules, which means 2×2GB Reg. ECC DDR-III/1333. I will let you know how it goes!

    • And here we go.

      Despite the missing µcodes, the X5690 is seemingly detected by the BIOS right away. Not sure how, but yeah. I still patched the newer ones into the BIOS afterwards, just to make sure:

      P6T Deluxe BIOS v2209 detecting a Xeon X5690
      The ASUS P6T Deluxe BIOS v2209 detecting a Xeon X5690 (click to enlarge)

      The ECC part however, well… I can only run memtest86 version 3.50, as all newer ones simply freeze or reset the machine immediately on any P6T Deluxe (V1) I ever tried, no matter which RAM, CPU or BIOS version. The memory speed is being misdetected, but the ECC detection should still work. It does detect the presence of ECC in the CPUs IMCs, however, it’s not being turned on on the memory itself, as you can see. Either because the necessary traces on the mainboard are simply not there at all or because the BIOS just ignores ECC. There is no option in the BIOS to manually switch it on:

      Memtest86 3.50 showing that a P6T Deluxe seemingly doesn't enable ECC even with a proper CPU and memory being present
      Memtest86 3.50 showing that a P6T Deluxe seemingly doesn’t enable ECC even with a proper CPU and memory being present (click to enlarge)

      Supposedly, Memtest86 has the best detection mechanisms for ECC being enabled on the memory, but we can’t ever be really sure I guess. Still, given the indicators I would assume that it’s relatively safe to say that ECC does not work on an ASUS P6T Deluxe, no matter which CPU you plug into it. Bad luck then, at least if memtest86 can be relied upon here.

      At least the Infineon TPM module I got really works in the board. But I’ll write a dedicated post about that some time in the future. :)

      Edit: Oh, and it seems to be running a nice low default VCore of 1.200V at full speed without turbo – meaning 3.46GHz – and also under full load. Currently, I have a tiny Intel heat sink mounted on it, and it’s been under full load for about 40 minutes now. The cooler is barely hand-warm! :)

      • The lack of ECC detection is EXACTLY what I saw during my 48gb memtest so this confirms that your board though it’s ATX is nearly identical to my microATX version, interesting! Oddly enough if you use HWMonitor in Windows the individual temp sensors on each ECC module do work on my board, this is a nice feature for monitoring heat output and mine never exceed 52 c under load :) It’s weird how you are forced to use 3.50 whereas on mine any version works and everything is properly detected including the speed of 1605mhz plus the cl of 9-9-9-24 without issue. The ECC feature is off as reported by my memtest…just like in your test…did your updated microcode resolve any instability with your memory? I’m thinking that your best bet for ECC might be to use the Hynix I recommended, or the Crucial…perhaps though I can only be 100% certain on the 8gb modules…

      • If only there was a way to add the ECC on/off option to our BIOS and upload the change to our current versions, similar to what was done with CPU microcodes…the strange thing is that the option is there on my oldest ASUS A8N-VM CSM, just that overclocking options are severely limited…what if it were possible to extract the ECC option out of that boards BIOS and merge the feature into our boards, after all, they are all ASUS boards…hhhmmmmm

        • Hey,

          Memtest is strangely broken on the ASUS P6T Deluxe. Only the first one though, the P6T Deluxe V2 (the one without onboard SAS) works fine. Maybe some weird BIOS bug or something. Some people say it’s a multicore bug and that disabling HT plus all cores other than a last one makes it work. It does change the behavior a little, allowing it to run a bit longer, but that’s about it. Must be a bug in the BIOS or in Memtest or both. No idea.

          Nice to hear that there are actually thermal sensors on your memory! And I always thought only FBDIMMs have those. :)

          In any case, I can try and take a look at a few BIOSes and see whether the option can be properly ported over. Changing existing ones is easy enough, but porting one seems extremely dangerous, there is a lot of data and information in the BIOS editor that I do not fully understand. Also I’m not sure if this can even be done without recompiling the BIOS from source. But who knows.

          Even that won’t help if the physical traces are missing on the board though.

          • I think the ECC traces are there, why else would your board even be able to recognize your X5690 WITHOUT microcodes??? If you can put the ECC code into your boards BIOS, and get it working I’ll pay you to add ECC on/off capability to mine!

            • You don’t need the ECC address lines / traces from the processor socket to the DIMM sockets for CPU detection. The CPU and its own ECC capability are being reported via MSRs (registers) I think. Physically, the detection data would flow over the QPI link to the X58 chipsets MCH, from there via the DMI bus to the ICH10R, and from there via either the LPC or SPI bus to the BIOS chip. Alternatively, the data flows directly from the CPUs MSRs to RAM, if the user requests it, via CPU-Z for instance. ECC traces being present or not doesn’t really affect anything but ECC functionality.

              So it MAY be the case that the copper traces are missing. I just don’t know for sure. I’d like to know the pinout of the DIMM, but on the back side of the mainboard, where you have multiple columns of pins. But I can’t find any pinout diagrams showing that. Otherwise I could make sure. Edit: Ah crap, no I couldn’t. Because the mainboard has multiple layers. Meh.

            • Dammit. I’m looking through a few ECC capable ASUS S1366 board BIOSes (P6T7 WS Supercomputer, P6T6 WS Revolution), and they don’t have any ECC options. It’s all fully automatic. I fear this is either impossible without hacking the BIOS code itself, or it’s just beyond my minimal capabilities. One would need to ask a real BIOS hacker…

              • What about the Tylersburg Supermicro???

                • I’ll take a look when I get home, maybe in 1-2 hours or so.

                  In the meantime, i found out that the ECC feature on modern AMD platforms CAN be activated at OS boot time when using Linux. To do it, you pass an override parameter to the EDAC (error detection and correction) kernel driver for AMDs IMCs. That will somehow reinitialize the memory subsystem without the need of a reboot, and turn on ECC even if the BIOS does not support it. See [here]!

                  Maybe I can study the source code to see what that part really does to the CPU. Also, maybe there is something similar for the Intel world?! If yes, any decent C programmer could write a program/driver that turns on the magic.

                  • Intel with their lousy restrictions :( . This is exactly the kind of thing I hear about time and time again how Intel purposefully implements limitations whereas AMD, knowing their tech is inferior at least gives the end-user a little more freedom to do with their hardware as they please, this reminds me of the Zosma quad-core I had that I unlocked to six cores while being able to overclock as well, all of this and having paid a quad-core price at the time! Intel simply doesn’t allow this without serious engineering knowledge! :(

                    • Yeah, well… That’s Intel for you. For AMD you don’t even need an Opteron for ECC, any FX will do.

                      In the meantime, I inspected two more things. The first being a few SuperMicro S1366 board BIOSes. They’re AMI too, and they do have more options. But even there, there is no ECC on/off switch. Automatic again. So we can’t get much out of that with more in-depth knowledge.

                      Second, I inspected the C source code of the EDAC drivers on Linux, that’s amd64_edac.c and i7core_edac.c from the Linux kernel source. Despite its name, the second is also handling ECC for Xeons, at least for Nehalems and Westmeres.

                      The code is a bit over my head though, I’d need more time for studying and understanding it (I am no kernel developer at all). For AMD there is a clear “force it” function, but the Intel driver is harder to understand. I thought that you’d flip the switch by writing a value to a machine-specific register (MSR) in the CPU, but I can’t quite grasp what’s going on in i7core_edac.c.

                      If the code is THERE, one could port it to a small kernel driver on Windows, and add a small user space tool to it, so that any user with administrative privileges could just switch ECC on or off at will. That is, if it’s even possible for Xeons like it is for AMD IMCs…

                      There is also an interesting thread about this on the [H]ard|Forums, see [here]. I’ve re-joined the forum and am contributing what I know there too. Maybe somebody more capable than me can pick this up and take it where I seemingly can’t.

    • Hello once more,

      I have studied the DDR-III DIMM and LGA1366 data sheets in depth in the past two days, and I had decided to verify with 100% certainty that either the ECC traces are or are not there on my mainboard. So first, I had to create proper pinout maps to determine where exactly each individual pin can be found.

      Here are the datasheets I used to determine that:

      And here are the pin maps. When looking at the socket, please note the yellow circles. These are the notches. Please pay attention to them, regarding socket alignment. In my case, the socket was upside-down on the board, so I had to rotate the image by 180°. Both the LGA1366 and the DIMM diagrams show the sockets like they can be seen when looking at the front side of the mainboard:

      LGA1366 ECC Pins
      ECC pins on the LGA1366 socket (click to enlarge)

      ECC pins on a DDR-III DIMM socket
      ECC pins on a DDR-III DIMM socket (click to enlarge)

      Next came the measurements using a digital multimeter, as suggested [here].

      First I verified that my measurements worked (the upside-down socket fooled me at first). When I could reliably verify the continuity of some DDR-III data pins like DQ[0] on the DIMM socket to DDR0_DQ[0], DDR1_DQ[0] or DDR2_DQ[0] (depending on channel) inside the LGA1366, I was ready to verify the ECC traces.

      The final result is quite sobering however. I can now say for sure that there are no ECC traces whatsoever on the ASUS P6T Deluxe. Since soldering cables to the pins to make those connections is highly unsafe regarding proper ECC operation, I will stop here. The BIOS is not even an issue at this stage – the mainboard simply doesn’t support ECC at all. And while I can’t say for your Rampage III Gene, you may wish to verify it yourself using a multimeter based on the data I provided. My assumption for ASUS would be that the ECC traces are truly only present on workstation mainboards.

      The end. :(

      • Well, unless you want to be really adventurous and solder…I’d just be happy having 48 (possibly even 96)gb of ecc memory simply switched off :( . It’s still worth investing in the ecc, since it is generally more reliable even with ecc switched off, and can even overclock a little bit too…so at least in my case, I can’t complain for now, knowing I’m still keeping up with the latest tech out there… I wonder though if perhaps the reason I could never get to 96 gb was because of the lack of ecc capability……even though everything else functions perfectly…oh and even ASUS workstation boards like the P6X58-E PRO don’t support ecc, though it does go up to 48gb…interesting…I think all the X5690 is doing for our boards is lifting a little bit of memory restriction in that we can use ecc, just not with it on…if only there were some efficient way to activate those traces…

        • Soldering cables to the rear of the DIMMs and the socket – the idea did cross my mind. However, when looking at some [DDR-III routing specs], you can see how tightly you need to dimension the lanes. I’m thinking it’s very, very easy to mess up the timing for the ECC lanes, thus defeating its purpose.

          I also have a certain fear that the same issue may affect the register buffers. I cannot prove that yet, but maybe the Reg. traces too are missing, and the buffers are simply being bypassed. I am unsure however if that’s even possible, I would need to do some more measurements for that.

        • Oh, just don’t buy that asus shite anymore, and get a real motherboard that supports ECC registered memory.

          And keep an eye on the memory prices and buy everywhere when there’s an error in the pricelist of one of their suppliers. That’s how I got 48 GB for about 60 euro, over two years ago.

          • That’d be the proper solution of course.

            I am – and I know this to be contradictory to using safety nets like ECC – still an overclocker however, and if I just get a SuperMicro board, that’s all over. I don’t wanna go back below 4GHz! ;)

            The reason why I’m replacing my ASUS P6T Deluxe with yet another P6T Deluxe is simple: That way I can spare me the OS reinstallation. Instead, I can just pull the SSD from my current system, plug it into the new one and reboot my you-know-which operating system. Cause it doesn’t like to boot from AHCI SATA controllers it doesn’t know yet. ;)

            • I just had an idea, if only it were possible to get an SR-2, doesn’t matter if it works or not, and use it as a donor for the ECC traces you were detailing earlier, then adapt and add them to your P6T Deluxe!!!

              • Oh no, that’s not feasible!

                The traces are integrated into the peripheral component board itself, and they will likely reside on multiple layers of the board. Modern mainboards consist of 4, 6, 8, 10 or even 12 layers stacked on top of each other. Traces may run on either the bottommost or topmost layer, however they may also just sit anywhere in between, see here, a 4 layer example:

                Multilayered PCB structure
                The multi-layer structure of modern component boards. Image is © Polar Instruments

                Plus, the way traces are routed is critical, and their length AND routing/positioning must be perfectly matched with the given layout of the mainboard. That’s what advanced board layouting software is for..

                The specs say that groups of ECC traces need to be on the same layer, and member traces of a single group should be isolated from each other by ground traces. Plus, groups shall not run in parallel on top of each other either.

                So all you can do is solder wires to the accessible pins, and there is simply no way for a layman to ensure that it’s going to be safe. Plus, it’s very risky work. One wrong pin, and you may fry the board, the memory and/or the CPU.

                Not all is lost though: One can still obtain a WS board from ASUS, like the P6T WS Professional or one of the other two I know of (P6T6 WS Revolution and P6T7 WS SuperComputer). Those do support overclocking as well as Xeons and unbuffered ECC out of the box! And all the way up to the X5690 too when using a new enough BIOS!

                • Those are fantastic boards, however, are there any MicroATX versions of these boards that also support ECC???

                  • Not to my knowledge, no. Given they’re workstation boards…

                    There are some µATX ASRocks for S2011-3 with ECC support, but I don’t really know any overclockable boards like that for S1366, not from ASUS or anyone else.

                    • Right, as I thought, only with X99 do you get both overclocking and ECC in a small microATX package, interestingly with X79 it’s the same as X58, no ECC, overclocking, and MicroATX all in one :( , kinda frustrating…especially since every other aspect works perfectly, right down to the temp sensors on the ECC memory…just no ECC…

          • THAT is the problem, for all the issues ASUS boards may have (which I agree at least from a support standpoint) they are excellent boards with outstanding overclocking abilities. Now, where in the hell do you find an X58 based board with overclocking AND ECC capability??? Only the EVGA Classified SR-2 has it, and it’s enormous! How about an ATX or MicroATX form factor with the same capability? Only the X99M I’ve seen from AsRock has it and even then the Xeon cpu’s it supports are multiplier locked, which isn’t such a problem for the older Xeons such as the X5690. Wouldn’t it be ideal to have a Xeon X5690 @ 4.5 Ghz, with 96GB memory with ECC turned on? At that clockspeed none of the new Xeon’s would be able to keep pace in single threaded apps with the X5690 running at 4.5 Ghz or above…so there is value in somehow adding ECC to a non-server board that can overclock, something server boards can’t do, to me this greatly diminishes their potential value, especially when you consider how overpriced server boards are!

      • Hello, thrawn! Appreciate your efforts, but I think you made a mistake somewhere. As example, ECC traces present on P6T SE. P6T Deluxe must have them too. To prove or disprove this assertion, I need high resolution image of motherboard’s back (memory slots area).

        • Hi Fedor,

          Hm, I’d say I made sure my results are solid, I retried my measurements multiple times. I also cross-checked with some data traces, for which I was able to confirm their presence. P6T SE is a newer board than the P6T Deluxe – which I think was the very first ASUS board for S1366. Maybe that’s why?

          Also, seeing the back of the board won’t help you. The P6T Deluxe (like all modern mainboards) is a multi-layered PCB, and ECC as well as ground traces can and are supposed to exist on different layers, as recommended by JEDEC.

          So, you’ll find a lot of traces seemingly missing to the naked eye, and yet they’re there somewhere inside the PCB. Of course if there WAS a clearly visible ECC trace, that would change things, but there simply is none…

          If you still want a photograph I can shoot one, but you’d need to give me a few days time at least.

          • OK, I’ll be waiting. Of course, I am familiar with the stages of the production and design of motherboards. Later, I’ll explain the reason for the need of photograph.

            • Hey Fedor,

              You know what? I know myself rather well. If I don’t do this right now, I’ll have forgotten all about it by tomorrow, ’cause I’ve got a lot of stuff going on right now. ;) So here is your photo right away (it’s a bit dirty, this is my first P6T Deluxe that I got in 2009, haven’t cleaned it yet):

              ASUS P6T Deluxe, rear DIMM area
              (Click to enlarge, 3072×2056, 2.26MiB, ~5 sec. loading time from this server)

              I slightly oversharpened the picture so you can see the surface traces better. Sorry for the optical distortion, but I just can’t do much better with the lenses and camera I have. ;)

              Unfortunately you can’t see the CPU socket from the rear because of the backplate, and I don’t really want to remove it…

              • Hi, thrawn!
                Thanks for photo, very good quality! :-)
                As I thought, ECC traces present on your motherboard and they are clearly visible for channel1:
                http://radikal.ru/fp/dc01ba4953f349fa880cacbf1d75ecde
                Can you verify connection between DDR3CPU pins? For channel1 on the photo: 39D36, 40F36, 45E33, 46G36, 158E37, 159F37, 164E34, 165G35.

              • Part of my comment was reformatted, I’ll try another way:
                DDR3..CPU pins for channel1 on the photo: 39..D36, 40..F36, 45..E33, 46..G36, 158..E37, 159..F37, 164..E34, 165..G35.

                • Hey Fedor,

                  Sorry for the reformatting problem!

                  I will probably not have any time today or tomorrow as there is a lot of work at the moment, and I’ll be home very late (and very tired). Maybe on Sunday if I am motivated enough…

                  Currently there is a lot going on, and I’m quite happy if I don’t have to lift a single finger at home. ;)

                    • Hello again Fedor,

                      Sorry for the delay, but I couldn’t take the measurement with ease. I had to solder a measurement cable to the back of the board, otherwise it’s pretty impossible to take measurements from both sides of the PCB.

                      Being the lazy bastard that I am, yeah… well. But I did it now, and I have to stand corrected: The traces are indeed connecting the pins!

                      Mind that I didn’t check them all, only DIMM #39 to CPU D36 and DIMM #40 to CPU #F36. At least for those I can confirm it’s there, so I don’t doubt the others being connected anymore either.

                      I guess that means the problem should rather be on the BIOS level instead.

  35. http://www.supermicro.com/products/motherboard/QPI/5500/X8DTL-3F.cfm , notice this server board takes 24gb un-buffered unregistered dram, yet it also takes up to 96gb registered ECC ram out of six slots, sounds familiar to our boards doesn’t it???

    • Intels 5500 Tylersburg. The “server” version of our X58. EVGAs Classified SR-2 (the board I took the Xeon µcodes from) is also based on it. But that’s Supermicro for you, they won’t mess up ECC support, or memory support in general. ;) Besides dual socket capability I don’t know if anything else really differs between the two Tylersburg chipsets, I’m too lazy to study the datasheets. :roll:

      I’m still not counting on my P6T Deluxe to have the necessary ECC traces on the board though. For it to work, a total of 8 DIMM pins need to be connected to the memory controllers, that’d be pins 9-10 and 45-46 on the front side of the DIMM and 128-129 as well as 164-165 on the back side, as seen [here] or [here] for instance. I tried to visually confirm that, but failed to do so, it’s a multilayered PCB after all. We’ll see…

      Edit: Xeon X5690 ordered! I presume it should arrive next week, which is when I’ll conduct ECC & µcode tests!

      • With all the “data” we have gathered together in your blog from the articles linked and info shared I doubt you’ll need to dig up anything from datasheets. :D Question, since your 980x is beginning to fail, as you’ve previously indicated, did you at least, to be safe, integrate the micro codes off of the SR-2 (how did you gain access to such a board, <those are legendary, was it from your work???) into your board. You don't have to wait until your X5690 arrives to integrate the micro code as you can, and should do that now, before your X5690 even arrives. I mean the 980x still works, right? That way, once your X5690 is integrated into your board the board will readily recognize all available features of the CPU, this should make testing easier. Also, I'd recommend holding off on the overclocking until you've successfully confirmed ECC functionality, oh, and not to be negative here but DON'T expect serious overclocking of the memory, you'll want to keep it operating as close to stock as possible (since it's ECC) and instead focus on multiplier settings as well as bclk…what I did was to lower my DRAM SPEED from 1600 default to 1333, then I kept increasing the bclk until I got the cores where I wanted them, to 160, from 133, and so far I haven't touched the multiplier, leaving it at 26…thankfully the X5690 is designed to handle way more heat than the 980x so not much to worry about there, hope everything goes well!!!

        • Morning!

          You don’t actually need the SR-2. All you need is to know which board supports the Xeons and has a comparable AMI BIOS. You can just download the BIOS, then extract all the individual µcodes from it using AMIs MMTOOL and patch them into the target BIOS. Problem solved! Here, I’ve assembled everything necessary for you, including a readily patched Rampage III Gene BIOS:

          Also – for research purposes – I’m planning to use the Xeon without the BIOS patch first, then run a few tests. After that, I’ll patch it to see if anything changes. Just curious! :)

          On top of that, I was never planning to OC my RAM. Even now, I’m keeping my RAM (which is DDR-III/1600) at DDR-III/1456 with relaxed latencies, just to make sure that large unbuffered bunch stays stable. The triple-channel setup is more than fast enough for my applications anyway, even at lower clock speeds and higher latencies. Plus, if I were to OC the RAM, that’d require more extensive stability testing, which I’m just too lazy for. ;)

          • Thanks! Now all I’ve got to do is to extract the microcodes off of the Xeon codes you uploaded and integrate them into my own BIOS which is 1003_10 as its already modified with a newer Marvell 9128 controller firmware I found in https://www.bios-mods.com/forum/Thread-Asus-Rampage-III-Gene-OROM-Mod?page=2 , someone on that particular thread at the end of it in the second page asks for an even newer firmware for the 9128, which I’ve failed to find, would it be too much trouble if you can find it and integrate it into the 1003_10 mod ZioGTS provides on the bios-mods site thread??? If you can upload THAT with the proper codes you already have I’d be incredibly grateful to you ;D.

          • BTW I tried downloading the Xeon micro codes zip file however it’s giving a 404, can you please re-upload the file so that I may get the codes from you, thanks!!!

            • My bad! It seems there was an encoding issue due to the “µ” character when uploading the file via the browser. Renaming the file on the local storage of the server seems to have fixed the issue (there was a bad character in the file name that shouldn’ve been there).

              Please retry the download and let me know if it works correctly for you now!

              Oh, and here is your BIOS:

              Seems I was wrong though. The original one already had the latest µcode 0036C214hex installed for CPUID 06C2hex, which would be the X5690. It appears that what was missing was rather some 45nm Xeon support?! Strange. In any case, this ROM is more complete now. :)

              • Perfect! Thank You, I’m sure anyone who runs into your site searching for X58 based info will be very grateful to have this great wealth of files readily available! You know, I have a bunch of desktops and I’ve been thinking of ways to make use of them, here’s a brief synopsis of the various hardware I have at my disposal, from oldest to newest…note all the boards are microATX: 1. ASUS A8N-VM CSM w/Opteron 180@2.76 Ghz+4GB DDR ECC, 2. Biostar P4M900-M7 FE 7.0 w/Xeon L5408@2.33 Ghz+4GB DDR2 non-ECC, 3. ASUS P5G41M w/Xeon X5470@3.45 Ghz+8GB DDR3 non-ECC, 4. ASUS Rampage III Gene w/Xeon X5690@4.17 Ghz+48GB DDR3 ECC, 5. ASUS M5A88-M w/FX-8300@4.2 Ghz+32GB DDR3 1866 non-ECC…notice that of my 5 desktops two use ECC memory. The Xeon I plan on using for heavy scientific research, simulations, development etc…which leaves me with the old AMD Opteron “server”. Do you think I can use that to host a website 24/7, and how many users do you estimate a slightly overclocked Opteron like that can handle??? Would it be enough to host a web-site like yours??? I know it’s obsolete however the CPU still scores a 1,700 passmark score keeping it within high-mid range status, so it still has some performance to offer even today…

                • Oh my!

                  While I made the bad decision of going with WordPress for my weblog software (it’s very CPU-intensive), with the WP-SuperCache plugin it works relatively well. You wouldn’t believe on WHAT kind of hardware.

                  Let’s be dramatic for a moment: This is XIN.at, a 1995 quad-socket Pentium PRO 1MB 200MHz machine with 2GB of FPM-DRAM and SCSI storage by Adaptec and IBM. The IBM PC Server 704, more info [here].

                  You may have noticed that posting comments here is rather slow; That’s because the processors are based on the first P6/686 cores in their final and greatest version from 1997. As said, 1MB L2 cache and 200MHz. The 2GB ECC+P memory – despite being 4-way bank interleaved – can barely breach 200MB/second.

                  In other words: Your hardware is just fine! When paired with well-configured software you should be able to handle at least 50 users at the same time, no problem! Likely you’d be able to handle far more.

                  Just make sure to run RAID-1 and hook it up to a UPS unit to avoid unnecessary downtimes, and you’re good to go with that Opteron! :)

                  • WOOOOOOOWWWWW!!!! Sorry I’m just too impressed by the age of your hardware and what it’s able to do!!!! I can’t even find ANY SAMPLE BASELINES FOR YOUR CPU, NOT ONE!!! Obviously you took efficiency of code into account, since your site is extremely clean (even with the silly tables heehee lol)! So what this means is you’re leveraging your server’s capability to it’s fullest extent by having such an efficient layout, this bodes extremely well for the prospects I have for my AMD Opteron then. One of the reasons this Opteron build impresses me so much is because it uses the very first ASUS board to have SATA II 3.0GBPS ports, plus I have an SSD so I don’t think that RAID will be necessary as a result…however with the amount of users potentially accessing I can see how RAID would be important in that case…so more than likely down the road.

                    • Even SSDs may die. Just saying. Or maybe, keep regular backup images of that SSD, should also be ok.

                      My HTML code (tables, *cough*) is about as ancient as the server itself. You should definitely not flatter me when it comes to HTML! I simply don’t know how to do better. ;) Modern HTML I just don’t really know too well. A little bit of CSS3 yes, but HTML5? Modern layers? Just not so much my thing. ;) And learning all of it? Well, by now you probably know that I can be very lazy. ;) Tables was how we did it in the 90s anyhow. :roll:

                      In any case, the thing most limiting will be the CPU in your case. If you have a SSD (or two), no worries there. SSDs are perfect for webservers! 4GB RAM are also more than enough. I’m running a ton of services on 2GB here, and my server still has 500MB free (counting both user and kernel space). Even with a more modern OS you should be good. Maybe you’d like to go for Linux or BSD UNIX anyway, even less RAM consumption then.

                      To sum this up, I count about a 1000 unique hits a day here, when it’s a busy day. Otherwise, numbers are in the 500-600 range. Even though I’m saying that the limiting factor is the CPU for you, that doesn’t mean you’ll ever actually hit that limit!

                      Never underestimate the impact crappy software can have on performance. Choose efficient, fast and secure software, and my personal prognosis would be that your Opteron could survive 20 years, just like my Pentium PRO server did..

  36. I CONFIRM CRUCIAL PART # CT2K8G3ERVLD8160B WORKS BEAUTIFULLY!!! 48GB ECC REGISTERED!!!!! HOWEVER, this is using Bios 1003 (final Bios released for ASUS Rampage III Gene) on an X5690 which unfortunately Windows identifies as an X 000 CPU. It’s very odd because the BIOS correctly identifies the CPU as an X5690, is there anyone else with this problem that was able to resolve it??? I know a microcode update to the BIOS would force the correct identification, however I cannot find any info on the microcode for the X5690 ANYWHERE :( I wonder if the BIOS on my board can identify the X5690 completely, then perhaps my board can go all the way to 96GB!!!! BTW, thrawn, here’s the part number for the 16gb modules which seem to be identical to the modules I’m currently using! CT16G3ERVLD4160B. If anyone here has their Xeon properly identified (unlike me :( , and is brave enough to try this out, buy two of these 16gb modules CT16G3ERVLD4160B and let us know if it works!

    • Good Morning Mr. WOOOT! ;)

      Nicely done! I’ve also been thinking about the 96GB idea already, but my problem is, that I’m relying too heavily on the high clock rate of my i7 980X (OC to 4004MHz), so going for a Xeon would mean giving up overclockability, unless I can find an engineering/qualification sample without the multiplier lock.

      I don’t quite see why 96GB wouldn’t work though, given the 8 ranks per channel limit as well as the 40-bit address bus. CT16G3ERVLD4160B is dual-rank even, so it’s well within spec. Also I’m not sure whether missing µcodes would really mess with memory detection. I mean I’m not saying “it definitely won’t”, but I’d assume it’d be ok.

      I haven’t done any µcode patching since the days of the glorious i440BX Seattle chipset and the Intel Pentium III-S Tualatin 0.13µm processors, but if you cannot find the raw µcodes for the X5690 on the web, you can try what we did back in the old days. Just find another motherboard that has a BIOS similar to yours (like made by AMI) which supports the Xeon X5690, and then extract the µcodes from that.

      Now I don’t know anything about modern BIOS tools anymore, but back in 2001 or so we would work with Award BIOSes and Awards cbrom tool. If there is something similar today, maybe you can extract the µcodes from a proper donor BIOS and patch them into the latest BIOS for your Rampage III Gene to reach full support for that processor.

      • Not necessarily, as my X5690 is 24/7 rock-solid at 4.10 Ghz as of right now and there’s a baseline on passmark under high-end showing an overclock achieved of 4.60 Ghz which has to be stable in order to complete the passmark test so WOOOOT!!! You don’t give up overclockability, and you gain increased memory capacity not to mention the trusted execution and cryptographic feature improvements as well, also the X5690 has a higher max temp and they’re soooo cheap now i’d say go for it!! I paid 800 bucks for mine two years ago (gotta love ebay ;) and I’ve seen them as low as 300 bucks!!!!! …I’m going to try the cbrom tool idea as I already located it (which works even in Windows Datacenter 2012 edition btw)….soooo keeping my fingers crossed, I’ll let you know if this goes well. Also, if you decide to go for the X5690, which I HIGHLY recommend, let me know if you can get more than 48 GB ram working, here’s another 16GB ddr3 ECC module I found that “may” work…it’s a Hynix part # HMT42GR7AFR4A-PB. ECC DDR3 16GB modules are fairly easy to find compared to non-ECC, so at this point it makes more sense to have an X5690 and overclock the pants out of it while still having the benefits of ECC like I do, you could say I have the best of both worlds, stable overclocked server with room for higher levels of performance WOOOOOOOOT!!!!

      • I’m actually thinking the X5690 paired with the Rampage III Gene somehow unlocks the X5690 so it doesn’t have the locked multiplier limitation, as confirmed by people having achieved up to 5.6 Ghz overclocks on this chip, making it incredibly powerful. I must add however that these people have used other X58 boards to achieve these clockspeeds, none were achieved using my particular board…though I have seen that on my bios there isn’t a cap on the multiplier, perhaps this is because it’s not fully identified??? Maybe fully identifying my X5690 would lock down the multiplier???? Would it perhaps be best to leave it in it’s half-identified state?????????? :/

        • Hey man,

          Typically, multipliers are locked in hardware, making it impossible to bypass the lock via BIOS or any software method since the Deschutes-Core Pentium II processor. That being said, I believe it is more likely that the Xeon X5690 is simply an unlocked CPU. While this doesn’t actually make much sense, there is no specification saying otherwise either. Also, even if what you’re setting here is simply the maximum EIST/turbo multiplier, you should be limited to 3.73GHz. So there is something more going on here, at least if you kept BCLK to 133MHz.

          Well, as of now, all I can do is shrug my shoulders. In any case, if you DO go ahead and patch those µcodes, and if it DOES limit your processors capability to use higher multpliers, you can still just flash the original v1003 BIOS back in.

          I have to admit, you’ve definitely put a bug in my ear right there though. 96GB of Reg. ECC are unbelieveably expensive, but right now, I’m going for a new case and completely new 12-disk SAS RAID-6 in that machine anyway, everything being super-expensive already anyway. IF I could run a X5690 at ~4GHz, that’d be awesome. 96GB would allow me to boost my NTFS file system caches up to a level where they could hold an entire 3D Blu-Rays splitted data for processing, greatly speeding up the pre-encoding process.

          Oh, and about that Trusted Execution stuff: While I was thinking about getting the Infineon-chip based TPM upgrade module for my P6T Deluxe, putting a TXT-capable CPU in it is potentially dangerous. Security benefit? Well, it can also mean something like “We just identified you master boot record housing Truecrypt, so we do not trust you, so we won’t let our software run on your machine”, if you catch my drift. Trusted Execution Technology + TPM may mean giving up a lot of freedom… Well, I can just leave the TPM out, and TXT won’t do much anyway. ;)

          Aaah, why you put bugs in my ears like that, eh? :twisted:

          • Oh, to really whet your appetite I’ll give you some more tidbits to digest, the crucial ECC ram I initially recommended, the 8 gb modules I currently am using, if you look closely at the details of this particular module, it’s low-profile, great for using with massive cpu coolers, operates at 1.35 volts (within ddr4 spec!), and best off all has a cl of only 9, which I’m running at even with my 4.2 Ghz clockspeed!!! So not only do I blow DDR4 away in latencies, however I get close to the voltages used in those modules. The 16 GB modules appear to come with a black heatsink, while retaining all of the characteristics of the 8 gb module, so I imagine they can propbably operate even better, assuming they work of course! Oh and, yes, my bclck is WAAAAAY higher than a measly 133 mhz, pfffftttt…oh and my vcore is only 1.3 volts, my cores never reach 80 c even at 4.28 Ghz….heeeheeeheeeee :D

            • Hey WOOOT!,

              As a matter of fact, I do believe I have a failing CPU at hand here. I’m frequently losing one of my memory channels, maybe a result of my Uncore or DRAM voltages being too high. So I’m thinking I’m having a failing memory controller in my 980X. Which is why this whole thing might be ever-so-more interesting!!

              I have to advise you on one thing though. I know you won’t like it, as I’ll come across as some kind of wise-ass, but please consider that memory latencies are highly deceiving, because they are measured all wrong. They measure them in clock cycles instead of time units (nanoseconds). As a result, they are skewed. CL10 at 2000MHz clock rate is MUCH quicker than CL10 at 1333MHz, latency-wise.

              To calculate effective latency, please apply the following formula (SI units only, physical clock rate in Hertz, time in seconds):

              Latency in clockcycles / clock rate in Hertz = Latency in Seconds

              Say you have DDR-III/1600, then the physical clock rate is 800MHz, or 800000000 Hertz. Say, the latency is CL9. So the effective latency is 9 / 800000000 = 0,00000001125 seconds. If you want that in a more readable nanoseconds, multiply by 1000000000. So that’d be 11.25ns.

              If you compare the latencies like that, you’ll see that actually effective latency hasn’t changed much in the past 10 years at all, despite CL-values getting higher and higher… well of course they are, clockspeeds are too. :) You can get a brief overview [here]German flag.

              Just wanted to say that, because memory latencies are so frequently misunderstood. :)

              • http://www.extremetech.com/computing/185797-forget-moores-law-hot-and-slow-dram-is-a-major-roadblock-to-exascale-and-beyond , and http://www.tomshardware.com/faq/id-2291218/ddr4-ddr3-breakdown.html both of these articles make the case, or at least mention, how latencies can be a problem for video editing, power usage, or, from what I’ve seen in other articles, database querying, wherein the higher latencies of DDR4 actually cause delays in queuing up tasks….so sorry to say, DDR4 isn’t the massive improvement it’s supposed to be, and between the voltages I’ve got, and my latencies, I won’t be moving to another memory standard for quite a while :)

                • So many replies! ;)

                  I’m not saying DDR-IV is faster at all. I’m just saying, that overall, latencies didn’t change much over the past years and those ever-increasing numbers only serve to deceive people.

                  Let’s compare the CAS of the fastest DRAMs we can currently buy, that’d be DDR-III/3100 CL12 and DDR-IV/3333 CL16, cancelled down to nanoseconds:

                  DDR-III/3100 CL12: 12 ÷ 1,550 = ~7,74ns

                  DDR-IV/3333 CL16: 16 ÷ 1,666 = ~9,6ns

                  So yes, the fastest DDR-III is quicker to respond and likely faster to complete a burst given it’s relatively equal bandwidth. But it ain’t worlds apart either. I’m guessing that as DDR-IV clockrates continue to rise, its effective latency will reach about the same we’ve seen before with DDR-I/II/III.

                  • Wasn’t this the same with every generation? At first the newer memory isn’t much faster at all.

                    • Pretty much, yes.

                      But… I do have the feeling that people just think newer memory sucks per se, because the latency values are “so high”.

                      Maybe my observation is skewed, but I got the impression that some people were comparing CL10 DDR-III SDRAM to ancient CL2 PC133 SDR-SDRAM, saying that the old RAM is soooo much quicker, because CL2 vs. CL10 etc..

                      In reality though, The CAS on PC133 CL2 SDRAM would take 15.04ns, whereas it only takes 13.75ns on mid-range DDR-III/1600 CL11. Notice the minor difference too.

                      I always had the feeling that many people were misguided by the relative units used here. SI units it shall be, all the way. ;)

              • Looks like it’s time to move up to the X5690, AWESOME :D, trust me when I say it’ll be worth it, and, if you manage to get 96GB running, I will be seriously intrigued! It’s perfect really because even with 96GB, you still aren’t at even half of what the IMC of the X5690 can handle, which is an absolutely massive 288GB!!!! Here’s a link, http://www.ebay.com/itm/Intel-Xeon-X5690-SLBVX-3-46GHZ-12MB-6-4GT-s-LGA-1366-Hex-6-Core-CPU-/371259327992?pt=LH_DefaultDomain_0&hash=item5670c7c5f8 , after a quick search on ebay I found a seller offering the X5690 at 214.99!!!!!!! This is too good to pass up let me know once you’ve got it I’ll send you all my timings and settings used (it’s pages worth heehee)!!!

                • Thanks for the link! I’ll have to go for a German/Austrian auction though, as this one costs me $15 in shipping and $53 in import charges. ;) But I’ve seen local people selling it for about 300€, which also sounds like a nice deal.

                • Hmm, I have serious doubts if it will work with higher densities.
                  288 GB is not a ‘natural’ computer number. With 2 QPI links, it might have to do with those three dimms per channel you see on some motherboards. 48 GB x 3 channels x 2 CPUs = 288 GB.
                  I really don’t know if both CPUs have to address the memory of their neighbor, but it just might.

                  With 2 dimms per channel you then would still be able to reach 96 GB.

                  But what kind of memory is this, registered?

                  • With such capacities, it’d have to be Reg. DDR-III naturally. There is no way of reaching 288GB with just unbuffered modules.

                    In any case, those Xeons have a 40-Bit address bus, which means that their practical limit is 1TiB of RAM. I think that it has been proven enough that the limits of the address bus CAN be reached, like with my 48GB stunt on a 36-Bit processor, meaning 64GB max. on the address bus with an “official” specification of 24GB max.

                    IMHO, the 288GB simply refer to the triple-channel configuration, just as you suspect. This may be just because of the ranks limit. 6 ranks per channel as by spec, right? So with 2-rank Reg. 16GB modules, you may match 288GB on 18 slots. Quite realistic!

                    I have already confirmed that Reg. DDR-III works on at least some X58 boards, including the P6T Deluxe and the P6T Deluxe V2. WOOOT! has made similar experiences. It is thus quite probable, that 96GB Reg. ECC might just work when using a Xeon that can surpass the 36-Bit address bus / 64GB max. memory limit that my i7 980X imposes… But before I pump serious money into this experiment, I have to confirm proper ECC functionality first. Otherwise, it’d not be worth the cash.

              • Thrawn, I had to dial down my overclock settings a little bit however I’ve managed to achieve a so far (based on 8 hours of prime95 64-bit, type-0 FFT torture test) 4.17 Ghz! This while keeping my ECC memory at it’s default 1600 Mhz and a Cl of 9…I won’t be perfectly satisfied with this clockspeed’s stability until it passes the 24 hour mark, though I must say it seems perfectly stable as I’m typing this reply to you and browsing various sites while the FFT torture test is taking place! Vcore is steady at 1.312 volts, very low considering I’ve seen these go as high 1.6-1.84!!! Overall I’m so far extremely happy as I achieved a whopping 704 Mhz increase in clockspeeds while preserving the X5690’s 130 watt TDP, simply amazing!

                • For me it’s fine, as long as I can reach 4GHz. Whether its 4, 4.1 or 4.2 doesn’t matter so much to me, but cosmetically I just can’t stand something like 3.9GHz or so. ;)

      • I did it! There wasn’t any need to update the microcode of my X5690 as there was an option in my Bios, A20M, which needed to be enabled in order for the os to properly identify the CPU, I still think it’s odd since the option references legacy os’s, which Windows 2012 Datacenter most definetely is not, though in the end it works so finally. This means that with the right tweaks the Rampage III Gene is an excellent choice for this cpu, provided you have BIOS version 1003 installed of course. I uploaded my baseline last night and achieved an 11,971 passmark score with only a 4.28 Ghz overclock, though it wasn’t completely stable and crashed shortly after having completed the benchmark suite, however this proves that the X5690 is DEFINITELY worthy of having, given that I did this while still having all 48GB of ECC memory fully operational and I still managed a full five-star score in passmark, WOW, just WOW!!! Furthermore I was able to boot into Windows at a whopping 4.5 Ghz, all of this being done using air-cooling mind you. Also the multiplier does seem to be locked however it goes all the way up to 27 so from there you just boost the bclock however at that point you’d just need the right cooling and it can be taken much higher from where I’ve got it, but even so, unless you just want to brag it clearly wouldn’t be necessary to go any higher as I already beat everything AMD has as of right now, including their FX9590. My point being ultimately that if you had any doubts about upgrading your 980x to the X5690, they’re effectively gone! Heck if you want, after you find one and get it running on your system I’ll give you all of my BIOS settings I used so you can achieve similar, or better results. I’m currently running prime95 with my machine settled on a so far 100% stable 4.2 GHZ! I should still get around an 11,600 passmark score which I’m happy with and once Cooler-Master releases the Kinetic cooler I’ll see if that can further reduce my temps for an even higher safe overclock! I’ll likely end up at around 4.4-5 Ghz for safe 24/7 operation once I’ve installed the new cooler…AWESOME!!!!

        • What the hell, lol?

          The address line A20 is the 21st line of the address bus of any x86 computer. The first 20 are used to address memory from 0 – 1MByte. The 21st allows the machine to address the HMA (high memory area) from 1MB-2MB, and is a “gate”, meaning it can be en- and disabled at will. This is extremely ancient stuff. On modern machines, A20 should always be enabled, as it’s a part of the 32, 36 or 40 address lines we currently have out of 64, to address all the memory. Typically, modern operating systems enable it during the boot process if it’s not already opened by the BIOS.

          How the A20 gate has anything to do with CPU detection is absolutely beyond me. 8-O

        • Ah, by the way…

          Why not the Xeon W3690? It seems to be equal to the X5690, only lacking one QPI link. But since I’m running a uniprocessor machine, I wouldn’t need the second QPI, and the CPU is even cheaper than the X5690…

          Edit: I see the W3690 doesn’t officially support more than 24GB RAM, although the specs DO list a 40-bit address bus, unlike W3680, which has a 36-bit one. Weird…

          • I have one big concern about the W3690, the fact that it’s temp is lower than the x5690, which means less overclocking…

            • Ah, you’re right, I didn’t spot that! Lower T(Case). I checked again and noticed that the VID / VCore range for the X5690 also starts a notch lower than for the W3690, 0.750V – 1.350V vs. 0.800V – 1.375V. So it seems there are several indicators for the X5690 trending to be of higher chip quality than W3690.

              X5690 it is then!

              • You’re basically ending up with the most powerful LGA 1366 cpu ever made, period ;D. There’s a reason Intel has kept the price so high for so long on these cpu’s, unfairly so but then businesses continue buying these so that probably would explain why. Make sure to enable A20! You mentioned memory addressing with that option, meaning that could have been the reason why when I tested the 16 gb modules only one worked, as soon as I added a second module the board failed and I ended up returning both modules to Crucial (they weren’t too happy). So if you have success with either part CT16G3ERVLD4160B or HMT42GR7AFR4A-PB (the Hynix modules appear to be cheaper however compatibility is completely unknown, the Crucial module did work, one, it was when I added the second one the board failed however I did have A20 disabled) let me know as this will give me the confidence to replace all of my current modules for the 16 gb variety.

                • Hey,

                  As a habit, I tend to have the A20 gate open on all machines, this one included. I never tried to switch it back off. In any case, you’d need to be patient. Migrating to my new RAID-6 array currently has the highest priority, with a stable machine of course. Since that’s extremely expensive, any memory upgrade would only happen further down the road. So this may take months, especially since the 16GB stones aren’t exactly cheap in larger numbers. I’ll keep you posted here, but like I said, it may take a long time before I can try this.

                  If you have email notifications active here, you’ll notice though. :)

                  Long before that though, I’ll borrow some Reg. ECC modules from work, and check out the P6T Deluxes capability to enable ECC as soon as the Xeon is here. I searched the web like crazy, but nobody ever really tried to verify whether my mainboard can actually use ECC tech…

                  Two possible problems would be that a.) the BIOS might block ECC or b.) that the physical address lines are simply missing on the mainboard. We’ll see…

                  • Well my Rampage III Gene has the same issue as your P6T Deluxe, there is no information regarding ECC However it just works! I understand regarding the timetable and don’t mind at all waiting as the wait will be well worth it!!! If you come across any 16gb ecc modules at your work try and test those if you can, or just to establish the board can work with ecc then 4gb modules will be fine since that’s how I started out before moving up to 8 gb modules, don’t mix up different modules since this is a server cpu and they’re extremely sensitive to working with mixed memory modules so try to test with matching pairs please, though I’m sure that you already know this so I just reiterate the importance though since these aren’t server class boards we’re using!

                    • God damn threaded display, soon we’ll need a new thread, cause this shit is getting too narrow again. ;)

                      In any case, the old version 4.20 of Memtest86 is supposed to be able to really determine whether ECC is active. I know ECC modules work, but to find out if it’s really active I need that Xeon and some memory from my workplace. We have no huge 16GB sticks there, only some 2GB ones, but for ECC testing those’ll be ok!

                      Naturally, I will never mix pairs – or rather triplets in this case – ever! IF I am going to do this in the future, I’m gonna buy a triple kit first, then test, and buy a second matching one if it works! Cleanly by the book!

        • And again. ;)

          I found out how to handle CPU microcodes for AMI BIOSes. It’s surprisingly simple! There is an AMI tool called MMTool, which can extract and insert µcodes into the BIOS. It’s a super-easy GUI tool actually. The X5690 and i7 990X processors share the same CPUID, 06C2hex, but it appears the µcode IDs are different (0036C214hex for the Xeon vs. 0036C213hex for the i7 990X). There are other tools too with which one could alter the BIOS version strings (“AMIBCP”), so in essence, I could update the latest BIOS for my P6T Deluxe – version 2209 – with X5690 µcode and re-release it as 2210! :)

          It seems the EVGA Classified SR-2 is a nice donor for newer Xeon µcodes by the way. :)

          • Well, check this out, I did a little search for the 16 gb part from Hynix and turns out they have a datasheet about their ECC modules and at the bottom of their list is a 32 GB module part # HMT84GR7AMR4A-G7/H9!!! It’s 1.35 v and quad-ranked, and it’s 349 dollars on ebay. The only aspect which sucks about this modules specs is its 1333 with a CL of 11 so you sacrifice bandwidth for capacity. So I’d go for the 16gb modules and stick with either of the two modules listed above, particularly the Crucial’s if you can…when you receive your X5690 let me know, start off with a multi of 26, voltage of 1.3 volts, then set your bclk to 160. From there turn off speedstep, TM, Turboboost, C1E, C-state, leave all other cpu configuration settings enabled, especially A20 lol! Set UCLK frequency to 3368, after setting DRAM frequency to 1600, it’ll actually give you 1604 which will require a bump up in dram voltage, so in my case it’s 1.36 instead of 1.35. Set your CL timing values to 9-9-9-24 and very important, check if you a setting in your BIOS called “memory configuration protect” and if so enable it as well as “memory recheck” as well. Also look for CPU Differential Amplitude setting and set it to 800mV, from there under Spread Spectrum and clock skew, disable spread spectrum completely and set clock skew delay to 700ps on both IOH and CPU. Finally set your QPI Link Data Rate to 7699 MT/s and you’ll have an excellent 4.17 Ghz in Windows plus these settings will provide you a great starting point to then continue increasing clockspeeds from there! ;) Let me know how it goes…

            • 32GB are out of the question for me, as my workstation OS (NT 5.2, or in other words XP x64, kept up-2-date with Server 2003 updates) doesn’t support more than 128GB of RAM – neither does any version of Windows 7. And Win8, sorry, but no way. ;) I’d rather go with my good old friends CentOS 6 Linux or FreeBSD 10 UNIX instead – which is planned in the very long run anyway.

              In any case, 128GB is the upper limit for now, so 32GB modules are out of the question, as I can’t really use 192GB of memory. My OS can’t, and even I can’t, practically speaking. 96GB I can still load with my YaCy distributed search engine node, 7z and multiple x264 video encoders. But 192GB?! Holy hell. That’s too much! :)

              Nah, I’ll aim for 16GB modules. But as I said. it’ll take time.

              About the overclocking: Thanks for the specs, especially the clock skew delay (which I have never touched). Currently I’m running BCLK 182MHz × 21 = 4004MHz Core, × 17 = 3.1GHz Uncore and × 18 = 3.28GHz QPI, with DRAM at DDR-III/1456. The P6T Deluxe is VERY picky about clock rules (very early X58 generation), so I may not match any of the following criteria, unless I wish to face massive instabilities, which can be quite limiting:

              • Uncore >= QPI (unstable when equal, no POST when Uncore > QPI)
              • (DRAM × 2) > Uncore (no POST)
              • QPI > 8GT/s (unstable)
              • You’re still thinking within the limitations imposed not so much by your board however the cpu as well! Keep in mind that the X5690 installed on your board should remove many of the instability limitations you are currently experiencing!!! Also, the clock skew delay setting should stabilize your other settings permitting improved stability, at least that’s what I found way back when I had my D0 920 running which that I managed to get up to 4.5 Ghz before selling it for 250 bucks a couple of years ago…got a nice ROI out of it…perhaps the improved stability from ECC memory combined with the X5690 is just what your board needs, since you mentioned your X58 based board being of an earlier generation I went onto Bios-mods.com and found this particular thread which may be of interest to you, https://www.bios-mods.com/forum/Thread-Asus-P6T-Deluxe-2209-With-Intel-Raid-ROM-v10 , perhaps adding additional capabilities to your BIOS will help in a holistic sense, let me know, I wish you success in this, the payoff will undoubtedly be worthwhile!

  37. Please, I have a ASUS P6X58D-E , you could tell me if ram of 8GB 10-10-10-27 working? Thank You!!

    • Hello Vanner,

      I can not guarantee you anything, as I do not possess your mainboard. What I can do is to make an educated guess here: Both my board and yours are ASUS, and your later boards’ BIOS is likely just a derivative of mine – the first X58 by ASUS – given the obvious similarities.

      So my guess would be: It’s very, very likely to work all the way up to 48GB! I just can’t say for 100% sure, because I can’t test it myself. If you want to be careful, I would suggest ordering the memory at a shop that accepts convenient returns, just in case.

  38. Thank you for great article. I use 24G(4×6) memory on ASUS X58 E WS(Workstation) and X5650 cpu(it can address 288GB memory). I have a question… Could I use 6 x 16G/non-ECC stick (Total 96GB) ?

    I’ve searched all internet and even Asus homepage didn’t mention the 16G sticks + X58 mother board.

    Thank you for reading and if it possible, please let me know it :D

    Have a Good Week.

    • Good morning Junseok,

      According to the [Intel® Xeon® Processor 5600 Series Datasheet, Volume 2] your processors IMC supports up to 48GB of unbuffered DDR-III in a 6 x 2R configuration, which means using dual rank memory is probably a must here.

      That doesn’t mean that it can’t work in theory. As long as you would stick with 2-rank memory, you could maybe get lucky despite the issue that 96GB of unbuffered memory might get quite shaky, and on your platform you can’t run registered memory. Even if the registered DRAM itself works (I have some Reg. ECC here in an ASUS P6T Deluxe V2 at work), the register buffers won’t, and neither will the ECC.

      That however isn’t the main problem at all. :(

      The bad part is, that to my knowledge there aren’t even any unbuffered 16GB DDR-III sticks to try this with. Just to make sure I have browsed a few memory stores and it seems those modules still don’t even exist.

      So I fear that your question is of a rather academic nature at this time. :(

      The best answer I could give right now would be “It could work in theory, but it needs to be tested in the future, as specs are not 100% clear and the memory modules in question aren’t available as of now”.

      Sorry I couldn’t be of more help here. :(

  39. U have the processor with this specs ?
    Intel® Core™ i7-980X Processor Extreme Edition
    (12M Cache, 3.33 GHz, 6.40 GT/s Intel® QPI)
    Max Memory Size (dependent on memory type) 24 GB
    Memory Types DDR3-1066
    and u use 6x8GB Crucial DDR-III/1600 CL9
    that runs @ 728×2 MHz?

    I have PROC – Intel® Xeon® Processor X5675 (12M Cache, 3.06 GHz, 6.40 GT/s Intel® QPI)
    RAM – 8GB (2x4GB) cmv8gx3m2a1333c9
    MB – P6X58E_WS
    and I whant to add 8GHz x 6 Slots
    any recomandations ? pls :)

    • Hello Mihai,

      Yes, my processor is a i7 980X, which is specified to 24GB maximum DRAM. And yes, I am successfully using 48GB DDR-III/1600 @ DDR-III/1456, or in other words 728MHz. As I am typing this, the machine is still working fine running 24/7 straight and it’s over 3 weeks now!

      The last week I had it running at full load transcoding my Blu-Ray movies and multiplexing them to MKV files. Also, I’ve been playing games (Torchlight II, FarCry 3, Mass Effect 3) without trouble.

      Now, first things first: I won’t say that it WILL work on your mainboard, but you got a damn nice one right there! :)

      My recommendation would be to go for Kingston HyperX or use Crucial BallistiX, like I did. If you buy at Amazon you can always send it back, if it turns out not to work.

      But so far, it seems to work fine on pretty much all ASUS X58 boards!

      Here, this is the kit I am using, it is a 2 module kit, so I just bought three kits and used them together: [Link]!

      It’s german, as I am from Austria, I hope you don’t mind.

      If you have any further questions please ask away!

      • thx a loot , I am in Germany now so is ok ;)
        thx a loot for the tip with the Amazon , I did not knew that I can send it back ::D

        Vielen Dank
        Mihai L.

      • Do u think I can try 6x 16G ?

        • That’s debatable… For 16GB modules you’d need registered modules (there are no unbuffered 16GB DDR-IIIs), and I am not so sure whether your X58 would accept that. The CPU would, and the CPU would also be able to cope with the resulting 96GB of memory.

          My CPU has a 36-Bit address bus, which means the physical maximum is set in stone at 64GB. However, your Xeon with its 40-Bit address bus could theoretically support up to 1TB of RAM. But I assume it depends on the BIOS/chipset whether Reg. ECC 16GB modules would work.

          Regular 8GB modules I would consider almost a safe bet. But 6x16GB Reg. ECC, now that’s…. daring! To say the least..

          I have no idea. You’d need to try for yourself. I think nobody ever has before! Ever..

  40. This could have something to do with the rank of the memory, single rank, double rank or quad rank.

    Your processor is somewhat like a Xeon 5600 (or 5500 ?), so this might be applicable:
    http://www.supermicro.com/manuals/motherboard/5500/MNL-1062.pdf
    Look at page 2-7 (page 29 / chapter 2-4), memory installation. A lot of situations will downgrade memory performance. Perhaps this might give you some insight.

    If that processor only supports 3 dimms in double rank or 6 dimms in single rank, that might be the difference..

  41. Well done dude,
    i hope it’s stable, in terms of enterprise class stability (our typical requirement) and not only boot to desktop and play Farmville for one hour stability :-)
    Anyway now you forced me to upgrade my server’s memory also… DAMN!

    • HA! Just go ahead and make sure you got some virtual machines up and running to eat away that RAM! :)

      Meanwhile, I have tested the setup using my Blu-Ray demultiplexing (and transcoding) workflow as well as my backup system, which easily fills up my memory during its process. After confirming data integrity, it really seems as if this is working just fine… No anomalies or instabilities so far!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre lang="" line="" escaped="" cssfile="">

(required)

(required)