Page 1 of 2 12 LastLast
Results 1 to 16 of 25

Thread: Supermicro Dual CPU motherboard

  1. #1
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Supermicro Dual CPU motherboard

    Hi everyone!

    So I recently decided to upgrade my file server to enable me to offload a lot of virtual machine tasks to it. Instead of spending fortunes on new and modern server hardware or run the 'risk' of using regular consumer hardware, I decided to do a bit of ebay-ing.

    I ended up with a LGA1356 motherboard, 2 x E5-2428L CPUs (60w 6C/12T), 24GB ECC registered Samsung RAM all for under £300.... happy days.

    The problem I seem to be having is getting everything working. I was hoping (as always) for plug and play, but whenever I run dual CPUs I hit issues with memory errors. I have checked the RAM, tested with RAM out of my old server and even some none-ecc stuff and I only ever have issues running under dual CPU's.

    Has anyone else got experience with Supermicro motherboards, dual CPU motherboards or really any advise on things to look out for.... hell, anyone just feeling super helpful and interested in me getting my 12 core overkill beast running and want to join in on the journey?
    Last edited by Biscuit; 05-05-2016 at 09:33 PM.

  2. #2
    Anthropomorphic Personification shaithis's Avatar
    Join Date
    Apr 2004
    Location
    The Last Aerie
    Posts
    10,857
    Thanks
    645
    Thanked
    872 times in 736 posts
    • shaithis's system
      • Motherboard:
      • Asus P8Z77 WS
      • CPU:
      • i7 3770k @ 4.5GHz
      • Memory:
      • 32GB HyperX 1866
      • Storage:
      • Lots!
      • Graphics card(s):
      • Sapphire Fury X
      • PSU:
      • Corsair HX850
      • Case:
      • Corsair 600T (White)
      • Operating System:
      • Windows 10 x64
      • Monitor(s):
      • 2 x Dell 3007
      • Internet:
      • Zen 80Mb Fibre

    Re: Supermicro Dual CPU motherboard

    Mostly with those boards, you need to put identical sets in each CPUs memory bank.

    I have also found them to be VERY picky about mixing RAM, even when in separate banks.

    How many RAM slots are there, whats the configuration of them and what DIMMs do you have?
    Main PC: Asus Rampage IV Extreme / 3960X@4.5GHz / Antec H1200 Pro / 32GB DDR3-1866 Quad Channel / Sapphire Fury X / Areca 1680 / 850W EVGA SuperNOVA Gold 2 / Corsair 600T / 2x Dell 3007 / 4 x 250GB SSD + 2 x 80GB SSD / 4 x 1TB HDD (RAID 10) / Windows 10 Pro, Yosemite & Ubuntu
    HTPC: AsRock Z77 Pro 4 / 3770K@4.2GHz / 24GB / GTX 1080 / SST-LC20 / Antec TP-550 / Hisense 65k5510 4K TV / HTC Vive / 2 x 240GB SSD + 12TB HDD Space / Race Seat / Logitech G29 / Win 10 Pro
    HTPC2: Asus AM1I-A / 5150 / 4GB / Corsair Force 3 240GB / Silverstone SST-ML05B + ST30SF / Samsung UE60H6200 TV / Windows 10 Pro
    Spare/Loaner: Gigabyte EX58-UD5 / i950 / 12GB / HD7870 / Corsair 300R / Silverpower 700W modular
    NAS 1: HP N40L / 12GB ECC RAM / 2 x 3TB Arrays || NAS 2: Dell PowerEdge T110 II / 24GB ECC RAM / 2 x 3TB Hybrid arrays || Network:Buffalo WZR-1166DHP w/DD-WRT + HP ProCurve 1800-24G
    Laptop: Dell Precision 5510 Printer: HP CP1515n || Phone: Huawei P30 || Other: Samsung Galaxy Tab 4 Pro 10.1 CM14 / Playstation 4 + G29 + 2TB Hybrid drive

  3. Received thanks from:

    Biscuit (05-05-2016)

  4. #3
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    Quote Originally Posted by shaithis View Post
    Mostly with those boards, you need to put identical sets in each CPUs memory bank.

    I have also found them to be VERY picky about mixing RAM, even when in separate banks.

    How many RAM slots are there, whats the configuration of them and what DIMMs do you have?
    This is the motherboard:
    http://www.supermicro.com/products/m...0/X9DBL-iF.cfm

    6 RAM slots (3 per CPU) and all of the Dimms I have are the same 4GB Samsung ECC Registered stuff (M392B5170EM1-CH9).

    Unfortunately, they are not in pairs anymore as I have got them all jumbled up, is it important to keep the pairs matched? I figured as each CPU is tri channel, there is one channel per dimm anyway so it was irrelevant.

  5. #4
    DDY
    DDY is offline
    Senior Member DDY's Avatar
    Join Date
    Jun 2006
    Posts
    2,740
    Thanks
    147
    Thanked
    566 times in 392 posts
    • DDY's system
      • Motherboard:
      • ASRock Z390M Pro 4
      • CPU:
      • i5 9600k
      • Memory:
      • 16GB (2x8GB) 3200MHz CL16 Corsair Vengence
      • Storage:
      • Adata SX8200 NVME 1TB
      • Graphics card(s):
      • RX 5700
      • PSU:
      • Seasonic Focus Gold 550W
      • Operating System:
      • Win 7 Pro
      • Monitor(s):
      • Dell U2715H

    Re: Supermicro Dual CPU motherboard

    Have you tried forcing the RAM to run at lower frequencies, say at 1066MHz or even 800MHz?

  6. #5
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    Quote Originally Posted by DDY View Post
    Have you tried forcing the RAM to run at lower frequencies, say at 1066MHz or even 800MHz?
    Not yet, but I have discovered that the system is infact not even stable with a single CPU.

    Memtest runs ok, but I thought I would try and run more substancial tests on it and the only way I know how to do that is with windows. The windows installer just hangs and throws errors back at me. Going to try Ubuntu now just to see if I can get any kind of stability out of the thing.

    Earlier on in the life cycle of this build I had stuff running and was transferring all my files but I had a few crashes and I noticed the PCH was mega hot. I'm starting to think that perhaps this overheating has killed it and the motherboard is now a dud :'(

  7. #6
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    With one stick of RAM and one CPU in there I managed to get Ubuntu installed and running ok. Even ran some stress tests just to see what would happen although Im not entirely convinced they are doing the same thing as something like prime...

    I just started experimenting with the IMPI thing though and im wondering if this can be used to help me troubleshoot when something is wrong.

  8. #7
    DDY
    DDY is offline
    Senior Member DDY's Avatar
    Join Date
    Jun 2006
    Posts
    2,740
    Thanks
    147
    Thanked
    566 times in 392 posts
    • DDY's system
      • Motherboard:
      • ASRock Z390M Pro 4
      • CPU:
      • i5 9600k
      • Memory:
      • 16GB (2x8GB) 3200MHz CL16 Corsair Vengence
      • Storage:
      • Adata SX8200 NVME 1TB
      • Graphics card(s):
      • RX 5700
      • PSU:
      • Seasonic Focus Gold 550W
      • Operating System:
      • Win 7 Pro
      • Monitor(s):
      • Dell U2715H

    Re: Supermicro Dual CPU motherboard

    I'm not familiar with Supermicro's IPMI, but I'd imagine there would be a system event log somewhere, maybe that'll offer some clues.

  9. #8
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    Well I stuck a second stick of RAM in and once again ubuntu was OK but unRAID just collapsed



    Loads of memory error messages there :/

  10. #9
    DDY
    DDY is offline
    Senior Member DDY's Avatar
    Join Date
    Jun 2006
    Posts
    2,740
    Thanks
    147
    Thanked
    566 times in 392 posts
    • DDY's system
      • Motherboard:
      • ASRock Z390M Pro 4
      • CPU:
      • i5 9600k
      • Memory:
      • 16GB (2x8GB) 3200MHz CL16 Corsair Vengence
      • Storage:
      • Adata SX8200 NVME 1TB
      • Graphics card(s):
      • RX 5700
      • PSU:
      • Seasonic Focus Gold 550W
      • Operating System:
      • Win 7 Pro
      • Monitor(s):
      • Dell U2715H

    Re: Supermicro Dual CPU motherboard

    Does the error log show any memory errors at DIMM1A?

    What happens when;
    a) You swap the two sticks around?
    b) A third stick of RAM is installed?

  11. #10
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    I have took the second stick out and booted back into unRAID. The thing that seemed to trigger the critical error was running the Emby media server and scanning my library. It would always give up at about 62%.

    I have cleared that container from the docker and re-added it, ran through the library update and it was ok.

    I then added the second stick, got to 94% and boom... same problem again.

    I swapped the sticks around and now it ran through without issues

  12. #11
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    OK I deleted and re-added the folders to force a thorough rescann and I got the error again. This time IPMI just went straight to CATERR

  13. #12
    DDY
    DDY is offline
    Senior Member DDY's Avatar
    Join Date
    Jun 2006
    Posts
    2,740
    Thanks
    147
    Thanked
    566 times in 392 posts
    • DDY's system
      • Motherboard:
      • ASRock Z390M Pro 4
      • CPU:
      • i5 9600k
      • Memory:
      • 16GB (2x8GB) 3200MHz CL16 Corsair Vengence
      • Storage:
      • Adata SX8200 NVME 1TB
      • Graphics card(s):
      • RX 5700
      • PSU:
      • Seasonic Focus Gold 550W
      • Operating System:
      • Win 7 Pro
      • Monitor(s):
      • Dell U2715H

    Re: Supermicro Dual CPU motherboard

    It is at this stage where running out of ideas I often remember the first rule of troubleshooting;

    Try the easiest things first.

    So, errm. Have you updated the BIOS?

  14. Received thanks from:

    Biscuit (30-04-2016)

  15. #13
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    Quote Originally Posted by DDY View Post
    It is at this stage where running out of ideas I often remember the first rule of troubleshooting;

    Try the easiest things first.

    So, errm. Have you updated the BIOS?
    I actually have but I forgot to post it in the thread as it had no positive impact

    That was learning experience in itself. To do it you have to either create a bootable dos image and run an application through that OR.....

    Pay Supermicro for a product key to unlock updating via IMPI. I mean, seriously SMC?!?! Why would you make simple BIOS updating a pay option?!

    There are no other options so I went the DOS route and used freedos.

    I have swapped the RAM out for the two sticks of crucial non-registered stuff out of my microserver and its been running all night though.

    Another thing I didnt mention is that I have pulled the whole thing apart. Its now in test bench bench style propped up on cardboard on top of the case to allow me to access all the slots and jumpers easier, the case isn't the easiest to work with.
    What that has also enabled me to do is swap the CPUs round so I now know that this is either a motherboard issue or a RAM issue as the problem doesnt seem to follow one of them. I'm starting to veer towards a simple RAM incompatibility, or perhaps some incorrect 'auto' settings in the BIOS that are causing the RAM to be instable, as this non registered stuff is working OK.

    Im adding a huge chunk of files to the Emby library again now to be doubly sure. I guess the operation on the server which is causing the CATERR is probably lots of small IOPS as it pulls loads of images and test data from websites.

    Would this be a common thing to cause memory errors and can anyone recomend a more reliable method of pushing the RAM in this way to ensure I'm stable?

    Next step is adding the second CPU back in and seeing if I can get that to run with Ubuntu before switching back to unRAID
    Last edited by Biscuit; 30-04-2016 at 01:17 PM. Reason: Removed tapatalk message

  16. #14
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    Well after being reasonably satisfied that single CPU with 2 chips of RAM was stable, I have booted Memtest86 v6 and it crashed... strangely, there isnt any errors in IMPI event log.

    Memtest runs fine on a single CPU, but the minute I set it to run dual CPUs I run into issues. Nothing reported in IMPI still
    Last edited by Biscuit; 30-04-2016 at 05:00 PM.

  17. #15
    Oh Crumbs.... Biscuit's Avatar
    Join Date
    Feb 2007
    Location
    N. Yorkshire
    Posts
    11,192
    Thanks
    1,392
    Thanked
    1,091 times in 833 posts
    • Biscuit's system
      • Motherboard:
      • MSI B450M Mortar
      • CPU:
      • AMD 2700X (Be Quiet! Dark Rock 3)
      • Memory:
      • 16GB Patriot Viper 2 @ 3466MHz
      • Storage:
      • 500GB WD Black
      • Graphics card(s):
      • Sapphire R9 290X Vapor-X
      • PSU:
      • Seasonic Focus Gold 750W
      • Case:
      • Lian Li PC-V359
      • Operating System:
      • Windows 10 x64
      • Internet:
      • BT Infinity 80/20

    Re: Supermicro Dual CPU motherboard

    After a bit of reading it seems there is some issues with memtest in multicore mode with certain setups. Seeing as im using an unusual configuration I guess I probably fall into that group.

    Decided to do a windows installation and run some tests im familiar with in there to test the stability of the system.

    Ran OCCT and at first I hit a few issues with incorrect sensor readings (probably nothing to be concerned about?) but then once I disabled monitoring of them it ran for a couple of hours without problem.
    Honestly though, im not hugely comfortable with this tool so I've gone for a P95 run.

    If you are wondering what that's like with 24 logical cores and the very limited GPU power on a server motherboard




    helpful

    Hopefully all will still be good in the morning.

  18. #16
    Dark side super agent
    Join Date
    Dec 2003
    Location
    Nirvana
    Posts
    1,895
    Thanks
    72
    Thanked
    99 times in 89 posts

    Re: Supermicro Dual CPU motherboard

    Fingers crossed. Can't help I'm afraid but I'm following with interest.
    An Atlantean Triumvirate, Ghosts of the Past, The Centre Cannot Hold
    The Pillars of Britain, Foundations of the Reich, Cracks in the Pillars.

    My books are available here for Amazon Kindle. Feedback always welcome!

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •