Page 2 of 3 FirstFirst 123 LastLast
Results 17 to 32 of 44

Thread: Slower concurrent disk access with NCQ ?

  1. #17
    Senior Member
    Join Date
    Jul 2007
    Posts
    524
    Thanks
    79
    Thanked
    67 times in 47 posts

    Re: Slower concurrent disk access with NCQ ?

    Quote Originally Posted by badass View Post
    *i.e.
    Opening of notepad when copying large file without NCQ may take 3 seconds, file copy takes 2 minutes. With NCQ, the file copy will take less than 2 minutes and notepad will take less than 3 seconds to load.
    In terms of discrete time needed to load from disk, yes.
    In terms of perceived time, these times may be interleaved in a manner which results in the more distant operation appearing to take longer, as it's commands are delayed, but in total the timings are shorter. That was what I was referring to.

    Edit: I just re-read my initial post, and it turns out I had already stated that. So why exactly people want to believe otherwise is beyond me Yes, I am willing to chart this all out if need be, but it is exam season, and I'd rather that right now, people read into how this works themselves. If you just plot out the access points over time for a random set of requests, unsorted and sorted, this is an immediately self-evident result. Individual response time may increase, to allow system total response time to increase. Text-book stuff, quite frankly, and not for the module I have an exam on tomorrow.

    Edit 2: Yes, the above is slightly over the top. Regulars know that I normally love explaining this sort of thing, and I'm just really stressed and really sad that I cannot put the effort into explaining how NCQ works right now

    It doesn't matter if the loading is faster, if there is a delay before it begins. Where the benefit from NCQ ordering reduces the time needed to load the program by a factor greater than the delay, then NCQ is beneficial.

    A lot of this will be dependant upon the SATA controller on the motherboard (as these can also do various 'optimisations' themselves), the operating system (which traditionally also does a small amount command queuing) and upon the vendor specific implementation of NCQ and the command buffer size.
    Last edited by Rosaline; 21-05-2008 at 10:46 PM.

  2. #18
    TiG
    TiG is offline
    Walk a mile in other peoples shoes...
    Join Date
    Jul 2003
    Location
    Questioning it all
    Posts
    6,213
    Thanks
    45
    Thanked
    48 times in 43 posts

    Re: Slower concurrent disk access with NCQ ?

    NCQ was designed for servers on scsi systems years ago in the first place, be it web servers, database servers, we're talking a lot more disk access than those tests above, concurrent heavy reads, writes, scsi still has far more io capability than sata.

    What sometimes I can't understand is why people worry about whether something loads in 2.8 seconds or 3 seconds.

    TiG
    -- Hexus Meets Rock! --

  3. #19
    Senior Member
    Join Date
    Mar 2005
    Posts
    4,570
    Thanks
    145
    Thanked
    309 times in 247 posts
    • badass's system
      • Motherboard:
      • ASUS P8Z77-m pro
      • CPU:
      • Core i5 3570K
      • Memory:
      • 32GB
      • Storage:
      • 1TB Samsung 850 EVO, 2TB WD Green
      • Graphics card(s):
      • Radeon RX 580
      • PSU:
      • Corsair HX520W
      • Case:
      • Silverstone SG02-F
      • Operating System:
      • Windows 10 X64
      • Monitor(s):
      • Del U2311, LG226WTQ
      • Internet:
      • 80/20 FTTC

    Re: Slower concurrent disk access with NCQ ?

    Quote Originally Posted by Rosaline View Post
    Edit: I just re-read my initial post, and it turns out I had already stated that. So why exactly people want to believe otherwise is beyond me Yes, I am willing to chart this all out if need be, but it is exam season, and I'd rather that right now, people read into how this works themselves. If you just plot out the access points over time for a random set of requests, unsorted and sorted, this is an immediately self-evident result. Individual response time may increase, to allow system total response time to increase. Text-book stuff, quite frankly, and not for the module I have an exam on tomorrow.
    Individual response times will only increase when you are not talking about multiple access.
    When you are talking about multiple access, with NCQ, percieved individual response times for items will not increase. They will decerease. Even if one item has only a few MB to load.
    This reordering doesn't take a few seconds. It takes milliseconds.
    So, yes, Wordpad may get its first bytes a few ms earlier without NCQ, it may not, however when we are taling about the app taking over a second to load, it will still finish loading faster with NCQ because it is getting data off the disk faster than it otherwise would without NCQ.

    If you notice anything slower when you are using NCQ, something is wrong. end of. It does not just make the several jobs get done quicker all together, every individual part of the multiple file copy, open etc will also be quicker. Any theory that says otherwise is based on a flawed understanding.
    "In a perfect world... spammers would get caught, go to jail, and share a cell with many men who have enlarged their penises, taken Viagra and are looking for a new relationship."

  4. #20
    Senior Member
    Join Date
    Jul 2007
    Posts
    524
    Thanks
    79
    Thanked
    67 times in 47 posts

    Re: Slower concurrent disk access with NCQ ?

    Badass, I do not have the time to debate this right now. I would love to, and as my posting history shows, I normally am quite happy to drag out the actual theory. However, I have exams going on at the moment, and so cannot spend the time correcting you.

    Please, go away and read into how it works, and the fact that some commands will hence be processed later than when they originally arrived at the hard drive. This is not even hard to understand.

    I will concur however that a delay in the magnitude of seconds is unlikely to be caused by NCQ, given the speed of disk rotation and low seek times. But you are quite frankly wrong that NCQ will always magically speed up individual response times - it depends on the load, and NCQ focuses on speeding up the system as a whole which under heavy load, speeds up most individual requests, but may favour only a few common ones under small loads.

    Please, read into this, and stop declaring others to have flawed understanding. If you like, I'll return to this in a few weeks to show how individual processes may find that their requests take longer. It is a simple text book issue that I am frankly shocked to have to debate at all.

    Edit: As already stated, though, a lot of this is highly dependant upon the performance of the system, hence why sweeping generalisations and accusations are ultimately futile.

  5. #21
    Gentoo Ricer
    Join Date
    Jan 2005
    Location
    Galway
    Posts
    11,025
    Thanks
    1,009
    Thanked
    944 times in 704 posts
    • aidanjt's system
      • Motherboard:
      • Asus Strix Z370-G
      • CPU:
      • Intel i7-8700K
      • Memory:
      • 2x8GB Corsiar LPX 3000C15
      • Storage:
      • 500GB Samsung 960 EVO
      • Graphics card(s):
      • EVGA GTX 970 SC ACX 2.0
      • PSU:
      • EVGA G3 750W
      • Case:
      • Fractal Design Define C Mini
      • Operating System:
      • Windows 10 Pro
      • Monitor(s):
      • Asus MG279Q
      • Internet:
      • 240mbps Virgin Cable

    Re: Slower concurrent disk access with NCQ ?

    Roseline, as far as your understanding goes of the hardware technology, you're absolutely correct, however, you can't ignore kernel disk I/O queuing factors, and how it works with NCQ. When I was speaking of NCQ, and how it optimises I/O in the way I described, I took OS optimisations into account. NCQ is also less aggressive in minimalist seeks compared to TCQ, which is why it won out, despite TCQ being technically superior, it's suboptimal for desktop kernels.
    Quote Originally Posted by Agent View Post
    ...every time Creative bring out a new card range their advertising makes it sound like they have discovered a way to insert a thousand Chuck Norris super dwarfs in your ears...

  6. #22
    Senior Member
    Join Date
    Jul 2007
    Posts
    524
    Thanks
    79
    Thanked
    67 times in 47 posts

    Re: Slower concurrent disk access with NCQ ?

    Aidanjt, are you referring to the following, as I can't find another post by you talking about NCQ:
    NCQ is only useful for stacking SATA commands in an order which is optimal for i/o across several sections of the disk surface in as little rotations as possible, so, great for 10 different processes doing i/o in different files... pointless for copying 2gb file from a to b. Your benchmark wouldn't make use of it.
    I think we're speaking on the same wavelength in general, and I did mention that the entire issue is complicated by things such as OS optimisations (which yes, it too tries to order them nicely in general).

  7. #23
    Gentoo Ricer
    Join Date
    Jan 2005
    Location
    Galway
    Posts
    11,025
    Thanks
    1,009
    Thanked
    944 times in 704 posts
    • aidanjt's system
      • Motherboard:
      • Asus Strix Z370-G
      • CPU:
      • Intel i7-8700K
      • Memory:
      • 2x8GB Corsiar LPX 3000C15
      • Storage:
      • 500GB Samsung 960 EVO
      • Graphics card(s):
      • EVGA GTX 970 SC ACX 2.0
      • PSU:
      • EVGA G3 750W
      • Case:
      • Fractal Design Define C Mini
      • Operating System:
      • Windows 10 Pro
      • Monitor(s):
      • Asus MG279Q
      • Internet:
      • 240mbps Virgin Cable

    Re: Slower concurrent disk access with NCQ ?

    I think we may be

    *edit* Also, good luck with the exams.
    Last edited by aidanjt; 22-05-2008 at 01:44 AM.
    Quote Originally Posted by Agent View Post
    ...every time Creative bring out a new card range their advertising makes it sound like they have discovered a way to insert a thousand Chuck Norris super dwarfs in your ears...

  8. #24
    Registered+
    Join Date
    Mar 2008
    Posts
    18
    Thanks
    0
    Thanked
    0 times in 0 posts

    Re: Slower concurrent disk access with NCQ ?

    PLEASE STOP the "I know NCQ better than thou" flame!!!!!!!!!!!!

    Thank you.

    It is completely irrelevant to this thread, because all we know, NCQ might not be used in the problem case at all. All we know is that the HW is in AHCI mode and an AHCI driver is controlling it.

    Also half second up or down does not matter. The problem is 3 seconds versus 100 seconds. A factor of THIRTY.

  9. #25
    Senior Member
    Join Date
    Jul 2007
    Posts
    524
    Thanks
    79
    Thanked
    67 times in 47 posts

    Re: Slower concurrent disk access with NCQ ?

    As already stated, that sort of time difference will not be down to NCQ at all, but more likely a driver or hardware fault.

    If things run fine with AHCI mode turned off, then there's an obvious fix

    Have you tried checking in the windows error log to see if there's any reported issues with the hard drive? That long a delay I would suspect to mean that it tries several times then falls back to a slower interfacing method.

  10. #26
    Senior Member
    Join Date
    Mar 2005
    Posts
    4,570
    Thanks
    145
    Thanked
    309 times in 247 posts
    • badass's system
      • Motherboard:
      • ASUS P8Z77-m pro
      • CPU:
      • Core i5 3570K
      • Memory:
      • 32GB
      • Storage:
      • 1TB Samsung 850 EVO, 2TB WD Green
      • Graphics card(s):
      • Radeon RX 580
      • PSU:
      • Corsair HX520W
      • Case:
      • Silverstone SG02-F
      • Operating System:
      • Windows 10 X64
      • Monitor(s):
      • Del U2311, LG226WTQ
      • Internet:
      • 80/20 FTTC

    Re: Slower concurrent disk access with NCQ ?

    Quote Originally Posted by Rosaline View Post
    Badass, I do not have the time to debate this right now. I would love to, and as my posting history shows, I normally am quite happy to drag out the actual theory.
    And that is the point.
    Theoretically NCQ can make individual processes slower, however because of the hundreds of transactions per second, in practice it never does.
    I understand how NCQ works just fine thank you and I understand what you are saying and the theory is indeed correct, however arguing this based on theory is like someone coming up lots of completely sound theory as to why the sky is green.
    I understand that you have exams to worry about so dont have the time now, but the only way to prove me wrong on this is to get some actual graphs where people have tested real world scenarios. You'll see that in practice the only thing that is measurably slower is single application disk access.

    EDIT: Just thought of a faaaar better analogy.

    Its like arguing with someone that Windows* copies files across a network when the fast ethernet cards are both running full duplex is in fact slower thean when you set both NIC's to half duplex.
    You find that in practice, with the NIC's set to half duplex, the copying of single files is 4 times faster. The theory suggests that full duplex should be a miniscule bit faster.

    *2000 and prior. Not sure of you get this effect with XP or Vista
    Last edited by badass; 22-05-2008 at 12:26 PM.
    "In a perfect world... spammers would get caught, go to jail, and share a cell with many men who have enlarged their penises, taken Viagra and are looking for a new relationship."

  11. #27
    Registered User
    Join Date
    Jul 2008
    Posts
    10
    Thanks
    0
    Thanked
    0 times in 0 posts
    • |FAT|Punisher's system
      • Motherboard:
      • Foxconn Mars (P35+ICH9R)
      • CPU:
      • Intel Q6600 @ 3.4 GHz
      • Memory:
      • 2x 2048 MB OCZ Platinum XTC DDR2-1000
      • Storage:
      • 2x WD2500YD @ RAID 0, 1x WD6400AAKS
      • Graphics card(s):
      • Gigabyte GeForce 8800 GTX
      • PSU:
      • be quiet! Straight Power 600W
      • Case:
      • NZXT Lexa Redline
      • Monitor(s):
      • Samtron 98PDF (19" CRT)
      • Internet:
      • 16000 kbps DSL

    Re: Slower concurrent disk access with NCQ ?

    This topic is somewhat older now, and I don't know if somebody still watches it, but I have the same problem as xerces8.

    I recently bought a Western Digital WD6400AAKS, which is a great drive, big, really really fast, cheap. I'm using a Foxconn Mars which has an Intel ICH9R. I have to run the ICH9R in RAID mode, because I have a RAID 0 consisting of two WD2500YDs. The RAID 0 works fine, like it used to do on an earlier mainboard with nForce4. The WD6400AAKS however shows the same behaviour xerces8 has seen: if there is only one application which reads (or writes, doesn't matter) huge amounts of data (sequentially), other applications accessing that drive hang completely (even the GUI) for a long time, meaning tens of seconds and more. Even the Explorer can hang when trying to access the drive.

    I have never seen a behaviour like that before. Of course everything gets slower (a lot slower), if several apps access the same drive, but every app still works and reacts. At least that's what I've experienced until now, during the last decade and more.

    My first thought was NCQ, too, and unfortunately I can't find a way to disable it. The jumper on the drive reduces the transfer mode to SATA generation 1, but NCQ seems to stay active (the Intel Matrix Storage Manager still says that it is "supported", it does not say if it is actually used). There's no option in the BIOS (because the controller has to stay in RAID mode), or in the RAID setup either. No option in Windows, neither in the Intel Matrix Storage Manager.

    So I'm not sure if it is NCQ, but I am completely sure that this behaviour is not "normal". The Windows event log doesn't show anything suspicious, by the way.


    Any ideas?

  12. #28
    Registered+
    Join Date
    Mar 2008
    Posts
    18
    Thanks
    0
    Thanked
    0 times in 0 posts

    Re: Slower concurrent disk access with NCQ ?

    Hi!

    Can you give more details about your setup ?
    What other disks do you have connected ?
    Does the problem happen only with a WD disk ?


    I tried a different HD on my system (WD MyBook Studio 500GB) and it has the same problem. I used RAID mode this time (used AHCI before).

    Regards,
    David

  13. #29
    Registered User
    Join Date
    Jul 2008
    Posts
    10
    Thanks
    0
    Thanked
    0 times in 0 posts
    • |FAT|Punisher's system
      • Motherboard:
      • Foxconn Mars (P35+ICH9R)
      • CPU:
      • Intel Q6600 @ 3.4 GHz
      • Memory:
      • 2x 2048 MB OCZ Platinum XTC DDR2-1000
      • Storage:
      • 2x WD2500YD @ RAID 0, 1x WD6400AAKS
      • Graphics card(s):
      • Gigabyte GeForce 8800 GTX
      • PSU:
      • be quiet! Straight Power 600W
      • Case:
      • NZXT Lexa Redline
      • Monitor(s):
      • Samtron 98PDF (19" CRT)
      • Internet:
      • 16000 kbps DSL

    Re: Slower concurrent disk access with NCQ ?

    I've filled in my system now, and if you visit "my homepage", you'll see a more detailed description at sysProfile.de.

    I have used a lot of WD discs (WD800JB, WD1600JB, WD2500YD, WD5000YS, WD6400AAKS), but this is the first hard disc ever that exposes this behaviour.

    As far as I know, RAID mode implicates AHCI.

  14. #30
    Registered User
    Join Date
    Jul 2008
    Posts
    10
    Thanks
    0
    Thanked
    0 times in 0 posts
    • |FAT|Punisher's system
      • Motherboard:
      • Foxconn Mars (P35+ICH9R)
      • CPU:
      • Intel Q6600 @ 3.4 GHz
      • Memory:
      • 2x 2048 MB OCZ Platinum XTC DDR2-1000
      • Storage:
      • 2x WD2500YD @ RAID 0, 1x WD6400AAKS
      • Graphics card(s):
      • Gigabyte GeForce 8800 GTX
      • PSU:
      • be quiet! Straight Power 600W
      • Case:
      • NZXT Lexa Redline
      • Monitor(s):
      • Samtron 98PDF (19" CRT)
      • Internet:
      • 16000 kbps DSL

    Re: Slower concurrent disk access with NCQ ?

    I think I have found the reason for our problems.

    In certain situations, these extreme delays became unacceptable, so I put more time into this problem.
    I gave the JMicron JMB361 on my board a chance, et voilà: no delays, no problems, everything is as it should be!

    Unfortunately, my Foxconn Mars has only an eSATA port, which is not an option for my internal 6400AAKS. And even worse: I was thinking about buying more 6400AAKS, for a decent RAID 5, because they're cheap, big and incredibly fast. That does not seem to be a good idea anymore...

    Unfortunately, I wasn't able to check if NCQ is activated on the JMicron controller.
    [Edit] HD Tune states that it is active. Or supported? I don't know, but NCQ is checked.

  15. #31
    Registered+
    Join Date
    Mar 2008
    Posts
    18
    Thanks
    0
    Thanked
    0 times in 0 posts

    Re: Slower concurrent disk access with NCQ ?

    So you moved the disk from the ICH port to the JMB ?

    I also have a JMB363. It gives even more trouble (with another HD, a WD MyBook Studio 500GB). I guess each controller has problems with different discs ...

  16. #32
    Registered User
    Join Date
    Jul 2008
    Posts
    10
    Thanks
    0
    Thanked
    0 times in 0 posts
    • |FAT|Punisher's system
      • Motherboard:
      • Foxconn Mars (P35+ICH9R)
      • CPU:
      • Intel Q6600 @ 3.4 GHz
      • Memory:
      • 2x 2048 MB OCZ Platinum XTC DDR2-1000
      • Storage:
      • 2x WD2500YD @ RAID 0, 1x WD6400AAKS
      • Graphics card(s):
      • Gigabyte GeForce 8800 GTX
      • PSU:
      • be quiet! Straight Power 600W
      • Case:
      • NZXT Lexa Redline
      • Monitor(s):
      • Samtron 98PDF (19" CRT)
      • Internet:
      • 16000 kbps DSL

    Re: Slower concurrent disk access with NCQ ?

    Yeah, I moved it from the ICH9R port to the JMB361. I used an eSATA-to-SATA cable because the JMB361 on my board has only one eSATA port.

    - And indeed, I have other problems now.
    From time to time, the JMB hangs completely, all applications accessing the drive hang completely, for several seconds. The Windows Event Log shows: "The device \Device\Scsi\JRAID1 did not respond within the timeout period." (translated from German, might not be an exact translation).

    The worst thing is: I have no clue if the reason for this is the JMicron controller (thus it would be a second, different problem), or if it is still the hard drive.

    What are your experiences with the JMicron?


    [Edit] I just realized that I had already encountered the "JMicron problem" with my WD5000YS, too!
    But what do I do now? I have no money to spend on a "real" RAID controller for several 100 bucks...

Page 2 of 3 FirstFirst 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. How to Speed up your Hard Disk :-) (Zak's Partition Theory)
    By Zak33 in forum PC Hardware and Components
    Replies: 123
    Last Post: 13-09-2008, 06:37 PM
  2. best disk cloning software?
    By DsW in forum Software
    Replies: 30
    Last Post: 02-07-2008, 09:01 PM
  3. help in accessing readable data on damaged hard disk
    By kulasa in forum Help! Quick Relief From Tech Headaches
    Replies: 1
    Last Post: 19-09-2006, 12:53 AM
  4. Belkin F5D6130 802.11b Wireless Access Point
    By Howard in forum Reader Reviews
    Replies: 4
    Last Post: 10-12-2004, 11:20 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •