Results 1 to 16 of 16

Thread: Finding the bottleneck...

  1. #1
    Member
    Join Date
    Sep 2003
    Location
    Oxfordshire
    Posts
    137
    Thanks
    9
    Thanked
    4 times in 3 posts

    Finding the bottleneck...

    Ok, so I am running some software and it's taking an age to complete what it is doing but the CPU and memory are not fully used (CPU: ~50%, RAM: ~1.05GB of 2GB). It's been going for about 20 minutes.

    Is there any way to identify where the bottleneck in my computer is? The software is reading a large data file from an external drive connected by eSATA.

    How can i speed this up? I can't see how more RAM would help (even if it's not being fully used) or a faster processor - I must use the data on different machines so striping the data accross 2 drives isn't really an option.

  2. #2
    YUKIKAZE arthurleung's Avatar
    Join Date
    Feb 2005
    Location
    Aberdeen
    Posts
    3,280
    Thanks
    8
    Thanked
    88 times in 83 posts
    • arthurleung's system
      • Motherboard:
      • Asus P5E (Rampage Formula 0902)
      • CPU:
      • Intel Core2Quad Q9550 3.6Ghz 1.2V
      • Memory:
      • A-Data DDR2-800 2x2GB CL4
      • Storage:
      • 4x1TB WD1000FYPS @ RAID5 3Ware 9500S-8 / 3x 1TB Samsung Ecogreen F2
      • Graphics card(s):
      • GeCube HD4870 512MB
      • PSU:
      • Corsair VX450
      • Case:
      • Antec P180
      • Operating System:
      • Windows Server 2008 Standard
      • Monitor(s):
      • Dell Ultrasharp 2709W + 2001FP
      • Internet:
      • Be*Unlimited 20Mbps

    Re: Finding the bottleneck...

    I assume you're using a dual core CPU. With a program that only run single threaded, it will NOT use more than 50% CPU (or 25% on a quad core).

    If you can get the program to process different things at the same time (e.g. running two copies). Then it should get to 100%. Otherwise your only choice is to get a faster CPU.
    Workstation 1: Intel i7 950 @ 3.8Ghz / X58 / 12GB DDR3-1600 / HD4870 512MB / Antec P180
    Workstation 2: Intel C2Q Q9550 @ 3.6Ghz / X38 / 4GB DDR2-800 / 8400GS 512MB / Open Air
    Workstation 3: Intel Xeon X3350 @ 3.2Ghz / P35 / 4GB DDR2-800 / HD4770 512MB / Shuttle SP35P2
    HTPC: AMD Athlon X4 620 @ 2.6Ghz / 780G / 4GB DDR2-1000 / Antec Mini P180 White
    Mobile Workstation: Intel C2D T8300 @ 2.4Ghz / GM965 / 3GB DDR2-667 / DELL Inspiron 1525 / 6+6+9 Cell Battery

    Display (Monitor): DELL Ultrasharp 2709W + DELL Ultrasharp 2001FP
    Display (Projector): Epson TW-3500 1080p
    Speakers: Creative Megaworks THX550 5.1
    Headphones: Etymotic hf2 / Ultimate Ears Triple.fi 10 Pro

    Storage: 8x2TB Hitachi @ DELL PERC 6/i RAID6 / 13TB Non-RAID Across 12 HDDs
    Consoles: PS3 Slim 120GB / Xbox 360 Arcade 20GB / PS2

  3. #3
    Member
    Join Date
    Sep 2003
    Location
    Oxfordshire
    Posts
    137
    Thanks
    9
    Thanked
    4 times in 3 posts

    Re: Finding the bottleneck...

    Yes, it's a dual core. But neither core reaches 100% usage, so something appears to be slowing the process down (which isn't the processor).

  4. #4
    Senior Member
    Join Date
    Jan 2008
    Location
    Q2DM1
    Posts
    259
    Thanks
    7
    Thanked
    20 times in 20 posts
    • Viper81's system
      • Motherboard:
      • Asus P8Z77-V PRO
      • CPU:
      • Intel i5 3570K
      • Memory:
      • 16Gb Corsair XMS3 1600MHz
      • Storage:
      • Many
      • Graphics card(s):
      • Asus 1Gb 6950
      • PSU:
      • Corsair 650W
      • Case:
      • Antec P182
      • Operating System:
      • Win 7 x64
      • Monitor(s):
      • 25.5" HP W2558HC
      • Internet:
      • Tin Can and String

    Re: Finding the bottleneck...

    You might be saturating the sata bus. I'd try using something like (process explorer) to check your system I/O. You should be able to see if it's a flat line (i.e. maxed out).
    Another option is that that is as efficient as that particular piece of software gets, if an algorithm is coded badly there's not much you can do to improve your performance.

  5. #5
    Senior Member
    Join Date
    May 2009
    Location
    Norfolk
    Posts
    474
    Thanks
    3
    Thanked
    26 times in 26 posts
    • pipTheGeek's system
      • Motherboard:
      • Asus P6T Deluxe
      • CPU:
      • Core i7 920 @ 3.6GHz
      • Memory:
      • 3 * 2Gb Corsair XMS @ DDR3 1800
      • Storage:
      • 300GB 15K SAS + 500Gb
      • Graphics card(s):
      • GTX570
      • PSU:
      • corsair 760i
      • Case:
      • Corsair 550d
      • Operating System:
      • Windows 7
      • Monitor(s):
      • Dell Alienware 23"
      • Internet:
      • VM 50Mb

    Re: Finding the bottleneck...

    No, arthurleung is right. It is a single threaded application that is CPU bound. The reason that it doesn't show a single core at 100% and the other idle is because windows is scheduling the 1 busy thread on each core randomly. Either get a faster CPU or a better piece of software that can use multiple cores.

  6. #6
    Senior Member
    Join Date
    Jan 2008
    Location
    Q2DM1
    Posts
    259
    Thanks
    7
    Thanked
    20 times in 20 posts
    • Viper81's system
      • Motherboard:
      • Asus P8Z77-V PRO
      • CPU:
      • Intel i5 3570K
      • Memory:
      • 16Gb Corsair XMS3 1600MHz
      • Storage:
      • Many
      • Graphics card(s):
      • Asus 1Gb 6950
      • PSU:
      • Corsair 650W
      • Case:
      • Antec P182
      • Operating System:
      • Win 7 x64
      • Monitor(s):
      • 25.5" HP W2558HC
      • Internet:
      • Tin Can and String

    Re: Finding the bottleneck...

    I don't quite follow, fray said that neither core is showing 100% (i.e not talking about the "cpu usage" percentage that taskmanager shows, but rather the individual core utilisation graphs) .
    If Windows is switching the single thread between cores why would you not see one of the cores hit 100%?

  7. #7
    Member
    Join Date
    Sep 2003
    Location
    Oxfordshire
    Posts
    137
    Thanks
    9
    Thanked
    4 times in 3 posts

    Re: Finding the bottleneck...

    hmm, viper81's comment seems sensible-any ideas?

  8. #8
    Banhammer in peace PeterB kalniel's Avatar
    Join Date
    Aug 2005
    Posts
    31,023
    Thanks
    1,870
    Thanked
    3,381 times in 2,718 posts
    • kalniel's system
      • Motherboard:
      • Gigabyte Z390 Aorus Ultra
      • CPU:
      • Intel i9 9900k
      • Memory:
      • 32GB DDR4 3200 CL16
      • Storage:
      • 1TB Samsung 970Evo+ NVMe
      • Graphics card(s):
      • nVidia GTX 1060 6GB
      • PSU:
      • Seasonic 600W
      • Case:
      • Cooler Master HAF 912
      • Operating System:
      • Win 10 Pro x64
      • Monitor(s):
      • Dell S2721DGF
      • Internet:
      • rubbish

    Re: Finding the bottleneck...

    Quote Originally Posted by Viper81 View Post
    If Windows is switching the single thread between cores why would you not see one of the cores hit 100%?
    Because that 'single-threaded' process might actually have lots of threads that have to run concurrently, leading to what looks like on average some CPU time being run on one core and some on the other.

    However, as soon as he mentioned large data then I/O becomes my primary suspect for a bottleneck. I/O is slow. I/O to a disk is even slower. I/O to an external disk is even slower still. Forget saturating the SATA bandwidth - you won't even get close to that.

    The only way to speed it up is to either read that data more efficiently (like sucking it all up into a buffer sequentually rather than having lots of little reads), or transfer less data (for eg. by using a compressed version of the data and uncompressing in memory).

    Or of course buy a faster disk.

  9. #9
    Senior Member
    Join Date
    May 2009
    Location
    Norfolk
    Posts
    474
    Thanks
    3
    Thanked
    26 times in 26 posts
    • pipTheGeek's system
      • Motherboard:
      • Asus P6T Deluxe
      • CPU:
      • Core i7 920 @ 3.6GHz
      • Memory:
      • 3 * 2Gb Corsair XMS @ DDR3 1800
      • Storage:
      • 300GB 15K SAS + 500Gb
      • Graphics card(s):
      • GTX570
      • PSU:
      • corsair 760i
      • Case:
      • Corsair 550d
      • Operating System:
      • Windows 7
      • Monitor(s):
      • Dell Alienware 23"
      • Internet:
      • VM 50Mb

    Re: Finding the bottleneck...

    Let me try an explain better.
    Checkout staff are threads
    Checkouts are cores
    You have one member of staff and give them 2 checkouts. If you now force them to switch between the 2 checkouts (which is what the windows thread scheduler is doing) and measure how busy all the checkouts are, you will find that as a collective whole they are about 50% busy.
    The windows scheduler causes the single busy thread to swap core becuase it has more threads that require cpu time than you have cores. Each time the single really busy threads gets a turn it arrives on whichever core windows has deemed free.
    This is why a single busy thread will give you a CPU usage of 50% on a dual core. It might be slightly over 50% if other tasks are using the other core, it might be slightly under if there are lots of context switches happening.
    If you want to check how memory and disk is being used then you can either look at perfmon which can tell you all sorts of things about what windows and your hardware is doing. Or you can take a look at spotlight for windows, it is made by quest and is now free. It shows the same information that perfmon captures but in a much easier to digest form.

  10. #10
    HEXUS.timelord. Zak33's Avatar
    Join Date
    Jul 2003
    Location
    I'm a Jessie
    Posts
    35,176
    Thanks
    3,121
    Thanked
    3,173 times in 1,922 posts
    • Zak33's system
      • Storage:
      • Kingston HyperX SSD, Hitachi 1Tb
      • Graphics card(s):
      • Nvidia 1050
      • PSU:
      • Coolermaster 800w
      • Case:
      • Silverstone Fortress FT01
      • Operating System:
      • Win10
      • Internet:
      • Zen FTC uber speedy

    Re: Finding the bottleneck...

    may I ask what the two hard drives are?

    What's in the external drive? What brand etc? And what's in your PC?

    Quote Originally Posted by Advice Trinity by Knoxville
    "The second you aren't paying attention to the tool you're using, it will take your fingers from you. It does not know sympathy." |
    "If you don't gaffer it, it will gaffer you" | "Belt and braces"

  11. #11
    Registered+
    Join Date
    May 2009
    Location
    Wiltshire
    Posts
    22
    Thanks
    0
    Thanked
    1 time in 1 post
    • JordanWillis's system
      • Motherboard:
      • Asus Striker II Formula 780i
      • CPU:
      • Core 2 Duo - E8600
      • Memory:
      • Corsair XMS2 - 800Mhz
      • Storage:
      • Seagate barracuda 7200rpm 750Gb
      • Graphics card(s):
      • 2 x XFX GTX260 XT Ed in SLI
      • PSU:
      • Corsair TX750w
      • Case:
      • Coolermaster HAF 932
      • Operating System:
      • Windows XP 32bit.
      • Monitor(s):
      • Samsung SM2333sw 23" 1080p
      • Internet:
      • Virgin Media 20Mb

    Re: Finding the bottleneck...

    If you do that sorta stuff that is quite intesive then you might wanna buy a fast dual core, average Quad.

    Things that speed up your Pc most is the Harddrive, what type of HDD are you reading from?

  12. #12
    lost
    Join Date
    Aug 2005
    Location
    North Manchester
    Posts
    324
    Thanks
    7
    Thanked
    37 times in 25 posts
    • llololloy's system
      • Motherboard:
      • gigabyte p67a-ud4
      • CPU:
      • i5 2500s
      • Memory:
      • 8gb ddr3
      • Storage:
      • 60gb ssd
      • Graphics card(s):
      • r9 280x
      • PSU:
      • Corsair HX 520
      • Case:
      • Coolermaster Cosmos
      • Operating System:
      • w10 x64
      • Monitor(s):
      • 24" + 2 * 23" 1080p
      • Internet:
      • Virgin 200meg

    Re: Finding the bottleneck...

    is it something that you have to do regularly?
    It may be that investing in new disks (or even raid) helps you out.
    MediaPC - ASRock H61M-DGS - 8gb ddr3 - i3 2120 - R9 270 - antec fusion black - lg ggc-h20l - windows 10 - denon avr-1911 - JVC 58" 1080p RPTV
    Server - MSI Z77A-G45 - 16gb ddr3 - i7 2600k - 4u Rack mountable - sas 1068e + expander - 3ware 9650se - 16 * 2tb + 2 * 3tb - TBS 6984, 6284 & 6920 - 2 * astrometa dvb-c - windows 7 pro
    My PC - See my system on left

    hexus.trust

  13. #13
    Member
    Join Date
    Sep 2003
    Location
    Oxfordshire
    Posts
    137
    Thanks
    9
    Thanked
    4 times in 3 posts

    Re: Finding the bottleneck...

    The external drive is a seagate 1TB Xtreme, it's connected to the computer by eSATA into a pci-express eSATA card. I'm not totally sure what drive is the enclosure, only that it's a 7200rpm drive. I'm not at the computer now so not sure what the internal drive is, it was an off the shelf hp machine bought about 9 months ago (could find out later).

    At the moment it's part of my work to deal with pretty large files of 3d data but i need to use more than one machine as different software is installed on different machines so i use the external drive as my working storage - so raid wouldnt really work for me.

  14. #14
    lost
    Join Date
    Aug 2005
    Location
    North Manchester
    Posts
    324
    Thanks
    7
    Thanked
    37 times in 25 posts
    • llololloy's system
      • Motherboard:
      • gigabyte p67a-ud4
      • CPU:
      • i5 2500s
      • Memory:
      • 8gb ddr3
      • Storage:
      • 60gb ssd
      • Graphics card(s):
      • r9 280x
      • PSU:
      • Corsair HX 520
      • Case:
      • Coolermaster Cosmos
      • Operating System:
      • w10 x64
      • Monitor(s):
      • 24" + 2 * 23" 1080p
      • Internet:
      • Virgin 200meg

    Re: Finding the bottleneck...

    if you had some money to spend on it, either a nas box with built in raid (can't recommend one, sorry, don't know enough about them) or building a small server might help.
    can get mobo, cpu and ram for under £100, then either use on board raid (which might be good enough) or linux raid, or get a pci-e raid5 card. that, in theory, could max out a gigabit lan connection. more you spend, faster it gets.
    Last edited by llololloy; 27-05-2009 at 10:11 AM. Reason: ...
    MediaPC - ASRock H61M-DGS - 8gb ddr3 - i3 2120 - R9 270 - antec fusion black - lg ggc-h20l - windows 10 - denon avr-1911 - JVC 58" 1080p RPTV
    Server - MSI Z77A-G45 - 16gb ddr3 - i7 2600k - 4u Rack mountable - sas 1068e + expander - 3ware 9650se - 16 * 2tb + 2 * 3tb - TBS 6984, 6284 & 6920 - 2 * astrometa dvb-c - windows 7 pro
    My PC - See my system on left

    hexus.trust

  15. #15
    Senior Member
    Join Date
    May 2009
    Location
    Norfolk
    Posts
    474
    Thanks
    3
    Thanked
    26 times in 26 posts
    • pipTheGeek's system
      • Motherboard:
      • Asus P6T Deluxe
      • CPU:
      • Core i7 920 @ 3.6GHz
      • Memory:
      • 3 * 2Gb Corsair XMS @ DDR3 1800
      • Storage:
      • 300GB 15K SAS + 500Gb
      • Graphics card(s):
      • GTX570
      • PSU:
      • corsair 760i
      • Case:
      • Corsair 550d
      • Operating System:
      • Windows 7
      • Monitor(s):
      • Dell Alienware 23"
      • Internet:
      • VM 50Mb

    Re: Finding the bottleneck...

    From what you said in your original post I still don't think the problem is anything other than CPU bound. Spotlight that I mentioned previously is available from here http://www.quest.com/spotlight-on-windows/
    You have to give name / e-mail address to download it, I have, and haven't recieved any spam as a result. It ought to be able to show you exactly what is happening, especially with regard to the disks. It will show you disk queue, number of operations per second, transfer time and % load for all logical and physical disks. If you see % load maxed out, or a sustained high disk queue then your problem is disks. If not then your problem is probably elsewhere. (High disk usage could be a symptom of high swap file use but spotlight will tell you if excessive paging is happening)

  16. #16
    Not a good person scaryjim's Avatar
    Join Date
    Jan 2009
    Location
    Gateshead
    Posts
    15,196
    Thanks
    1,231
    Thanked
    2,291 times in 1,874 posts
    • scaryjim's system
      • Motherboard:
      • Dell Inspiron
      • CPU:
      • Core i5 8250U
      • Memory:
      • 2x 4GB DDR4 2666
      • Storage:
      • 128GB M.2 SSD + 1TB HDD
      • Graphics card(s):
      • Radeon R5 230
      • PSU:
      • Battery/Dell brick
      • Case:
      • Dell Inspiron 5570
      • Operating System:
      • Windows 10
      • Monitor(s):
      • 15" 1080p laptop panel

    Re: Finding the bottleneck...

    Quote Originally Posted by pipTheGeek View Post
    From what you said in your original post I still don't think the problem is anything other than CPU bound.
    Then I question your knowledge of how computers work. Any speed problem involving moving files is 90% likely to be bound by disk IO, which is by far the slowest thing in your computer. For one thing, fray hasn't complained about his computer running slowly otherwise, just that this file is taking a long time to open. If the computer is still responding quickly while the process is ongoing, then it's not a CPU issue.

    How big is the file you're trying to open, fray? And what program are you using to try to open it? Some programs will have a limit to the amount of virtual memory they may access, and it could be that the file is too big for the program's working set (if the program has been badly set up).

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. E6550 - would this bottleneck a 4870/90?
    By funke_munke in forum Graphics Cards
    Replies: 4
    Last Post: 03-04-2009, 03:39 PM
  2. Finding old, dead people.
    By WhiteGiant in forum General Discussion
    Replies: 22
    Last Post: 09-01-2009, 09:26 PM
  3. Replies: 13
    Last Post: 10-12-2008, 09:02 PM
  4. Bottleneck in 3DMark?
    By EarlGrey in forum PC Hardware and Components
    Replies: 5
    Last Post: 08-02-2008, 12:18 PM
  5. finding a scrapyard
    By 5lab in forum Automotive
    Replies: 13
    Last Post: 24-06-2006, 12:15 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •