Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm having a serious issue with dell server PE2950. The server has really slow IO rates, so slow that I'm not able to use it anymore

I tried few things to solve this:

  1. changing disks to new disks (configured them as raid1)
  2. changing perc card + perc cables
  3. reinstalling the OS of course, had to cause of changing of disks, centos 5.5 x64bit
  4. firmware update to everything
  5. virtual disks policy: No Read Ahead,Write Back, disk cache policy disabled.
  6. openmanage doesn't alert about anything, also i ran dell's diag tests, everything passed, also dell didn't see anything in deset log.
  7. dell offered to reseat everything, including the cpu, we did that as well, still io rates are slow

I have several PE2950 servers, and I never had such a thing with any of those. All have similar or exact hardware as this one, all configured the same, with the same os centos 5.5 x64, same disks, same raid, same policy.

Just for comparison:

the problematic PE2950 server:

[root@bad ~]# time sh -c "dd if=/dev/zero of=/tmp/ddfile bs=8k count=200000 && sync"  

200000+0 records in  
200000+0 records out  
1638400000 bytes (1.6 GB) copied, 27.7946 seconds, 58.9 MB/s  

real    0m33.968s  
user    0m0.531s  
sys     0m26.000s

good PE2950 server (with the exact same hardware):

[root@good ~]# time sh -c "dd if=/dev/zero of=/tmp/ddfile bs=8k count=200000 && sync"  

200000+0 records in  
200000+0 records out  
1638400000 bytes (1.6 GB) copied, 3.19999 seconds, 512 MB/s  

real    0m7.694s  
user    0m0.053s  
sys     0m4.057s 

Hopefully you will have an idea what can cause the problem.

share|improve this question
add comment

3 Answers

If you can I'd try moving the disk controller, cables and disks from the good server to the bad and vice versa, see if the problem stays with the disks/controller or the systemboard. If it stays with the systemboard then put the disks back and swap the CPU/s, again see if the problem moves/stays, then possibly the memory. If it stays with the disks, then try split the disks and controllers. Basically divide and eliminate.

share|improve this answer
add comment
  1. What PERC type is used?
  2. Is there a BBU? Is it alive?
  3. What is the array state when you run the test?
  4. Do you see the same results when booted from a liveCD?
  5. Is there anything loading the disk IO?
  6. Are those SAS or SATA disks?
  7. When updating all firmware, did you also update the HDD firmware?
share|improve this answer
add comment

I had similar issues with PE2900 and PERC 6/i, very slow writes (~ 40 MB/s) with Ubuntu 12.04 and I noticed that when I boot from the CD in rescue mode the speeds were good (~ 200 MB/s), so I started blacklisting linux modules and found out the cause - i5000_edac and edac_core.

vi /etc/modprobe.d/blacklist-edac.conf

blacklist i5000_edac
blacklist edac_core
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.
{"cc_re__cc_re__cc_by_sa_3_0": ["        <li><a href=\"http://area51.stackexchange.com\" title=\"proposing new sites in the Stack Exchange network\">Area 51</a></li>\r\n                        <li><a href=\"http://careers.stackoverflow.com\">Stack Overflow Careers</a></li>\r\n                    \r\n                </ol>\r\n            </td>\r\n    </tr>\r\n</table>\r\n                </div>\r\n            </div>\r\n\r\n            <div id=\"copyright\">\r\n                site design / logo &#169; 2014 stack exchange inc; user contributions licensed under <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\" rel=\"license\">cc by-sa 3.0</a> \r\n                with <a href=\"http://blog.stackoverflow.com/2009/06/attribution-required/\" rel=\"license\">attribution required</a>\r\n            </div>\r\n            <div id=\"svnrev\">\r\n                rev 2014.4.18.1558\r\n            </div>\r\n            \r\n        </div>\r\n    </div>\r\n    <noscript>\r\n        <div id=\"noscript-warning\">Server Fault works best with JavaScript enabled<img src=\"http://pixel.quantserve.com/pixel/p-c1rF4kxgLUzNc.gif\" alt=\"\" class=\"dno\"></div>\r\n    </noscr"], "copyright__copyright__copyright": [" and I noticed that when I boot from the CD in rescue mode the speeds were good (~ 200 MB/s), so I started blacklisting linux modules and found out the cause - i5000_edac and edac_core.\n\nvi /etc/modprobe.d/blacklist-edac.conf\n\nblacklist i5000_edac\nblacklist edac_core\nshare|improve this answer\nadd comment\n\nYour Answer\n\n\u00a0\ndiscard\n\nBy posting your answer, you agree to the privacy policy and terms of service.\n\nNot the answer you're looking for? Browse other questions tagged or ask your own question."]}