I recently installed CentOS 5 on a server with a Promise PDC20621 SATA Raid card in it (according to lspci). This particular card, of course, is a FAKE raid device, meaning that the physical card is nothing more than a regular SATA controller, and they provide drivers that emulate RAID functionality. This is supposed to be useful for Windows users that don’t have a native software raid service available, but it is kindof useless for Linux since most distros provide md for creating a software raid device.
When trying to create a new software raid array, I would get a bunch of errors about the devices being busy, like this:
[root@www ~]# mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm: layout defaults to left-symmetric mdadm: chunk size defaults to 64K mdadm: Cannot open /dev/sda1: Device or resource busy mdadm: Cannot open /dev/sdb1: Device or resource busy mdadm: Cannot open /dev/sdc1: Device or resource busy mdadm: Cannot open /dev/sdd1: Device or resource busy
lsof didn’t show any processes that were using these files, and it took a little while to finally find out that ‘dmraid’ was the culprit. dmraid is the linux driver for fake raid controllers like the Promise FastTrak and nVidia on-board SATA controllers. From what I could tell, it is loaded from initrd and automatically attaches itself to any partitions that are of type ‘fd’ (Linux raid autodetect).
After a few hours of googling for answers, I had become pretty familiar with the topic. Many of the search results were from people trying to get mdraid working for these devices before it was stable and widely included in distros.
Unfortunately, it looks like the default CentOS 5 install has the dmraid drivers built into the initrd, and there was no way to disable it from taking control of the drives. I tried looking for an argument to pass to the kernel to disable dmraid support, but couldn’t find anything. A few of the posts and emails that I came across on the subject suggested removing the ‘dmraid’ package, and a few people appeared to have some success with that. But when I tried a ‘yum erase dmraid’ on my box, it wanted to remove the kernel, which would probably be bad.
After a little more searching, I found that mkinitrd had an option to rebuild the initrd without dmraid support. The was an upgrade available for my kernel, so I did a ‘yum update’ to install a new one, which also gave me one to fall-back to if this didn’t work. Once the new kernel was running, I installed the ‘kernel-devel’ and ‘kernel-headers’ packages to pull down some necessary headers, then ran this command to create a new initrd without the troublesome dmraid drivers:
mkinitrd --omit-dmraid /boot/NO_DMRAID_initrd-2.6.18-8.1.6.el5.img 2.6.18-8.1.6.el5
Then, simply change /etc/grub.conf to create an option that pointed to my new initrd. My /etc/grub.conf looks like this:
default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu ## My new non-dmraid boot option title CentOS (2.6.18-8.1.6.el5) WITHOUT DMRAID GARBAGE root (hd0,0) kernel /vmlinuz-2.6.18-8.1.6.el5 ro root=/dev/hda1 initrd /NO_DMRAID_initrd-2.6.18-8.1.6.el5.img ## The regular option title CentOS (2.6.18-8.1.6.el5) root (hd0,0) kernel /vmlinuz-2.6.18-8.1.6.el5 ro root=/dev/hda1 initrd /initrd-2.6.18-8.1.6.el5.img ## My working backup option: title CentOS (2.6.18-8.el5) root (hd0,0) kernel /vmlinuz-2.6.18-8.el5 ro root=/dev/hda1 initrd /initrd-2.6.18-8.el5.img
Now, I just rebooted off the first option, and it didn’t load all of the dmraid junk. I can now access the partitions without the ‘resource busy’ problem, and create a software raid array like I’m used to.
Thank you very much for this information. I have an old dual P3 box with two Promise SATA cards. After installing Fedora 7, I would get that device is busy message for /dev/sda1 every time I tried to create a raid 5 array. I removed dmraid using yum remove dmraid, rebooted and then it allowed me to create my raid.
Now I can take a breather.
Thanks again,
Stile
So “dmraid” is for BIOS FAKE raid and “mdadm” is for software RAID. Have I got it right?
Felipe,
I guess that is one way to put it. As I understand it the dmraid utility is for managing Raid devices that have some kind of BIOS assistance, such as the low-end Promise FastTrak and Nvidia NVRaid controllers that are commonly built onto motherboards.
The mdadm utility is used for managing software raid devices that are controlled entirely through the Linux operating system
I have the following card: Promise FastTrack 4310. It is a 4-port 3Gb/s PCI raid controller card I bought at my local store.
Is it FAKE raid?
http://www.promise.com/product/product_detail_eng.asp?segment=undefined&product_id=165
> Felipe
It looks to me like that card would be a ‘fake’ raid controller. From that page:
‘The FastTrak TX4310 leverages Promise’s proven RAID 0/1/5/10 software engine for fast….,’
Which seems to indicated that the RAID functionality is implemented in software instead of in the hardware as a true raid card.
Yes, the Promise FastTrack 4310 is a FRAID. I installed it and tried it with both the CentOS 5.2 and Fedora 9. Thanks Brandon for pointing out the different usage for dmraid and mdadm. I had returned the Fastrack 4310, but I will buy it again and try it out with the dmraid.
Also, in my trial of doing the software raid 1 with CentOS 5.2, I found that if you detach the first sata drive in the mirror, then the system does not boot. I tried plugging second drive (originally pulugged into SATA port 2 Master; otherwise known as SATA port 3) into the first sata port (SATA port 1 Master).
I had tested the Nvidia built in RAID on the motherboard (also FRAID); all builtin motherboard RAIDs are FRAID from what I understand. My test showed that when using the FRAID (Fake RAID vs the SRAID (Software RAID), then regardless which drive is disconnected in the mirror, the system still boots.
So, if I understand, I should be able to make use of dmraid for both the Promise FRAID or the Nvidia FRAID?
I will try both FRAIDs with the dmraid if this is the case.
What comment do you have regarding the bootability of the system with RAID 1 implementation using Software RAID of the system?
Mr A – With software raid, I typically configure a 100 MB or so /boot partition as RAID 1, so that you can boot off any of the disks in the event of a failure.
Hi there, on RHEL5 2.6.18-164.el5, the –omit-dmraid was replaced by –without-dmraid.
great article, helped a bit!
🙂
Thanks a ton. 2.5 years later and it’s still valuable information.
Hi. I have followed the instructions, including riche’s tip, but problem persist. Some advice?
How can i probe that dmraid is down?
Thanks.
Hi Brandon,
Kudos and massive respect goes to you, after around 3 hours of googling and 2 hours of f’ing around with dmraid being enabled on a new server with “fakeraid” intel card. You are an absolute lifesaver.
Thankyou Thankyou Thankyou!
Cheers,
Mitch
http://www.TheMarketingDeviant.com
Hello;
I couldnot get the drive hot, getting the same error as above, “device or resource busy”.
I tried to use
mkinitrd –without-dmraid /boot/NO_DMRAID_initrd-2.6.18-164.e15.img 2.6.18-164.e15
but I got the error No modules available for kernel “2.6.18-164.e15”.
Any idea how to work arround this?
Thanks
Another potential cause of this problem is the multipath daemon.
http://polibyte.com/blog/multipath_blocking_mdraid