The time has finally come. Linux guys, please come help me set up my RAID5 fileserver

[SiM]

I know RAID is not a substitute for backup... but until bluray burners and blanks fall in price, I will have to risk it!
I don't want to buy another 3TB of drives for backup!

This partition thing is driving me mad! So now I don't want any of the OS on the RAID partition. How can I do that while maximising the RAID partition size? I don't mind not using a GUI. Webmin seems like it will do the job for setting up Samba and everything else.

[peterb]

OK - use one (smallish) drive for /boot / and /var and swap (if you want a separate partition for that. Then use the remaining drives with one partition on each and raid those for /home (or /filestore - however you are going to do that.

Or overlay Logical volume management over the RAID array and allocate virtual partitions as you like (you can include the sap file on that - but you will still need a small (100Mb) partition for /boot.

LVM gives you lots of flexibility for dynamically altering partition sizes.

or - creat a 100Mb partition on each drive and RAID that as RAID 1 for /boot. Create a 1Gb partition on each drive - RAID 5 for the swap file.

a 25 GB partition on each drive for / and the remainder of each drive used as one big partition for /home or /filestore - whatever you are going to all it. Both those will be RAID 5.

Or set aside 10 Gb on each drive as an logical volume and use that for / and swap file (non RAID) and use the remainder of each disk for RAID 5. (I think you said ypou are using 3 drives - if using more, allocate a bit less from each drive) but that is adding to the complexity of setting up - and is a path you might not want to go down.

[SiM]

You have confused me a bit there... so is it ok as I have it if I use LVM?

I'll google around to learn about LVM

Thanks for being patient with me lol

[peterb]

Sorry - I was editing as you posted...

Yes, you can overlay LVM on top of a RAID system (but you still need a basic partition for /boot so you can load the drivers to read the array/logical volumes.)

It is an added layer of complexity though so for he sake of simplicity you might want to sacrifice a little bit of storage to avoid it. (But I am fairly conservative in adopting these things - it took a while before I convinced myself to go with LVM - and I haven'tused mdam in anger - only experimental - in fact I am so conservative that I'm only now thinking about changing the windows computer from Win2K to XP!)

[SiM]

OK, I think you are correct, the added layer of complexity is probably not worth it (for me!)... So I will reinstall and sacrifice 10GB on each drive. Partition planing all over again... yay

Drive 1:
128mb /boot
2gb swap
~7gb /var

Drive 2:
10gb /

Drive 3
10gb /home

Does that look ok?

It looks like I need to reinstall anyway. After the array rebuilt (took all night), the OS doesn't want to load now... its stops on Verifying DMI Pool Data

[peterb]

do you mean 10G for /home? If you are going to create a separate branch (say /share) for the shared filestore, 10Gb may be a bit much (you as user (rather than as admin) will be the only one with anything there. Of course you can have /home/share on a separte partition anyway - but in that case /home doesnb't need to be 10 Gb.

That might be a bit confusing... cast your mind back to what I said about the file system. /home could contain data (say your user name is sim - there would be a directory /hom/sim. You could create a share filestore which is mounted n /home - so it appears as /home/share - but in reality the share directory is a separate partition or raid drive - say md0 - defined in /etc/fstab)

I know this is turning into a bit of a saga - but deciding the partition layout is pretty critical at the outset - not because the system won't work, but in order to make optimum use of the drive space, while retaining sufficient flexibility if your needs change.

If you don't use LVM (and I think I'd agree with your last comment) you can resize partitions with Gparted - with one proviso - I don't think Gparted supports the xfs file system - but there may be other xfs tools available tha do allow partition resizing. (Actually, to be accurate, you would resize the file system so it didn't occupy the whole partition, the resize the partition to fit the filesystem size, then add a filesystem (which most people inaccurately describe as 'formatting the disk' to the newly unallocated space. Its actually easier to do than to explain - but not something to undertake lightly - messing with partition structures always contains an element oif risk - the primary partition table must be the most critical 64 bytes on a disk - foloowed closely by the extended partition tables )

[SiM]

Oh I see... I can down size the /home to 5gb (or less?), but what else shall I put on the remaining space on the drive? /usr? or /opt? or shall I leave it unpartitioned?

Ideally I wouldn't want to mess with partitions once it is set up.

In a completely different/random idea, would it be better if I don't do RAID, just keep the 3x1TB drives separate AND then put my other 1TB in an external enclosure for backing up the most prized 1TB of data. It sounds like that would be a lot easier for partitioning too, and the RAID risk (the risk that I will screw up the partition) is gone too...... but that would be less fun

[peterb]

/home can be anysize or on anydrive you like. It is both a directory and can be a mount point for say share - so you could have in /home the following directories sim, user1, sims_wife, sims_girlfriend share. They would all appear under home with the full paths as /home/sim /home/user1 /home/sims_wife /home/sims_girlfriend /home/share. who knows what drives each directory is on - who cares? Only you as admin. In that scenario /home/share is a mountpoint - the actual files that belong to share would be on a different drive. If that drive was unmounted /home/share would still exist - but there would be nothing in that directory (the files would still exist of course and would re-appear when the drive/filesystem was re-mounted)
How is that drive allocated?

In /etc/fstab which will contain entries like

/boot /dev/hda1
/ /dev/hda2
swap /dev/hda3
/var /dev/hda4
/home /dev/hda5
/home/share /dev/hdb1

(the entries are more complex than that - look at yours) but in this simplified case the first column is the mount point - the second is where they are mounted.. so

drive hda contains 5 partitions, drive hdb contains one, and is mounted as /home/share at boot time. home/share exists in the root structure all the time - as does /boot - but they are mount points for the relavant drive/partition. the / mount point contains all the subsequent ones - and the files, but we know from this that /var is located on another partition - the user doesn't though - and doesn't need to - it is just a directory. (And referring back to iscsi - that drive could in theory be anyware in the world, installed in a computer, with access over the internet - it would be slow but it would work or on any computer on your newtwork.)

It sounds complicated - and as I said earlier, it can be a difficult concept to grasp - but once the concept 'clicks' it is wonderfully elegant!

I do urge you to invest in one of the books I recommended earlier - it will save significantly reduce your learning curve and help you get the best out of an initially complex but very powerful and flexible operating system!

I agree with you about RAID - of course you could just set up a minimal system now and add the raid later once you are familiar with the system - you don't have to partition all of the drive(s) or use all the partitions at this stage. Try a conventional set up, with small file store and play with that, then increase the capability and facilities later.

[SiM]

/home can be anysize or on anydrive you like. It is both a directory and can be a mount point for say share - so you could have in /home the following directories sim, user1, sims_wife, sims_girlfriend share. They would all appear under home with the full paths as /home/sim /home/user1 /home/sims_wife /home/sims_girlfriend /home/share. who knows what drives each directory is on - who cares? Only you as admin. In that scenario /home/share is a mountpoint - the actual files that belong to share would be on a different drive. If that drive was unmounted /home/share would still exist - but there would be nothing in it.

How is that drive allocated?

In /etc/fstab which will contain entries like

/boot /dev/hda1
/ /dev/hda2
swap /dev/hda3
/var /dev/hda4
/home /dev/hda5
/home/share /dev/hdb1

(the entries are more complex than that - look at yours) but in this simplified case

drive hda contains 5 partitions, drive hdb contains one, and is mounted as /home/share at boot time. home/share exists in the root structure all the time - as does /boot - but they are mount points for the relavant drive/partition.

It sounds complicated - and as I said earlier, it can be a difficult concept to grasp - but once the concept 'clicks' it is wonderfully elegant!

I do urge you to invest in one of the books I recommended earlier - it will save significantly reduce your learning curve and help you get the best out of an initially complex but very powerful and flexible operating system!

I understand that all. But doesn't there need to be some space in /home for some settings or something? Or sometimes might an application put something into the /home?
I will downsize /home to 2GB then. But what other partitions can I put into the free space.

I would buy a book, the only reason why I am hesitating is that they don't cover servers as well (one of the Amazon reviews said that). There are a couple of free ebooks (including Ubuntu Unleashed)... I will give them a read first...

[peterb]

/home is only for user data - an application won't write or install anything there. (only exception might be a mailserver) but /home and the user directory is created automatically when you creat a user account. By default the user's home directory is /home/user_name

Don't get too hung about a server. A server is only something that is running server applications - in your case this will be Samba. I run Fedora - but I installed it as a desktop machine and then installed the server applications I needed on top of that. It sits there happily serving web pages to the net, acting as a mail transfer agent (postfix) for the computers on the internal network, back-up mta for an externally hosted domain mailserver, a POP£ and IMPA mail server, and an internal Samba file server with private and open filestores. It probably isn't optimally configured, but it works - and it isn't a particularly high end machine. Oh yes, it folds as well (but quite slowly!) And sometimes I fire up opera or firfox and do a bit of surfing. When I'm away from home I ssh into it and usethe CLI.

I say that not to show how clever I am (I'm not) but to show that it is a wonderfully versatile OS - and how the terms server and client can be confused. In some roles it is a server - in others it is a client. Underneath it is the same operating system. My knowledge is largely self taught over about 4 years - with the help of some of the books I mentioned, and from people here like Directhex, Ikonia and Danceswithunix - and others.

As for books - If all else fails - go into a good Waterstones or similar and do a bit of book browsing. (Not the one at Trafalgar Square - their computer section is abysmal!)

[SiM]

OK after a couple of hours reading, I am getting the hang of it quite well...

I have it installed again, without the raid array, to play around with... managed to put it all on 3x5gb... since I am not installing a gui that should be more than enough...

Webmin is really good... If you haven't looked at it Peterb, give it a go! So far I have only used it for remote command shell and to make the IP static (was easier than doing it in the gui!!) but I can use it to setup the raid partitions too. There are loads of pages/settings. Its got all the stuff for web/mail server management too... Its brilliant!

sudo halt

[aidanjt]

If you're using a fileserver, the best (and simpilest) way to deal with storage is to use md to create a smallish (~6GB) root partition to boot off (which must be raid1, bootloaders can't deal with raid0), another md partition for storage with the RAID personality of your choice, and use Linux Volume Management (lvm) on top of that to create partitions for your various mount points.

I've used various methods for allocating storage, and this has been the most hassle free method for me in both the short term setup, and long term managment.

Using my setup as an e.g. you can get an idea of what goes where:
/dev/sda1 swap
/dev/sda2 md0
/dev/sda3 md1
/dev/sda4 md2
/dev/sdb1 swap
/dev/sdb2 md0
/dev/sdb3 md1
/dev/sdb4 md2
/dev/sdc1 swap
/dev/sdc2 md1
/dev/sdc3 md3
/dev/sdd1 swap
/dev/sdd2 md1
/dev/sdd3 md3
/dev/sde1 swap
/dev/sde2 md1
/dev/sde3 md3

Each swap partition is 1GB in size, which combined makes 5GB of swapspace, each swap partition can be added and removed on-the-fly which is why I didn't use it on top of an md partition. md0 is my root partition in raid1 for my OS and programs. md1 is in raid0 and I use that for temporary state data that only needs fast I/O and anything lost there is either replaceable or expendable as far as the OS and myself is concerned. md2 is in raid1 and provides 'safe' storage of system and network data without much concern for raw performance. And finally md3 in raid5 performs secure and reasonably fast file I/O, pretty much all my stuff gets dumped here, it hosts /home, /srv, and /media/files.

Now here's where things get really interesting, using LVM I created a single volume group with md1, md2, and md3 bound into a single volume group (creatively called 'vgsys', aren't I clever?), when I create volumes, I can pick at creation time as to which volume or volumes to layer that logical volume on top of, and better still, if I decide I put that partition somewhere stupid, I can move it onto another physical volume. And of course, md raid volumes can be grown in size, lvm partitions can be grown, shrunk, and moved around physical volumes, physical volumes can be added and removed at will.

As for reading material, I can't say I've ever had a book that's better than man, info, and google.

If you want something clickity clickity then ClarkConnect, OpenFiler, FreeNAS, or some other 'embedded' type NAS software will probably be less taxing on your head.

[peterb]

Aidnjt gives the setup - far more concisely than I did!

I must say I did find a book invluable!

Tried Webmin, but I found that once I got the hang of the commands using a CLI eitrher locally or over SSH wasn't a lot more hassle.- - but if it works for, thats good (lots of ways of cooking stuff in Linux!

[aidanjt]

Same here, I tried webmin many times over the years, and I can't say it's solved many more problems than it created. The problem with webmin is it tries to deal with hundreds of configuration syntaxes at the same time in their fullest, and it doesn't execute any of them rather well, I also felt it was far too slow and error prone for what it should do.

[SiM]

Cheers guys, I will read up on LVM... Although, I really just want one big ass raid partition for the /home/share...

I haven't really used webmin for many things, but I will bear that in mind and avoid using it if I can...

FreeNAS was my first thought, except Ubuntu is more flexible and reliable... not to mention it gives me an excuse to learn how to use Linux

[SiM]

OK, after much thought, I have decided to forget about RAID and just keep one of the drives as an external backup... This should make OS installation a lot simpler

Is there any reason not to use NTFS? I am thinking of making the shared partitions NTFS so I can plug them into a Windows PC for quick access/copying.