Mtv Europe (![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small} mtve) wrote,@ 2005-03-15 00:34:00 |
|
400GB software RAID-5 in under $350.
1) shopping
Not Maxtor (see diagram 5). 400GB disks are too expensive per byte. So Seagate or Western Digital. USN shop.
| Item | Price | Q-ty | Cost |
| WD2000JB-00GVA0 | 107.00 | 3 | 321.00 |
| MR-10KPR rack | 6.48 | 3 | 19.44 |
| IDE cable | 1.00 | 2 | 2.00 |
|
Total: |
342.44 | ||
2) installation
3) partitioning (for each ad1, ad2, ad3)
fdisk -BI ad1 ignore "invalid mbr"
disklabel -wB ad1s1 auto
disklabel -e ad1s1 should look like this:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
c: 390721905 0 unused 0 0 # (Cyl. 0 - 387620*)
h: 390721905 0 vinum # (Cyl. 0 - 387620*)
4) Vinum config:
drive a device /dev/ad1s1h
drive b device /dev/ad2s1h
drive c device /dev/ad3s1h
volume raid5
plex org raid5 512k
sd length 0 drive a
sd length 0 drive b
sd length 0 drive c
vinum create vinum.conf vinum init raid5.p0 wait for "up", kick it again if it get stuck echo 'start_vinum="YES"' >>/etc/rc.confnewfs /dev/vinum/raid5 tunefs -n enable /dev/vinum/raid5 echo '/dev/vinum/raid5 /.1 ufs rw,nodev,nosuid,userquota,groupquota 2 2' >>/etc/fstab mount -a quotaon -a quotacheck /.1
5) copying hundreds of CDs and DVDs - TBD, duh.
6) the funny part - testing. power off ad3, master ad2 also goes off:
Mar 14 21:44:38 heart /kernel: ad2s1h: hard error reading fsbn 563967633 of 281983785-281983912 (ad2s1 bn 563967633; cn 559491 tn 11 sn 12) trying PIO mode
Mar 14 21:44:38 heart /kernel: ad2: DMA problem fallback to PIO mode
Mar 14 21:44:38 heart /kernel: ad2: read interrupt arrived earlyad2: read error detected (too) late
Mar 14 21:44:38 heart /kernel: vinum: raid5.p0.s1 is crashed by force
Mar 14 21:44:38 heart /kernel: vinum: raid5.p0 is degraded
Mar 14 21:44:48 heart /kernel: ad3: READ command timeout tag=0 serv=0 - resetting
Mar 14 21:44:48 heart /kernel: ata1: resetting devices ..
Mar 14 21:44:48 heart /kernel: ad2: removed from configuration
Mar 14 21:44:48 heart /kernel: ad3: removed from configuration
Mar 14 21:44:48 heart /kernel: vinum: raid5.p0.s2 is crashed by force
...
Mar 14 21:44:49 heart /kernel: vinum: raid5.p0.s2 is stale by force
Mar 14 21:44:49 heart /kernel: vinum: raid5.p0 is faulty
Mar 14 21:44:49 heart /kernel: vinum: raid5 is down
...
system hangs here (well, duh, it's ata after all or may be it's too hard for vinum), after reboot
# vinum ls raid5.p0.s0
S raid5.p0.s0 State: R 0% PO: 0 B Size: 186 GB
*** Start raid5.p0.s0 with 'start' command ***
# vinum start -w raid5.p0.s0
Can't start raid5.p0.s0: Input/output error (5) no idea what it means
# vinum ls raid5.p0.s0
S raid5.p0.s0 State: R 0% PO: 0 B Size: 186 GB
*** Revive process for raid5.p0.s0 has died ***
# vinum start -w raid5.p0.s0
(hangs forever here)
(from another shell)
# ps -ax|grep vinum
15 ?? DLs 0:00.00 vinum: vinum daemon (vinum)
206 p0 D+ 0:00.01 vinum start -w raid5.p0.s0
207 p0 Z+ 0:00.00 (vinum) f*cking great
# vinum ls raid5.p0.s0
S raid5.p0.s0 State: R 0% PO: 0 B Size: 186 GB
*** Revive of raid5.p0.s0 has stalled ***
# vinum setstate up raid5.p0.s0 ok now
# vinum setstate up raid5.p0.s1
# vinum setstate up raid5.p0.s2
# vinum start raid5.p0 well, more or less, it wasn't exactly like this
# vinum rebuildparity -vV raid5.p0
Parity incorrect at offset 0x2ac000
Rebuilding at 2816 kB (0%) Parity incorrect at offset 0x2c8000
...
and here it will another few hours to build it up
Preliminary conclusion - should be different channels.
To be continued.
