The usual IT babble
Posts tagged EXT3
MySQL: Beware of sync_binlog on EXT3
Feb 23rd
Well, I just glazed again over my my.cnf for our web-cluster because I just moved a database from one cluster to another and getting quite different performance from it. So, as I expected, there is a slight difference between both configuration files:
@@ -55,8 +58,6 @@
innodb_log_group_home_dir = /var/lib/mysql/db
innodb_log_file_size = 512M
innodb_thread_concurrency = 8
-sync_binlog = 1
And apparently, according to the MySQL Performance Blog that’s really, really bad (as well, we’re currently running without write caching, as the battery module of the storage is dead).
Linux: Getting information about an EXT3 filesystem
Feb 13th
You know, I’m not getting any younger. It’s getting harder remembering every damn command … so here is how you get information out of your EXT3 filesystem:
sles10sp2 ~ [0] > tune2fs -l /dev/sda2 | grep "^Filesystem"
Filesystem volume name:
Filesystem UUID: 8eec8235-4d9e-4b58-acf9-3c68c977d5ea
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal resize_inode filetype
needs_recovery sparse_super large_file
Filesystem state: clean
Filesystem OS type: Linux
Filesystem created: Tue May 27 10:48:56 2008
MD (Multiple Devices) weirdness
Dec 29th
Well, I don’t think my problem has anything to do with the DawiControl card anymore. I did a little experiment today. I created a 1TiB EXT3 file system on a single drive (one of the new 1TiB drives obviously) and started syncing data over to it (roughly 800MiB).
Now, then I unmounted the drive(s), ran fsck -C -f /dev/sd${deviceletter}1 and it went through without any trouble. Then I removed the partition and created a 1GiB partition on each drive, which I then used to build a new device mapper RAID5 array (with EXT3 on top …).
And guess what happened after I copied the data over, unmounted the file system and ran fsck ? Sure, same thing as yesterday. Now, this means either it’s a mdadm bug, while creating the array or really MD’s fault (which I can rule out, since the same happens on 2.6.25 as well as on 2.6.28) … *shrug*
SIL 3114 barfing
Dec 28th
Well, after I had so much trouble with the USB converter (which isn’t really suited for Linux), I went ahead and bought a DawiControl DC-154 (which is using a SIL3114) controller to migrate my stuff.
After fucking up the new RAID array with the 1TB disks on the old controller (luckily I had the old hard disks still lying around, which still contained the RAID array), I plugged the 1TB disks onto the new controller and started building the array. So after 760 minutes (that’s nearly 13 hours) of synchronizing the newly created array, I was finally able to create the file system — that should be without trouble, right ?
Well, yeah … it was … So I started putting the data on the newly created array (using rsync). Only problem: something seems to be corrupting data (as in EXT3 is barfing up a lot of file system errors).
Dec 28 08:47:21 epimetheus [67092.652866] EXT3-fs: mounted filesystem with ordered data mode.
Dec 28 09:53:20 epimetheus [71058.253027] EXT3-fs error (device md2): ext3_add_entry: bad entry
in directory #23371810: directory entry across blocks - offset=260, inode=18964552,
rec_len=26988, name_len=115
Dec 28 09:53:20 epimetheus [71058.305558] EXT3-fs error (device md2): ext3_add_entry: bad entry
in directory #23371810: directory entry across blocks - offset=260, inode=18964552,
rec_len=26988, name_len=115
(fsck.ext3 is returning much, much more ..)
After putting the blame on EXT3, I tried out reiserfs (yeah, yeah I know .. baaaad idea). Well, at first it didn’t put out any errors, but running fsck.reiserfs turned up errors that looked a lot like the ones fsck.ext3 returned.
Then, I started looking at the array size (since I was curious), and it said the new array on four 1TB disks is ~760GB. Now according to my improper math, using 4* 1000GB drives the total usable amout of disk space should be something like 2793.96GB, and not ~760GB. *shrug*
I’m out of idea’s right now, and I’m gonna wait till January till I do anything else.
USB weirdness
Dec 23rd
Well, I was at work for a brief moment, where I grabbed me one of our SATA->USB bridges, since I need to migrate some (~750GB) data of the old raid-array and onto a new one. The troublesome about that is simply, that the current RAID controller only supports four attached devices, that’s why I do have to use something like this … Sure I could have bought a new RAID controller, but why spend 45+ EUR on something, that you can solve differently ?
Well, after figuring that I need to change my kernel config yet again (didn’t have USB support till Tue Dec 23 ~16:45:00 CET 2008) I attached the adapter to two adjacent USB ports. And shortly after copying 4-10MB, the transfer would result in a read-only EXT3 file system with something like this in the syslog:
Dec 23 17:23:57 epimetheus usb 1-2: new high speed USB device using ehci_hcd and address 2
Dec 23 17:23:57 epimetheus usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
Dec 23 17:23:57 epimetheus usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Dec 23 17:23:57 epimetheus usb usb3: Product: OHCI Host Controller
Dec 23 17:23:57 epimetheus usb usb3: Manufacturer: Linux 2.6.25-hardened-r11 ohci_hcd
Dec 23 17:23:57 epimetheus usb usb3: SerialNumber: 0000:00:02.1
Dec 23 17:23:57 epimetheus Initializing USB Mass Storage driver...
Dec 23 17:23:57 epimetheus usb 1-2: reset high speed USB device using ehci_hcd and address 2
Dec 23 17:23:57 epimetheus usb 1-2: device descriptor read/64, error -71
Dec 23 17:23:58 epimetheus usb 1-2: reset high speed USB device using ehci_hcd and address 2
Dec 23 17:23:58 epimetheus usb 1-2: device descriptor read/64, error -71
Dec 23 17:23:58 epimetheus usb 1-2: reset high speed USB device using ehci_hcd and address 2
Dec 23 17:23:58 epimetheus usb 1-2: device not accepting address 2, error -71
Dec 23 17:23:59 epimetheus usb 1-2: reset high speed USB device using ehci_hcd and address 2
Dec 23 17:23:59 epimetheus usb 1-2: device not accepting address 2, error -71
Dec 23 17:23:59 epimetheus usb 1-2: USB disconnect, address 2
Dec 23 17:23:59 epimetheus sd 4:0:0:0: Device offlined - not ready after error recovery
Dec 23 17:23:59 epimetheus sd 4:0:0:0: [sde] Result: hostbyte=0x01 driverbyte=0x00
Dec 23 17:23:59 epimetheus end_request: I/O error, dev sde, sector 1229380295
Dec 23 17:23:59 epimetheus end_request: I/O error, dev sde, sector 1229380535
Dec 23 17:23:59 epimetheus Aborting journal on device sde1.
Dec 23 17:23:59 epimetheus ext3_abort called.
Dec 23 17:23:59 epimetheus EXT3-fs error (device sde1): ext3_journal_start_sb: Detected aborted journal
Dec 23 17:23:59 epimetheus Remounting filesystem read-only
Dec 23 17:23:59 epimetheus EXT3-fs error (device sde1) in ext3_ordered_writepage: IO failure
Dec 23 17:23:59 epimetheus __journal_remove_journal_head: freeing b_frozen_data
Dec 23 17:23:59 epimetheus __journal_remove_journal_head: freeing b_frozen_data
Dec 23 17:23:59 epimetheus __journal_remove_journal_head: freeing b_frozen_data
Dec 23 17:23:59 epimetheus __journal_remove_journal_head: freeing b_frozen_data
Dec 23 17:23:59 epimetheus __journal_remove_journal_head: freeing b_committed_data
Well, now what ? I googled a for a bit, apparently this happens when EHCI tries to write to the device and gets a timeout, cause the device is rather slow — or whatever (or the device drops down to USB 1.1). So, after disabling EHCI, the transfer has been running for about three hours now, and roughly only 1/12 of the data transferred to the external disk. Only trouble with that is, that even USB 1.1 is kinda slow to transfer 750GiB 
Followup: Well, due to USB 1.1 being slow as a snail, I went surfing for alternatives using Windows (since I know that the bridge does full USB 2.0 with Windows without any troubles). And guess what I found ?
There’s an EXT2/3 device driver for Windows XP, yay! So I’m copying with full 100Mbit speed right now *shrug*