btrfs 의 파일시스템이 망가졌나? 복구해보자.

btrfs 를 사용하고 있는데, 아직 익숙하지 않은지, 뭔가 이상한 부분들이 있다.

virtualbox 가상머신을 사용하던중 프리징 되는 현상이 발생했다. 그냥 강제종료를 하니 이제 뜨지 않는다.ㅜㅜ

커널에러가 발생하는 듯 싶다. 커널호환이 안되는 것인가?

다음과 같은 커널에러가 발생한다.

# cat kern.log-btrfs-panic
[  115.018518] ------------[ cut here ]------------
[  115.018540] kernel BUG at /build/buildd/linux-3.13.0/fs/btrfs/ctree.c:3166!
[  115.018563] invalid opcode: 0000 [#1] SMP
[  115.018578] Modules linked in: pci_stub vboxpci(OF) vboxnetadp(OF) vboxnetflt(OF) vboxdrv(OF) snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel dm_multipath scsi_dh aesni_intel aes_x86_64 lrw gf128mul glue_helper snd_hda_intel snd_hda_codec ablk_helper snd_hwdep cryptd rts5139(C) snd_pcm arc4 btusb snd_page_alloc rtl8723ae rtl8723_common rtl_pci rtlwifi snd_seq_midi snd_seq_midi_event mac80211 joydev snd_rawmidi serio_raw snd_seq cfg80211 snd_seq_device lpc_ich nouveau i915 snd_timer ttm mei_me drm_kms_helper mei drm snd i2c_algo_bit soundcore bnep rfcomm bluetooth mxm_wmi msi_wmi sparse_keymap binfmt_misc wmi parport_pc video mac_hid ppdev lp parport hid_generic hid_logitech_dj usbhid hid btrfs libcrc32c raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor psmouse ahci raid6_pq libahci raid1 raid0 alx mdio multipath linear dm_mirror dm_region_hash dm_log
[  115.018939] CPU: 1 PID: 732 Comm: btrfs-endio-wri Tainted: GF        C O 3.13.0-35-generic #62-Ubuntu
[  115.018966] Hardware name: Micro-Star International Co., Ltd. CR42 2M/GE40 2OC/MS-1492, BIOS E1492IMS.30O 01/29/2014
[  115.018996] task: ffff880222c8e000 ti: ffff8800b9a04000 task.ti: ffff8800b9a04000
[  115.019018] RIP: 0010:[<ffffffffa00f1a31>]  [<ffffffffa00f1a31>] btrfs_set_item_key_safe+0x161/0x170 [btrfs]
[  115.019063] RSP: 0018:ffff8800b9a05b70  EFLAGS: 00010246
[  115.019079] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 00000005e87b0000
[  115.019099] RDX: 0000000000000000 RSI: ffff8800b9a05c76 RDI: ffff8800b9a05b8f
[  115.019119] RBP: ffff8800b9a05bc8 R08: 0000000000000001 R09: ffff8800b9a05b90
[  115.019139] R10: 00000005e87b8000 R11: 00000000ffffffff R12: ffff8800b9a05b7e
[  115.019160] R13: ffff88003612a7e0 R14: ffff8800b9a05c76 R15: ffff880114d79680
[  115.019181] FS:  0000000000000000(0000) GS:ffff88022fa80000(0000) knlGS:0000000000000000
[  115.019204] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  115.019221] CR2: 00007f5d0c011088 CR3: 000000018e42b000 CR4: 00000000001427e0
[  115.019242] Stack:
[  115.019249]  ffff880035e61000 0118ffffa012a959 006c000000000000 1800000005e87b00
[  115.019274]  6c00000000000001 00000005e87b0000 ffff880114d79680 00000005e87b0000
[  115.019298]  0000000000000000 0000000000000ef7 ffff88003612a7e0 ffff8800b9a05cc0
[  115.019323] Call Trace:
[  115.019349]  [<ffffffffa0127ae1>] __btrfs_drop_extents+0x421/0xad0 [btrfs]
[  115.019374]  [<ffffffff810aafe2>] ? autoremove_wake_function+0x12/0x40
[  115.019406]  [<ffffffffa0128d10>] btrfs_drop_extents+0x60/0x90 [btrfs]
[  115.019435]  [<ffffffffa01188cc>] insert_reserved_file_extent.constprop.53+0x6c/0x290 [btrfs]
[  115.019467]  [<ffffffffa011e375>] btrfs_finish_ordered_io+0x2e5/0x570 [btrfs]
[  115.019495]  [<ffffffffa011e885>] finish_ordered_fn+0x15/0x20 [btrfs]
[  115.019523]  [<ffffffffa014219a>] worker_loop+0x15a/0x5c0 [btrfs]
[  115.019550]  [<ffffffffa0142040>] ? btrfs_queue_worker+0x310/0x310 [btrfs]
[  115.019572]  [<ffffffff8108b4a2>] kthread+0xd2/0xf0
[  115.019588]  [<ffffffff8108b3d0>] ? kthread_create_on_node+0x1c0/0x1c0
[  115.019608]  [<ffffffff8172ecbc>] ret_from_fork+0x7c/0xb0
[  115.019625]  [<ffffffff8108b3d0>] ? kthread_create_on_node+0x1c0/0x1c0
[  115.019644] Code: 48 8b 45 bf 48 8d 7d c7 4c 89 f6 48 89 45 d0 0f b6 45 be 88 45 cf 48 8b 45 b6 48 89 45 c7 e8 97 f2 ff ff 85 c0 0f 8f 48 ff ff ff <0f> 0b 0f 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55
[  115.019750] RIP  [<ffffffffa00f1a31>] btrfs_set_item_key_safe+0x161/0x170 [btrfs]
[  115.019779]  RSP <ffff8800b9a05b70>
[  115.029262] ---[ end trace 642f12bbe4eff1cc ]---

일단 마운트 해제하고, btrfsck 를 해본다.

# btrfsck --repair /dev/vg_data/lv_data
enabling repair mode
Checking filesystem on /dev/vg_data/lv_data
UUID: c5cd286a-980e-414e-b10a-aacbf5c5e449
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
root 5 inode 280 errors 180, file extent overlap, file extent discount
found 18378234624 bytes used err is 1
total csum bytes: 58871948
total tree bytes: 193949696
total fs tree bytes: 48832512
total extent tree bytes: 58753024
btree space waste bytes: 45925595
file data blocks allocated: 3653029335040
 referenced 52821159936
Btrfs v3.12

뭔가 고치긴 한것 같다.

그런데, virtualbox 를 다시 실행해도 같은 에러가 발생한다.

커널버전을 낮춰볼까?

다시 체크를 해보니

# btrfsck /dev/vg_data/lv_data
Checking filesystem on /dev/vg_data/lv_data
UUID: c5cd286a-980e-414e-b10a-aacbf5c5e449
checking extents
checking free space cache
free space inode generation (0) did not match free space cache generation (57150)
free space inode generation (0) did not match free space cache generation (57150)
free space inode generation (0) did not match free space cache generation (57151)
free space inode generation (0) did not match free space cache generation (57150)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57134)
free space inode generation (0) did not match free space cache generation (56404)
free space inode generation (0) did not match free space cache generation (57135)
free space inode generation (0) did not match free space cache generation (26053)
free space inode generation (0) did not match free space cache generation (56713)
free space inode generation (0) did not match free space cache generation (57137)
free space inode generation (0) did not match free space cache generation (57139)
free space inode generation (0) did not match free space cache generation (57148)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (56684)
free space inode generation (0) did not match free space cache generation (57124)
free space inode generation (0) did not match free space cache generation (56680)
free space inode generation (0) did not match free space cache generation (56646)
free space inode generation (0) did not match free space cache generation (57050)
free space inode generation (0) did not match free space cache generation (56697)
free space inode generation (0) did not match free space cache generation (56681)
free space inode generation (0) did not match free space cache generation (57078)
free space inode generation (0) did not match free space cache generation (57087)
free space inode generation (0) did not match free space cache generation (57118)
free space inode generation (0) did not match free space cache generation (57078)
free space inode generation (0) did not match free space cache generation (57107)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (57140)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57083)
free space inode generation (0) did not match free space cache generation (52539)
free space inode generation (0) did not match free space cache generation (56699)
free space inode generation (0) did not match free space cache generation (57141)
free space inode generation (0) did not match free space cache generation (57088)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (57055)
free space inode generation (0) did not match free space cache generation (57087)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (56527)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (47463)
free space inode generation (0) did not match free space cache generation (57081)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (57097)
free space inode generation (0) did not match free space cache generation (57056)
free space inode generation (0) did not match free space cache generation (57141)
free space inode generation (0) did not match free space cache generation (57150)
free space inode generation (0) did not match free space cache generation (57150)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57070)
free space inode generation (0) did not match free space cache generation (57090)
free space inode generation (0) did not match free space cache generation (57148)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57122)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57118)
free space inode generation (0) did not match free space cache generation (57147)
free space inode generation (0) did not match free space cache generation (57097)
checking fs roots
root 5 inode 280 errors 180, file extent overlap, file extent discount
found 45291307607 bytes used err is 1
total csum bytes: 58871948
total tree bytes: 193966080
total fs tree bytes: 48832512
total extent tree bytes: 58769408
btree space waste bytes: 45951279
file data blocks allocated: 3653012492288
 referenced 52804317184
Btrfs v3.12

이런 엄청난 메시지가..

btrfsck 로는 해결이 안되는 것 같다.

그냥 btrfs 를 버리고 ext4 로 해야겠다.
btrfs 의 데이터를 다른곳으로 옮기고, mkfs.ext4 로 다시 생성, 다시 복사. 끝.

ext4 로 바꾸고 다시 해보니 일단은 정상적인 듯 하다.

btrfs 는 문제해결능력을 더 키운후에 다시 도전해야 겠다.

data 는 그냥 바꾸면 되는데, root 는 어떻게 할지 고민해봐야겠다.
btrfs => ext4 로 바로 바꿔주는 방법을 찾아야 겠다.