home | list info | list archive | date index | thread index

Re: [OCLUG-Tech] how to recover logical volumes from a damaged volume group?

  • Subject: Re: [OCLUG-Tech] how to recover logical volumes from a damaged volume group?
  • From: "Robert P. J. Day" <rpjday [ at ] crashcourse [ dot ] ca>
  • Date: Sat, 15 Feb 2014 07:19:01 -0500 (EST)
On Fri, 14 Feb 2014, Alex Pilon wrote:

> On Fri, Feb 14, 2014 at 12:23:08PM -0500, Robert P. J. Day wrote:
> >   the problem -- ironically following instructions *i* had written
> > for how to create a bootable SD card, he took a 2G image, and
> > "dd"ed it by accident to /dev/sdb instead of /dev/sdc (being so
> > used to having a single drive system that he unthinkingly chose
> > /dev/sdb as the target as he had always done).
>
> Ergo why you never use dd if you're going to take instructions
> literally, and always double-check the target. Maybe add a hint to
> `lsblk -f` to get an overview of the block devices and filesystems
> on your system, before writing the disk image? For those new to
> Linux, a note on the dynamic block device naming might be useful
> too.

  all good ideas, except i already have such a warning on that page,
and the instructions are for readers ostensibly with enough experience
to be working with embedded linux, so there's only so much
hand-holding i can do. :-P

> >   i asked about this on the fedora user list, and got a couple
> > suggestions regarding running "pvck" and "pvscan" and so on, but
> > i'm not an LVM expert, so i want to make sure i start with
> > utilities that do nothing but *check* the disk, and if i can
> > activate the volume group and LVs on that second disk with nothing
> > but the contents of the backup file, that would be ideal.
>
> pvscan likely just checks the usual location of the LVM metadata.
> http://people.redhat.com/agk/talks/linuxtag_2006/LVM2-LinuxTag2006.html
> says that that's “near the start of every PV”. Not sure if it's
> referring to the old format. If it's not, and the newer format
> doesn't have redundant superblocks, then having overwritten it, I
> doubt `lvm pvscan` could do anything useful.
>
> Two, even if you can restore the metadata for the LVMs, I wouldn't
> count on the filesystem being able to handle the corruption. I
> disclaim being an expert in the matter, but still.
>
> > p.s. there is a backup of the disk, so it's not like this is an
> > urgent operation; it's more like an intellectual challenge.
>
> Maybe you could extract data using sleuthkit if you had the
> patience—try scanning inodes, dumping their blocks and extents, and
> running `file` on them (yes, if there are any forensics experts on
> this list, call me primitive and uneducated), great fun.  Photorec
> probably could extract some data too.
>
> I've been told that had the data been so important, to just bring it
> to a professional, **without touching it yourself**, unless you
> really don't trust the professional and want to take a disk image
> for your own purposes. Never mount the filesystem read-write, and do
> not fsck it unless you know what you are doing.

  well, as i mentioned, the data is in fact on a backup so it's not
like this is a mission-critical salvage operation; it's more like (as
i said) an intellectual challenge.

  and, even from a position of relative ignorance in terms of LVM
forensics and recovery, if all that's happened is the overwriting of
the first 2G of the hard drive, and with the full contents of the
/etc/lvm/backup file for that volume group, it would *seem* that this
shouldn't be that hard.

  the equivalent using regular partitions would be if someone wiped
out the partition table and, say, the first partition -- if someone
handed you *exactly* what the partition table used to look like, it
would be a trivial operation to at least recover the undamaged
partitions, by simply running "fdisk" and manually restoring the
partition table. all i'm looking for is the LVM equivalent of exactly
that.

  and if readers aren't sure what i'm talking about, if you're running
LVM, go into /etc/lvm/backup as root and look at the text file
corresponding to your volume group. that looks like a pretty detailed
inventory of your volume group, its LVs and their relationship to the
enclosing physical volume. i can't believe that's not enough info to
recover an undamaged logical volume.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================