LVM: flexible storage
Volumes, resizing, and snapshots.
A raw partition is a fixed slice of one disk. Think of it like a concrete wall between two rooms: once it is poured, moving it means demolition. If your /var partition fills up while the one next to it sits half empty, you cannot shove the boundary over to borrow space. You back up, repartition, and restore, usually with the machine offline. LVM (Logical Volume Manager, the set of tools that pool and slice your storage by programming a layer inside the Linux kernel) replaces that concrete wall with a sliding one.
Here is the mental model. Your disks are barrels of water, all poured into one shared tank. From that tank you draw off buckets of whatever size you like, and you can top up or resize a bucket without ever touching the barrels. LVM works in three layers that do exactly that, and once you see them you can read any server's storage at a glance.
The physical volume (PV) is a raw disk or partition you have handed to LVM. Running pvcreate on a device writes a small label and some metadata to the front of it, marking that device as LVM's to use. The volume group (VG) is the tank: one pool that fuses one or more PVs into a single blob of space. The logical volume (LV) is the bucket you draw out, the flexible partition you put a filesystem on and mount. Because every LV draws from the same pool, you can grow it, shrink it (carefully), move it across physical disks while it is in use, and snapshot it.
See What You Already Have
Before you change anything, know what is there. On a server you inherited, the storage layout is a map of what matters: which volumes hold data, which are half empty, and whether anything exists that should not. An LV you did not create, or a snapshot nobody remembers taking, is worth a hard look. A snapshot can quietly hold a copy of data as it was before someone tampered with it, and it can equally be a place an attacker parked something. Start with lsblk to see the whole block-device tree.
That tells you sda2 is an LVM member and sdb is a spare disk doing nothing. For the LVM-native view, three commands mirror the three layers: pvs lists physical volumes, vgs lists pools, lvs lists volumes. Read them together and you know exactly how much room is free and where.
The pool (data-vg) has one PV and under 9 GiB free. The single volume app is 90 GiB, and its Attr string shows a then o at the fifth and sixth flags (active, then open), which means it is mounted and in use right now. That last fact matters for the next part.
Grow A Volume Without Downtime
When app fills up, you do not repartition. You take space from the pool and stretch the volume into it, then grow the filesystem sitting on top so it can use the new room. For ext4 (the common default Linux filesystem) and XFS (a high-performance filesystem standard on Red Hat systems and many data volumes), that final step happens while the filesystem stays mounted and serving traffic. No downtime. The pool here is nearly empty though, so first you add the spare disk to it.
Now the pool has room. lvextend -L +50G adds 50 gibibytes to the volume; the plus sign means grow by that much (write -L 140G instead to set an exact target). resize2fs then grows the ext4 filesystem to fill the enlarged volume. On XFS you would run xfs_growfs /srv/app against the mountpoint instead, and remember that XFS can only grow, never shrink. You can fold both steps into one with lvextend -r (also spelled --resizefs), which resizes the filesystem for you.
The extent count in that output is LVM's allocation unit at work: extents are the fixed-size chunks (4 MiB each by default) the pool hands out, so 140 GiB is 35840 of them. Confirm the change the way any storage change should be confirmed, by looking at what the filesystem itself reports, not at what you hoped happened.
The Device-Mapper Layer Underneath
LVM does not talk to your disks directly. It programs the kernel's device-mapper (dm), the subsystem that builds virtual block devices which translate reads and writes down to the real hardware. The same machinery encrypts disks for LUKS (Linux Unified Key Setup, the standard for full-disk encryption) through a target called dm-crypt, and it runs LVM's own mirror and RAID (redundant array of independent disks, several drives presented as one) modes. Every LV appears under /dev/mapper as VG-LV, with any dash in a name doubled, which is why the volume app in group data-vg shows up as /dev/mapper/data--vg-app. dmsetup shows the live mapping, including which physical devices a volume now spans.
Read that as: the app device (major 252) is stitched together from sdb (8:16) and sda2 (8:2). One volume, one filesystem, two physical disks, invisible to anything running above it. That is the payoff of the pool. It is also a reminder for defenders that /dev/mapper is part of your attack surface: anyone with root can map a device straight to raw blocks and bypass the filesystem's permissions entirely, so an unexpected entry here deserves the same suspicion as an unexpected process.
Snapshots: A Freeze-Frame Of A Volume
A snapshot is a freeze-frame. It is not a full copy of the volume. A full copy would cost as much time and space as the data itself. Instead, a snapshot keeps a record of the original and stays valid by writing down only what changes after the moment you take it. The trick is copy-on-write (COW): the instant a block on the live volume is about to be overwritten, LVM first copies the old block aside into the snapshot's reserve, then lets the write land. Blocks nobody touches are never copied. Read the snapshot and you get the old block from the reserve if it changed, or straight from the live volume if it did not. It is like photocopying only the ledger pages someone is about to cross out, and leaving the rest in place.
The classic use is a consistent backup. Freeze the volume, back up the still image at your leisure while the live volume keeps changing, then delete the snapshot. You take one with lvcreate -s, giving it a name and a reserve size for the changes.
Origin app now reads owi-a (the o marks it as a snapshot origin), and app-snap points back at it. The Data% column is the one to watch: it is the percent of the 5 GiB reserve consumed so far, climbing as the live volume changes. Mount the snapshot read-only, take your backup from that frozen point, unmount, and drop it the moment you are done.
Snapshots As An Incident-Response Tool
The same freeze-frame is a first move in IR (incident response, the work of investigating and containing a breach). A box is acting strange and you suspect it is compromised. Before evidence changes under your feet, snapshot the volume to capture its exact state, then image that still copy to storage elsewhere while the machine keeps running. A snapshot device is not the size of its reserve: read it and you see the full 140 GiB origin exactly as it looked at the instant you froze it, unchanged blocks served from the origin and changed blocks from the COW store. That is what makes it a clean forensic image. dd (the raw disk-copy tool) reads it block for block, and you hash the result so its integrity can be proven later.
The attacker's angle on all of this is the mirror image of the defender's. If someone knows your backups run off a small snapshot, flooding the origin with writes fills that reserve, invalidates the snapshot, and blinds your backup without touching a single log line. Newer setups use thin provisioning, where snapshots draw from a shared thin pool rather than a fixed reserve and do not invalidate the same way. They carry their own failure mode: if the thin pool as a whole runs dry, every volume on it can flip to read-only at once. The discipline is identical, watch the pool and alert before it fills.
Put lvs in your monitoring. A snapshot whose Data% is creeping toward 100 is a backup about to break or a forensic image about to be lost, and it stays silent until it is too late. Take snapshots on purpose, size them for the writes you expect during their life, and remove them the second the backup or image is captured.
Try this
Work through “Snapshots As An Incident-Response Tool” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.
Takeaway
The trap worth remembering here: a snapshot that fills up is destroyed. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.