The downside of expanding your storage through bigger disks

August 31, 2014

As I mentioned recently, one of the simplest ways of expanding your storage space is simply to replace your current disks with bigger disks and then tell your RAID system, file system, or volume manager to grow into the new space. Assuming that you have some form of redundancy so you can do this on the fly, it's usually the simplest and easiest approach. But it has some potential downsides.

The simplest way to put the downsides is that this capacity expansion is generally blind and not so much inflexible as static. Your storage systems (and thus your groups) get new space in proportion to however much space (or how many disks) they're currently using, and that's it. Unless you already have shared storage, you can't reassign this extra new space from one entity to another because (for example) one group with a lot of space doesn't need more but another group with only a little space used now is expanding a lot.

This is of course perfectly fine if all of your different groups or filesystems or whatever are all going to use the extra space that you've given them, or if you only have one storage system anyways (so all space flowing to it is fine). But in other situations this rigidity in assigning new space may cause you heartburn and make you wish to reshape the storage to a lesser or greater amount.

(One assumption I'm making is that you're going to do basically uniform disk replacement and thus uniform expansion; you're not going to replace only some disks or use different sizes of replacement disks. I make that assumption because mixed disks are as much madness as any other mixed hardware situation.)

Comments on this page:

Another issue worth thinking about is that while you get more storage space, you may not increase I/O bandwidth commensurately: that is, if you have capacity issues both in terms of data at rest and data in flight, simple replacement is not likely to give you much headroom. If you have lots of cold data, this isn't a problem, but with larger disks and a heavy I/O load, you can end up "stranding" storage behind busy I/O.

Written on 31 August 2014.
« How to change your dm-cache write mode on the fly in Linux
We don't believe in DHCP for (our) servers »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Aug 31 00:35:08 2014