Skip to main content

A Shack in the Woods, Crumbling at the Core

Imagine a run down shack. The kind I’m thinking of is like the cabin my grandfather bought in the late 60’s. It was about 16’x20’, built in the 30’s and then left to rot. It had wooden blocks for its foundation, it sagged about a foot in one corner as the wood blocks shifted and rotted, it was musty, moldy and damp. It was the very definition of deferred maintenance. A menagerie of non-human mobile life forms called it home. It was a shack, kind of like the one pictured here.
There are applications like that, lots of them. We probably all know where they are. Most of us will only have to look someplace in our data centers. Think of an old shack when you think of those applications.

What do you do with a shack like that? You could paint it, put new doors and windows in it, add on a bedroom and a porch, maybe even add plumbing and a bathroom. You could even secure it (from the rain) by fixing the roof. But what about the foundation? Sure you can remodel and add on to a house with a rotten foundation, but to do so is to negate much of the value of the other improvements. Your bathroom will have a sagged floor, your doors and windows won’t fit right, and your shiny new front porch will detach from the shack as the shack rots itself into the ground. You’ll have a maintenance headache that doesn’t really do what it should.

The obvious first step is to jack up the shack and build a concrete foundation. If you bite the bullet and fix the foundation first, the rest of the improvements can be built on something solid and long lasting. If the foundation is solid and square your improvements will be cheaper and easier to build. Skip that part and everything else you do will be difficult and expensive to build and poorly done at the end anyway. It’ll be a square addition added on to a crooked cabin.
Back to your rotting application - the one with the unsupported operating system and database, broken data model, the dead, archaic, insecure code that nobody understands. Are you re-skinning it to look nice? Are you grafting on features and business logic onto the sagging foundation?  Or are you digging out a new footing and pouring concrete?

The business unit doesn’t know or care about the foundation, they only care about the user interface and the features. Our job is to convince them that they need to spend time and money on the foundation, not just the features.

Tell them the story about the shack in the woods.


  1. Good analogy, but there's a problem: it only works if application owners are like long-term homeowners, not house flippers. If you can throw on fresh paint, new doors and windows, add a bedroom and redo the kitchen to make the place sell, the rotting foundation becomes the next poor sucker's problem. And even if you're not flipping, there's that new house they're building down the street for you that's supposed to be much better than this one (not that you're asking how they're building its foundation, either).

    I have no idea how to analyze or solve this. Are long-term employees more likely to care about problems that may happen five years from now? Are Highly Paid Consultants much less likely to? I don't know, but I think the problem with the shack in the woods is that a lot of people will just hope it becomes Somebody Else's Problem, and I wouldn't be surprised if that happens (at least subconsciously) with applications, too.


Post a Comment

Popular posts from this blog

Cargo Cult System Administration

Cargo Cult: …imitate the superficial exterior of a process or system without having any understanding of the underlying substance --Wikipedia During and after WWII, some native south pacific islanders erroneously associated the presence of war related technology with the delivery of highly desirable cargo. When the war ended and the cargo stopped showing up, they built crude facsimiles of runways, control towers, and airplanes in the belief that the presence of war technology caused the delivery of desirable cargo. From our point of view, it looks pretty amusing to see people build fake airplanes, runways and control towers  and wait for cargo to fall from the sky.
The question is, how amusing are we?We have cargo cult science[1], cargo cult management[2], cargo cult programming[3], how about cargo cult system management?Here’s some common system administration failures that might be ‘cargo cult’:
Failing to understand the difference between necessary and sufficient. A daily backup …

Ad-Hoc Versus Structured System Management

Structured system management is a concept that covers the fundamentals of building, securing, deploying, monitoring, logging, alerting, and documenting networks, servers and applications. Structured system management implies that you have those fundamentals in place, you execute them consistently, and you know all cases where you are inconsistent. The converse of structured system management is what I call ad hoc system management, where every system has it own plan, undocumented and inconsistent, and you don't know how inconsistent they are, because you've never looked.

In previous posts (here and here) I implied that structured system management was an integral part of improving system availability. Having inherited several platforms that had, at best, ad hoc system management, and having moved the platforms to something resembling structured system management, I've concluded that implementing basic structure around system management will be the best and fastest path to…

The Cloud – Provider Failure Modes

In The Cloud - Outsourcing Moved up the Stack[1] I compared the outsourcing that we do routinely (wide area networks) with the outsourcing of the higher layers of the application stack (processor, memory, storage). Conceptually they are similar:In both cases you’ve entrusted your bits to someone else, you’ve shared physical and logical resources with others, you’ve disassociated physical devices (circuits or servers) from logical devices (virtual circuits, virtual severs), and in exchange for what is hopefully better, faster, cheaper service, you give up visibility, manageability and control to a provider. There are differences though. In the case of networking, your cloud provider is only entrusted with your bits for the time it takes for those bits to cross the providers network, and the loss of a few bits is not catastrophic. For providers of higher layer services, the bits are entrusted to the provider for the life of the bits, and the loss of a few bits is a major problem. These …