Infrastructure that was built faster than it was understood
I'm Jakob. Catalift is my practice, and it's just me. I work out what a system actually does, where that differs from what you asked for, and then make it safe to operate.
The jokes, stated plainly
Each one is a real failure mode with the name changed.
“Justified state”
What got built isn't what you asked for. There's a rationale explaining why that was the better call, and you were never asked to agree with it.
“Plan determinism: not offered at any tier”
The plan that was reviewed and the plan that applied were not the same. A provider version, a module or a data source moved in between.
“Improvisation on failed API calls: cannot be disabled”
The apply failed after the merge and nobody looked. Half the change is live, and what it created just before stopping may never have reached the state file.
“Recorded and not surfaced”
The warning was in the plan output. The plan output was four thousand lines, so nobody read it.
What I do about it
Roughly in this order, because each step depends on the one before it.
Establish what is actually running. An inventory of the real environment, including the resources that exist because somebody fixed an outage by hand and never came back.
Surface the decisions that were made for you. You know what the business needs. Where a choice about cost, availability, or access got made by default, or by something guessing, I put it back in front of you with what I'd pick and why.
Reconcile state with reality. Get to the point where a plan can be run and believed, and where the difference between the two is a decision rather than a surprise.
Narrow what can change. Real permission boundaries, secrets that rotate, and environments that can be rebuilt from source instead of from memory.
Leave documentation that matches the system. Written for the next person to touch it, who will not be me.
Where to start
A fixed-scope look at the environment, ending in a written assessment. You keep it whether or not you hire me for the work it describes.
Getting in touch
Tell me what you're running and what you're afraid to touch.