Inherited a simple marketplace website that matches job seekers and hospitals in healthcare. Typically, all you need for this sort of thing is a web server, a database with search

But the precious devs decided to go micro-services in a container and db per service fashion. They ended up with over 50 docker containers with 50ish databases. It was a nightmare to scale or maintain!

With 50 database for for a simple web application that clearly needs to share data, integration testing was impossible, data loss became common, very hard to pin down, debugging was a nightmare, and also dangerous to change a service’s schema as dependencies were all tangled up.

The obvious thing was to scale down the infrastructure, so we could scale up properly, in a resource driven manner, rather than following the trend.

We made plans, but the CTO seemed worried about yet another architectural changes, so he invested in more infrastructure services, kubernetes, zipkin, prometheus etc without any idea what problems those infra services would solve.

Depressed since yesterday.

Updated all our clients Dialers. Stellar performance. Suddenly one of 15 can’t hang up three way calls.

It’s one of our biggest clients. And they just started. We upgraded the dialers so the answering machine detection would improve for them and it did, along with vast performance upgrades as well. Suddenly, this issue.

2 days in they pull the plug until we fix it. The issue is sporadic and we cannot reproduce. No one else is having the issue. I can’t even debug it properly as it’s a third party dialer with no customizations on it. I found out where the error is, but no idea the workflow they got it to happen with or why. It’s so frustrating. It happens using the dialer native interface, and our integration via api calls. The channel doesn’t get sent to the command for some random reason, and only sometimes.

So even if it’s fixed they don’t trust the system. Now they are losing the full integration we have with the crm and dialer and it’s going to be a mess of data for them. All because of this one issue. They love the CRM though...

If they had just stayed on one more day I’m sure I could have found it. Now I have to play forensic scientist and look through old data, without being able to see the client code that was causing the issue.

Just threw some cash down to be able to talk to the dialer engineers and hopefully see what’s up. What a nightmare. And I have so many other projects for the platform due so soon...

Sigh. Super depressing.

Meeting with another dev team whose application needs to interface with ours. A few topics and Q&A sessions later, a dreadful feeling started to creep up on me. That moment when you grep'd the other team's architects and technical lead for any combination of common sense and grep returned no results. This is going to be a long day

I am amazed at human stupidity.

I always enjoyed the idea of DevOps: to use virtual machines and constant integration in order to avoid errors and free the developers of hard-to-setup environments and somehow-it-works compilations.

I am amazed how [company I used to work for] managed to turn this into a nightmare.

Just imagine: silent forests, the smell of flowers, no developer trust to the point your devs can’t either make docker environments cause reasons nor they can access your actual machines programmatically because they are filthy peasants, forcing them to do everything manually: every deployment will be a frustrating editing process which takes up to an hour, but here lies the trick... it will still have continuous integration... or better: every feature will be deployed as if it was a release.

The true peak of illumination:
Turning a tool into a disease.

Take a sip of tea, manager... you deserve it.

Just thought about this job because I keep being tempted to just start my own company. The more I think about it, the less being employed makes sense, given my end goal.

This is influenced by my current situation but best tool: Visual Studio. Versatile and rich debugger, good language integration for what I do. Worst tool: eclipse. What the fuck is this permacrashing nightmare of an application. And what the actual fuck are these keybinds.