Guy called in because he wanted to get an IP white listed on a server. He wasn't authorized so i told him to send an email from an authorized email address.

He didn't like that very much and asked if another engineer was available (he talked to him more often so he thought that engineer would just do it. We need those kind of requests by email.)

Walked over to my colleague and explained what that client asked for.
'let him send an email!'

Told him i ready told the client that but that the client wanted to talk to him instead.

'sure, connect him through and then come back so you can hear him after i ask him to mail us!'

Connected him through. Client explained the situation.

Then he says with the sweetest voice and a 'get rekt' face: 'could you send me an email about that? 😊'

Let's just say that the client sounded everything but happy xD

Long rant ahead, but it's worth it.

I used to work with a professor (let's call him Dr. X) and developed a backend + acted as sysadmin for our team's research project. Two semesters ago, they wanted to revamp the front end + do some data visualization, so a girl (let's call her W) joined the team and did all that. We wanted to merge the two sites and host on azure, but due to issues and impeding conferences that require our data to be online, we kept postponing. I graduate this semester and haven't worked with the team for a while, so they have a new guy in charge of the azure server (let's call him H), and yesterday my professor sends me (let's call me M), H and W an email telling us to coordinate to have the merge up on azure in 2-3 days, max. The following convo was what I had with H:

M: Hi, if you just give me access to azure I'll be able to set everything up myself, also I'll need a db set up, and just send me the connection string.
H: Hi, we won't have dbs because that is extra costs involved since we don't have dynamic content. Also I can't give you access, instead push everything on git and set up the site on a test azure server and I will take it from there.
M: There is proprietary data on the site...
H: Oh really? I don't know what's on it.
<and yet he knows we have no dynamic data>
M: Fine, I'll load the data some other way, but I have access to all the data anyway, just talk to Dr. X and you'll see you can give me access. Delete my access after if you want.
H: No, just do what I said: git then upload to test azure account.

Fine, he's a complete tool, but I like Dr. X, so I message W and tell her we have to merge, she tells me that it's not that easy to set it up on github as she's using wordpress. She sends me instructions on what to do, and, lo and behold, there's a db in her solution. Ok, I go back to talking to H:

M: W is using a db. Talk to her so we can figure out whether we need a database or not.
H: We can't use a database because we want to decrease costs.
M: Yes I know that, so talk to her because that probably means she has to re-do some stuff, which might take some time. Also there might be dynamic content in what she's doing.
H: This is your project, you talk to her.
<I'm starting to get mad right now>
M: I don't know what they had her do apart from how it interfaces with what I've done.
H: We still can't have databases.
M: Listen, I don't do wordpress, and I'm not gonna mess with it, you talk to her
H: I won't do any development
<So you won't do any dev, but you won't give me access to do it either?>
M: Man, the bottleneck isn't the merging right now, it's the fact that W needs a db
H: I know, so talk to her
M: THE RESTRICTION TO NOT HAVE DATABASES IS NOT MINE, IT'S YOURS, YOU TALK TO HER. I can't evaluate whether it's a reasonable enough reason or not since I don't know the requirements or what they're willing to spend.
H: It's your project.
M: Then give me fucking access to azure and I'll handle it, you know you'll have to set up wordpress again regardless whether we set it up the first time.
H: Man just do your job.

At this point I lost it. WHAT A FUCKING TOOL. He doesn't wanna do dev work, wants me to go through the trouble of setting up on a test subscription first, and doesn't want to give me access to azure. What's more, he did shit all and doesn't want to anything else. Well fuck you. I googled him, to see if he's anyone important, if he's done anything notable which is why he's being so God damn condescending. MY INTERNSHIP ALONE ECLIPSES HIS ENTIRE CV. Then what the fuck?

There's also this that happened sometime during our talk:

M: You'll have to take to Dr. Y so he'll change the DNS to point to the azure subscription instead of my server.
H: Yea don't worry, too early for that.
M: DNS propagation takes 24 hours...
H: Yea don't worry.

DNS propagation allows the entire web to know that your website is hosted on a different server so it can change where it's pointing to. We have to do this in 2-3 days. Why do work in parallel? Nah let's wait.

I went over his head and talked to the professor directly, and despite wanting to tell him that he was both drunk and high the day he hired that guy, I kept it professional. He hasn't replied yet, but this fucker's pompous attitude is just too much for me alone, so I had to share.

PS: I named his contact as Annoying Prick 4 minutes into our chat. Gonna rename him cz that seems tooooooo soft a name right now.

Oh, man, I just realized I haven't ranted one of my best stories on here!

So, here goes!

A few years back the company I work for was contacted by an older client regarding a new project.

The guy was now pitching to build the website for the Parliament of another country (not gonna name it, NDAs and stuff), and was planning on outsourcing the development, as he had no team and he was only aiming on taking care of the client service/project management side of the project.

Out of principle (and also to preserve our mental integrity), we have purposely avoided working with government bodies of any kind, in any country, but he was a friend of our CEO and pleaded until we singed on board.

Now, the project itself was way bigger than we expected, as the wanted more of an internal CRM, centralized document archive, event management, internal planning, multiple interfaced, role based access restricted monster of an administration interface, complete with regular user website, also packed with all kind of features, dashboards and so on.

Long story short, a lot bigger than what we were expecting based on the initial brief.

The development period was hell. New features were coming in on a weekly basis. Already implemented functionality was constantly being changed or redefined. No requests we ever made about clarifications and/or materials or information were ever answered on time.

They also somehow bullied the guy that brought us the project into also including the data migration from the old website into the new one we were building and we somehow ended up having to extract meaningful, formatted, sanitized content parsing static HTML files and connecting them to download-able files (almost every page in the old website had files available to download) we needed to also include in a sane way.

Now, don't think the files were simple URL paths we can trace to a folder/file path, oh no!!! The links were some form of hash combination that had to be exploded and tested against some king of database relationship tables that only had hashed indexes relating to other tables, that also only had hashed indexes relating to some other tables that kept a database of the website pages HTML file naming. So what we had to do is identify the files based on a combination of hashed indexes and re-hashed HTML file names that in the end would give us a filename for a real file that we had to then search for inside a list of over 20 folders not related to one another.

So we did this. Created a script that processed the hell out of over 10000 HTML files, database entries and files and re-indexed and re-named all this shit into a meaningful database of sane data and well organized files.

So, with this we were nearing the finish line for the project, which by now exceeded the estimated time by over to times.

We test everything, retest it all again for good measure, pack everything up for deployment, simulate on a staging environment, give the final client access to the staging version, get them to accept that all requirements are met, finish writing the documentation for the codebase, write detailed deployment procedure, include some automation and testing tools also for good measure, recommend production setup, hardware specs, software versions, server side optimization like caching, load balancing and all that we could think would ever be useful, all with more documentation and instructions.

As the project was built on PHP/MySQL (as requested), we recommended a Linux environment for production. Oh, I forgot to tell you that over the development period they kept asking us to also include steps for Windows procedures along with our regular documentation. Was a bit strange, but we added it in there just so we can finish and close the damn project.

So, we send them all the above and go get drunk as fuck in celebration of getting rid of them once and for all...

Next day: hung over, I get to the office, open my laptop and see on new email. I only had the one new mail, so I open it to see what it's about.

Lo and behold! The fuckers over in the other country that called themselves "IT guys", and were the ones making all the changes and additions to our requirements, were not capable enough to follow step by step instructions in order to deploy the project on their servers!!!

[Continues in the comments]

When someone's calls and completely loses his shit (swearing etc etc) because we HAVE TO FUCKING HELP HIM BECAUSE THOUSANDS OF PEOPLE RELY ON THIS SITE WHICH IS DOWN.
Explained him calmly that its an unmanaged server which we literally don't have access to. Keeps on calling me all kinda things and then says he'll email some login details.

Bossman walks in, collegue goes like 'yo guess what just happened with linuxxx!'
Le me explains the story.

Bossman: email me his number. I'm going to call him. You treat my employees with fucking respect.

You know what?

Young cocky React devs can suck my old fuckin LAMP and Objective-C balls.

Got a new freelance job and got brought in to triage a React Native iOS/Android app. Lead dev's first comment to me is: "Bro, have you ever used React Native".

To which I had to reply to save my honor publicly, "No, but I have like 8 years with Objective-C and 3 years with Swift, and 3 years with Node, so I maybe I'll still be able help. Sometimes it just helps to have a fresh set of eyes."

"Well, nobody but me can work on this code."

And that, as it turned out was almost true.

After going back and forth with our PM and this dev I finally get his code base.

"Just run "npm install" he says".

Like no fuckin shit junior... lets see if that will actually work.

Node 14... nope whole project dies.

Node 12 LTS... nope whole project dies.

Install all of react native globally because fuck it, try again... still dies.

Node 10 LTS... project installs but still won't run or build complaining about some conflict with React Native libraries and Cocoa pods.

Go back to my PM... "Um, this project won't work on any version of Node newer than about 5 years old... and even if it did it still won't build, and even if it would build it still runs like shit. And even if we fix all of that Apple might still tell us to fuck off because it's React Native.

Spend like a week in npm and node hell just trying to fucking hand install enough dependencies to unfuck this turds project.

All the while the original dev is still trying TO FIX HIS OWN FUCKING CODE while also being a cocky ass the entire time. Now, I can appreciate a cocky dev... I was horrendously cocky in my younger days and have only gotten marginally better with age. But if you're gonna be cocky, you also have to be good at it. And this guy was not.

Lo, we're not done. OG Dev comes down with "Corona Virus"... I put this in quotes because the dude ends up drawing out his "virus" for over 4 months before finally putting us in touch with "another dev team he sometimes uses".

Next, me and my PM get on a MS Teams call with this Indian house. No problems there, I've worked with the Indians before... but... these are guys are not good. They're talking about how they've already built the iOS build... but then I ask them what they did to sort out the ReactNative/Cocoa Pods conflict and they have no idea what I'm talking about.

Why?

Well, one of these suckers sends a link to some repo and I find out why. When he sends the link it exposes his email...

This Indian dude's emails was our-devs-name@gmail.com...

We'd been played.

Company sued the shit out of the OG dev and the Indian company he was selling off his work to.

I rewrote the app in Swift.

So, lets review... the React dev fucked up his own project so bad even he couldn't fix it... had to get a team of Indians to help who also couldn't fix it... was still a dickhead to me when I couldn't fix it... and in the end it was all so broken we had to just do a rewrite.

None of you get npm. None of you get React. None of you get that doing the web the way Mark Zucherberg does it just makes you a choad locked into that ecosystem. None of you can fix your own damn projects when one of the 6,000 dependency developers pushes breaking changes. None of you ever even bother with "npm audit fix" because if security was a concern you'd be using a server side language for fucking server side programming like a grown up.

So, next time a senior dev with 20 years exp. gets brought in to help triage a project that you yourself fucked up... Remember that the new thing you know and think makes you cool? It's not new and it's not cool. It's just JavaScript on the server so you script kiddies never have to learn anything but JavaScript... which makes you inarguably worse programmers.

And, MF, I was literally writing javascript while you were sucking your mommas titties so just chill... this shit ain't new and I've got a dozen of my own Node daemons running right now... difference is?

Mine are still working.

So I got the job. Here's a story, never let anyone stop you from accomplishing your dreams!

It all started in 2010. Windows just crashed unrecoverably for the 3rd time in two years. Back then I wasn't good with computers yet so we got our tech guy to look at it and he said: "either pay for a windows license again (we nearly spend 1K on licenses already) or try another operating system which is free: Ubuntu. If you don't like it anyways, we can always switch back to Windows!"

Oh well, fair enough, not much to lose, right! So we went with Ubuntu. Within about 2 hours I could find everything. From the software installer to OpenOffice, browsers, email things and so on. Also I already got the basics of the Linux terminal (bash in this case) like ls, cd, mkdir and a few more.

My parents found it very easy to work with as well so we decided to stick with it.

I already started to experiment with some html/css code because the thought of being able to write my own websites was awesome! Within about a week or so I figured out a simple html site.

Then I started to experiment more and more.

After about a year of trial and error (repeat about 1000+ times) I finally got my first Apache server setup on a VirtualBox running Ubuntu server. Damn, it felt awesome to see my own shit working!

From that moment on I continued to try everything I could with Linux because I found the principle that I basically could do everything I wanted (possible with software solutions) without any limitations (like with Windows/Mac) very fucking awesome. I owned the fucking system.

Then, after some years, I got my first shared hosting plan! It was awesome to see my own (with subdomain) website online, functioning very well!

I started to learn stuff like FTP, SSH and so on.

Went on with trial and error for a while and then the thought occured to me: what if I'd have a little server ONLINE which I could use myself to experiment around?

First rented VPS was there! Couldn't get enough of it and kept experimenting with server thingies, linux in general aaand so on.

Started learning about rsa key based login, firewalls (iptables), brute force prevention (fail2ban), vhosts (apache2 still), SSL (damn this was an interesting one, how the fuck do you do this yourself?!), PHP and many other things.

Then, after a while, the thought came to mind: what if I'd have a dedicated server!?!?!?!

I ordered my first fucking dedicated server. Damn, this was awesome! Already knew some stuff about defending myself from brute force bots and so on so it went pretty well.

Finally made the jump to NginX and CentOS!

Made multiple VPS's for shitloads of purposes and just to learn. Started working with reverse proxies (nginx), proxy servers, SSL for everything (because fuck basic http WITHOUT SSL), vhosts and so on.

Started with simple, one screen linux setup with ubuntu 10.04.

Running a five monitor setup now with many distro's, running about 20 servers with proxies/nginx/apache2/multiple db engines, as much security as I can integrate and this fucking passion just got me my first Linux job!

It's not just an operating system for me, it's a way of life. And with that I don't just mean the operating system, but also the idea behind it :).

So a friend of Mine asked me to check their Mail server because some emails got lost. Or had a funny signature.

Mails were sent from outlook so ok let's do this.

I go create a dummy account, and send/receive a few emails. All were coming in except one and some had a link appended. The link was randomly generated and was always some kind of referral.

Ok this this let's check the Mail Server.

Nothing.

Let's check the mail header. Nothing.

Face -> wall

Fml I want to cry.

Now I want to search for a pattern and write a script which sends a bunch of mails on my laptop.
Fuck this : no WLAN and no LAN Ports available. Fine let's hotspot the phone and send a few fucking mails.

Guess what? Fucking cockmagic, no funny mails appear!

At that moment I went out and was like chainsmoking 5 cigarettes.

BAM!

It hit me! A feeling like a unicorn vomiting rainbows all over my face.

I go check their firewall. Shit redirected all email ports from within the network to another server.

Yay nobody got credentials because nobody new it existed. Damn boy.

Hook on to the hostmachine power down the vm, start and hack yourself a root account before shit boots. Luckily I just forgot the credentials to a testvm some time ago so I know that shit. Lesson learned: fucking learn from your mistakes, might be useful sometimes!

Ok fucker what in the world are you doing.

Do some terminal magic and see that it listens on the email ports.

Holy cockriders of the galaxy.

Turns out their former it guy made a script which caught all mails from the server and injected all kind of bullshit and then sent them to real Webserver. And the reason why some mails weren't received was said guy was too dumb to implement Unicode and some mails just broke his script.

That fucker even implented an API to pull all those bullshit refs.

I know your name "Matthias" and I know where you live and what you've done... And to fuck you back for that misery I took your accounts and since you used the same fucking password for everything I took your mail, Facebook and steam account too.

Git gut shithead! You better get a lawyer

So this happened today.

Client: hey I sent this ticket, what's the status/have you located the issue?
Me: well, it says it quite obviously in the error message...? (i actually said that, toned down afterwards a little)
Client: where's the error message then?
Me: 5th line....? It's literally there in plain english?
Client: ok so what does it mean?
Me:..............? "marked as spam by the receiving server"?!
Client: yeah ok but what does that mean?

😐

Thing to keep in mind: they're a web dev/email solutions company.

😐😩

When someone, after explaining it for 10 FUCKING TIMES, still doesn't get that WE ARE NOT HOSTING HIS MOTHERFUCKING SERVER SO WE CAN'T FIX HIS FATHERFUCKING COCKSUCKING EMAIL ISSUES.

FUCKING HELL.

Today I discovered that we have a CSV export button for an order transaction system, on a page which is completely disconnected from the rest of the website.

It is only being called by an internal server, used by our Data department.

They run selenium to click the button.

Then they import the CSV into a database.

That database is accessed by an admin panel.

That admin panel has an excel export button.

Which is clicked by our CFO. But he got bored of clicking, so he uses IFTTT to schedule a download of the XLS and import it in Google Sheets.

That sheet uses a Salesforce data connector.

Marketing then sends email campaigns based on that Salesforce data...

😒

A client called today because their email wasn't arriving at the receipants inbox but bouncing back with a 'poor MTA rating' error.

Checked about every blacklist I know and our server was definitely not blacklisted. Must be the receipants host which for some reason was blacklisting his specific email address.

Told the client that it wasn't a problem on our side and that he had to request a whitelist himself (we'd do it but it wasn't a specific server problem so we're not going to spend time on that).

Fair enough, he'd do that.

Calls back. "Well, the other party says that your server definitely has a poor rating, it's on your side!!"

Alright, this is getting annoying. Gave him a few blacklist checking sites links and told him to run his domain AND our server IP through it. Indeed came back completely clean.

"But the other party said it's poor rating on your side so I'd think tha........"

YEAH WHY DON'T YOU SHOVE THAT OTHER PARTY UP YOUR FUCKING ASS. I'VE SHOWN YOU PROOF THAT IT'S DEFINITELY NOT ON OUR FUCKING SIDE, EXPLAINED IT TO YOU AND SO ON. MAYBE, FOR ONE FUCKING SECOND, TAKE INTO CONSIDERATION THAT THE OTHER PARTY IS FUCKING LYING?!?!?

FUCK OFF.

I suddenly remembered this after being gone from my previous company for nearly a year.

So, I worked there as a tech supporter and Linux engineer.
What would often happen was clients calling with an issue regarding software of some sorts and about half the time, instead of LOOKING AT THE GODDAMN ERROR MESSAGE they'd just click it away fast and complain shit wasn't working.

I specifically remember this one case:

*big client mails complained that one of their clients' email isn't working. Screenshots weren't possible apparently so after emailing back and forth for way too long, we decide to do a screen sharing session (which we never do).*
(for the record, already emailing for hours, client very frustrated, me as well because the behavior of the software sounds impossible)

Me: alright, close everything, then open it again so I can see what happens.
Client: *opens mail client, error appears, client clicks error away faster than an arch user being able to mention they use arch*
Me: uhm.... I assume you already know what that message said and that it has nothing to do with the issue?
Client: it has nothing to do with the issue.
Me: okay... But have you at least looked the message?
Client: no but it has nothing to do with the issue.
Me: but, how'd you know if you won't look at it?
Client: it has nothing to do with the issue, okay?
Me: okay.... so, what's happening here?
Client: the user isn't receiving email anymore at this point!
Me: alright, have you checked the settings and everything?
Client: of course, all good
Me: okay but can we at least restart the software again to at least check the error message?
Client: FINE. *restarts client (pun intended, of course)*
Error message: username or password incorrect, can't connect to the server.
Client:..........
Client:............
Client:...............
Client:..................
Client:.....................
Client:..................
Client:...............
Client:............
Client:.........
Client: 😐
Client: 😶
Client: 😅
Client: 😬
Client:..... Right, I changed the password...
Client: *sets correct password*
*poof, error message gone*
Client:..... Thanks 💀
Me: you're welcome 😄

💀

Da Fuck!?!
Yesterday I found some abnormal activity on my server, someone was trying to brute force my ssh as root since two days! Started raging and installed fail2ban (which automatically bans an IP if it fails to log X times and eventually sends me an email). Woke up this morning to find that a fucking Chinese guy/malware spent the whole night trying to brute Force me!
Fucking cunt! Don't you have any better to do!!
My key is a 32 characters long encrypted key, with the ban he can try 3 passwords /2 hours, good luck brute forcing it you bitch!

I was hired as a senior software engineer. During handover I found out I'm actually replacing the CTO.

I queried why he was leaving and got a simple "just want a break from working" which I found odd.

Fast forward and now I also just want a break from work, permanently. This place has followed every bad practise and big no-no out there. Every bit of software is a built in house knockoff janky piece of crap that doesn't work and makes people's jobs 5000 times harder.

The UI looks worse than Windows 3.1, absolutely horrendous code formatting, worst database structure I've ever seen.

The mere mention of using a team communication tool results in being yelled at from the CEO whom communicates purely via email, who then gets annoyed when you don't reply because they sent the email to a client instead of you.

We get handed printed out "tickets" to work instead of the so called "amazing in house ticket system" built using PHP 5 and is literally crammed into an 800x600 IFrame. Yes a F$*#ing IFRAME!

It's not like we have an outdated TFS server that has work items we can use...

Why not push for changes you say. I have, many times, tried to suggest better tools. The only approval I've gotten is using PhpStorm. Everything else is shutdown immediately and you get the silent treatment.

The CEO hired me to do a job, then micromanages like crazy. I can't make UI changes, I can't make database changes, why? They insists they know best, but has admitted multiple times to not knowing SQL and literally uses a drag and drop database table builder.

Every page in the webapps we make are crammed into 800x600 iframes with more iframes inside iframes. And every time it's pointed out we need to do something, be it from internal staff or client suggestions, the CEO goes off about how the UI is industry leading and follows standards.. what in the actual f....

Literally holding on by a thread here. Why hire a CTO under the guise of being a senior developer but then reduce the work that can be done down to the level of a junior?

Sure the paycheck is really nice but no job is worth the stress, harassment and incompetent leadership from the CEO.

They've verbally abused people to the point they resign, best part is that was simply because the CEO made serious legal mistakes, was told about it by the employee then blamed it on others.

Had this a few weeks ago. A customer send an email telling that her emails didn't load anymore in the web interface.

Fair enough, tested it and indeed, no emails while there were definitely emails on the server and nothing in the logs or anything.

Replied saying that I'd look into it.

Few hours later she messaged back all happy about it working again and thanking me so much for all the effort I put into getting it to work again!

My thought at that moment: "oh shit completely forgot about that ticket... Wait it's working again? Oo I didn't do anything... 😅"

Love it how some things just solve themselves haha

Our company got attacked last month by what i believe was a code time bomb from a ex employee. And it was brutal, website hacked, email server not responding, locked out from database servers. The IT department asked for my help and I was more than happy to do it. Long story short I got every thing back working smoothly. The IT guys ask for a favor to not include this in my monthly progress report. Fine by me. But then they went out and tokd the top management that they are teaching me about the networks and servers so thats why I was working with them last month. Fucking assholes. Not going to help them any more.

Someone's outlook wasn't connecting to a mail server.

Fair enough, colleague started debugging in the morning!

It worked fine on any client on linux/mac.

After a while the swearing started to come, for some reason outlook thought that the url used for incoming/outcoming email was offline, worked on any other system.

We all left him alone for the rest of the day.

At the end I walked to his desk aan went:
Me: hey man is it working already? *very sweet smile*
Him: *gives a death stare* fucking die 😡

😆😅

This is super childish but it's the gameserver insidstry and karma is a bitch.

TLDR: I hacked my boss

I was working for a gameserver and I did development for about 3 months and was promised pay after the network was released. I followed through with a bunch of dev friends and the guy ended up selling our work. He didn't know that I was aware of this as he tried to tell people to not tell us but one honest person came forward and said he sold our work for about 8x the price of what he owed ALL OF US collectively.

I proceeded to change the server password and when he asked why he couldn't log in I sent him an executable (a crypted remote access tool) and told him it was an "encryption tunnel" that makes ssh and file transfers secure. Being the idiot that he is he opened it and I snagged all of his passwords including his email and I changed them through a proxy on his machine to ensure I wouldn't get two factored with Google. After I was done I deleted system 32 :3

Worst thing you've seen another dev do? Long one, but has a happy ending.

Classic 'Dev deploys to production at 5:00PM on a Friday, and goes home.' story.

The web department was managed under the the Marketing department, so they were not required to adhere to any type of coding standards and for months we fought with them on logging. Pre-Splunk, we rolled our own logging/alerting solution and they hated being the #1 reason for phone calls/texts/emails every night.

Wanting to "get it done", 'Tony' decided to bypass the default logging and send himself an email if an exception occurred in his code.

At 5:00PM on a Friday, deploys, goes home.

Around 11:00AM on Sunday (a lot folks are still in church at this time), the VP of IS gets a call from the CEO (who does not go to church) about unable to log into his email. VP has to leave church..drive home and find out he cannot remote access the exchange server. He starts making other phone calls..forcing the entire networking department to drive in and get email back up (you can imagine not a group of happy people)

After some network-admin voodoo, by 12:00, they discover/fix the issue (know it was Tony's email that was the problem)

We find out Monday that not only did Tony deploy at 5:00 on a Friday, the deployment wasn't approved, had features no one asked for, wasn't checked into version control, and the exception during checkout cost the company over $50,000 in lost sales.

Was Tony fired? Noooo. The web is our cash cow and Tony was considered a top web developer (and he knew that), Tony decided to blame logging. While in the discovery meeting, Tony told the bosses that it wasn't his fault logging was so buggy and caused so many phone calls/texts/emails every night, if he had been trained properly, this problem could have been avoided.

Well, since I was responsible for logging, I was next in the hot seat.

For almost 30 minutes I listened to every terrible thing I had done to Tony ever since he started. I was a terrible mentor, I was mean, I was degrading, etc..etc.
Me: "Where is this coming from? I barely know Tony. We're not even in the same building. I met him once when he started, maybe saw him a couple of times in meetings."
Andrew: "Aren't you responsible for this logging fiasco?"
Me: "Good Lord no, why am I here?"
Andrew: "I'll rephrase so you'll understand, aren't you are responsible for the proper training of how developers log errors in their code? This disaster is clearly a consequence of your failure. What do you have to say for yourself?"
Me: "Nothing. Developers are responsible for their own choices. Tony made the choice to bypass our logging and send errors to himself, causing Exchange to lockup and losing sales."
Andrew: "A choice he made because he was not properly informed of the consequences? Again, that is a failure in the proper use of logging, and why you are here."
Me: "I'm done with this. Does John know I'm in here? How about you get John and you talk to him like that."
'John' was the department head at the time.
Andrew:"John, have you spoken to Tony?"
John: "Yes, and I'm very sorry and very disappointed. This won't happen again."
Me: "Um...What?"
John: "You know what. Did you even fucking talk to Tony? You just sit in your ivory tower and think your actions don't matter?"
Me: "Whoa!! What are you talking about!? My responsibility for logging stops with the work instructions. After that if Tony decides to do something else, that is on him."
John: "That is not how Tony tells it. He said he's been struggling with your logging system everyday since he's started and you've done nothing to help. This behavior ends today. We're a fucking team. Get off your damn high horse and help the little guy every once in a while."
Me: "I don't know what Tony has been telling you, but I barely know the guy. If he has been having trouble with the one line of code to log, this is the first I've heard of it."
John: "Like I said, this ends today. You are going to come up with a proper training class and learn to get out and talk to other people."

Over the next couple of weeks I become a powerpoint wizard and 'train' anyone/everyone on the proper use of logging. The one line of code to log. One line of code.

A friend 'Scott' sits close to Tony (I mean I do get out and know people) told me that Tony poured out the crocodile tears. Like cried and cried, apologizing, calling me everything but a kitchen sink,...etc. It was so bad, his manager 'Sally' was crying, her boss 'Andrew', was red in the face, when 'John' heard 'Sally' was crying, you can imagine the high levels of alpha-male 'gotta look like I'm protecting the females' hormones flowing.

Took almost another year, Tony released a change on a Friday, went home, web site crashed (losses were in the thousands of $ per minute this time), and Tony was not let back into the building on Monday (one of the best days of my life).

I worked at a place where the help desk guys did the good ol' "I'll send an email from your laptop if you walk away without locking it and tell everyone lunch is on you" routine. After it happened to me about 3 times I was like, "I gotta get this help desk prick back!" So after several failed attempts at walking by his pc when he walked away it instantly hit me how I can punk him back.....SO, I logged onto SQL Server, clicked open a new query window and typed up a dbmail command and on the @from parameter I set it to the help desk guy's email address. His face was PRICELESS when I was shooting off emails to the entire IT dept on behalf of him WHILE he was sitting in front of his PC. Lesson is: don't fuck with dev help desk dude! 😎😜

Last september:
I get an email from a customer who want to end the contract. I inform him that the contract runs to the end of March and it will be terminated the last of march.

Customer agrees and I do not hear a word from him. Until now when I closed down all his services.

Angry email:
Why did you close the server and email? are you **** stupid!?

I reply:
21/9 2016, you wanted to terminate all services you had with us. That means that your domains, DNS-services, VPS, email and Office 365 was going to be closed today.

Why is this not uncommon?!

Product sending an email: Can I confirm feature A is all set for its release on April 30th?

Me: ... what? no that feature is going out with Feature B, that was pushed back to June because of the server issue.

Product: No, the release plan document says April 30th for this.

Me: ... theres 6 copies of this doc now. Someone is after deleting my comments saying "releasing with Feature B". Oh look heres a link to another doc that says this. See Feb14th "Will go out with Feature B". This is because they are touching the same code, we can't separate them now without re-writing it.

*Me to myself*: Ha product are going to hate this, their shitty processes have finally caught up with them.

*next day*

Other manager: So heres my plan for the app release x, y, z.

*Me to myself*: ... his plan? this is my app, I mange this. What the hell is this?

*reads email thread*

*Me to myself*: ... oh so product really didn't like my reply, took me off the thread, sent a response to all the other managers asking for alternatives, CC'ing upper management. The same upper management I had a private conversation with yesterday about how shit our product team are.

*cracks knuckles*

I'm going to enjoy writing this reply.

My team handles infrastructure deployment and automation in the cloud for our company, so we don't exactly develop applications ourselves, but we're responsible for building deployment pipelines, provisioning cloud resources, automating their deployments, etc.

I've ranted about this before, but it fits the weekly rant so I'll do it again.

Someone deployed an autoscaling application into our production AWS account, but they set the maximum instance count to 300. The account limit was less than that. So, of course, their application gets stuck and starts scaling out infinitely. Two hundred new servers spun up in an hour before hitting the limit and then throwing errors all over the place. They send me a ticket and I login to AWS to investigate. Not only have they broken their own application, but they've also made it impossible to deploy anything else into prod. Every other autoscaling group is now unable to scale out at all. We had to submit an emergency limit increase request to AWS, spent thousands of dollars on those stupidly-large instances, and yelled at the dev team responsible. Two weeks later, THEY INCREASED THE MAX COUNT TO 500 AND IT HAPPENED AGAIN!

And the whole thing happened because a database filled up the hard drive, so it would spin up a new server, whose hard drive would be full already and thus spin up a new server, and so on into infinity.

Thats probably the only WTF moment that resulted in me actually saying "WTF?!" out loud to the person responsible, but I've had others. One dev team had their code logging to a location they couldn't access, so we got daily requests for two weeks to download and email log files to them. Another dev team refused to believe their server was crashing due to their bad code even after we showed them the logs that demonstrated their application had a massive memory leak. Another team arbitrarily decided that they were going to deploy their code at 4 AM on a Saturday and they wanted a member of my team to be available in case something went wrong. We aren't 24/7 support. We aren't even weekend support. Or any support, technically. Another team told us we had one day to do three weeks' worth of work to deploy their application because they had set a hard deadline and then didn't tell us about it until the day before. We gave them a flat "No" for that request.

I could probably keep going, but you get the gist of it.

Contrary to most people I really love to receive email related to jobs when I'm in holiday. I keep important alerts on.

It's like:
email: ***urgent, server down***
me (sipping mojito by the pool): fuck them. let's them deal with that
email: ***requirements all wrong, must develop the feature again***
me (enjoying a dinner): oh, I told them 100 times!, fuck all of them, work for me now, stupid moron.
email: I destroyed by mistake the db with an update..."
me (dancing like crazy): ahahaha I told you that support guys should not have access to production db, fuckfuck you, fix it yourself!!!

and so on..... I don't know, it just boost my pleasure during holiday.

RANT Incoming
Not necessarily dev related but I need to get this off my chest.

So a bit of a backstory. I had to stay late from school the other day and ended up having to take an Uber home. The ride was fine lady was nice. Everything seems to be going well and there were no signs of any payment failure.

Then yesterday, I had to stay late again. I never said that I had an outstanding balance on my account. Apparently Uber was having problems charging my Android pay account.
So I ended up being stuck at school for like 3 hours. Great!😑

So I emailed Uber when I got home. And this is when I started pulling my hair out. I don't know how many replies I had, but each time I had to tell them that I was not using a prepaid card.

This was one of my replies:
"I'm sorry, are you real? If you are, here is a quick summary of the issue. I am using ANDROID PAY with my CHASE DEBIT CARD. Not, NOT, NOT a prepaid card. I happen to know that CHASE DEBIT CARD(which is the card I use, in case you have already forgotten) works with uber because MY FATHER USES THE EXACT SAME TYPE OF CARD with uber. He uses a CHASE DEBIT CARD(again I use that same type of card as well). So by using LOGIC I am able to deduce that a CHASE DEBIT CARD is in fact compatible. AGAIN THIS IS NOT A PREPAID CARD!!! If the card is incompatible, WHY DOES THE APP ALLOW BE TO ADD IT?!?! Also in response to your last email... Because I am using Android pay, do you really think that an ANDROID would be able to use APPLE pay? Also Google wallet is DISCONTINUED! Finally, PayPal DOES NOT CONNECT TO UBER. Returns a "Server Error." So please stop wasting my time with generic help solutions. Believe me, I have already googled my issue, and nothing comes up. That is why I contacted Uber. I want my driver to be paid, and, uber had made it SO painful with unhelpful "Solutions" to problems that don't even APPLY TO MY ISSUE. No not even mention PREPAID cards in your reply or I will consider you a robot built by monkeys banging their heads on a keyboard. Uber HAS my VALID payment information, USE IT! If there is a phone number I can call, please, enlighten me"

And the response was:
"Thanks for reaching out with this.

Happy to help with this issue you are having.

After reviewing your I can see that the only payment method associated with your account is an ANDROID PAY card and it is also a prepaid card. Some cards and methods are not compatible with our billing processes and can't be used with Uber. This includes prepaid cards."

So I concluded that they are monkeys.

Then Uber banned me from logging into my account because I didn't pay.
So now it is impossible for me to pay because I can't do anything with my account.

Now they want my SSN and a bunch of other shit that I won't give them.

I told them that they were being illogical, and I got the exact same response about the prepaid bullshit.

So I sent them this photo as a goodbye.

I get my driver's licence next weekend, so I won't need Uber anymore. YAY!

Also mind grammatical errors, I talked it in and am to lazy to proofread

Me: Well, it's time to make a new app!
* opens up VS Code *
* opens folder selection dialog *
* creates a new folder called "notes app" *
* yarn inits that folder *
* installs react and react-dom *
* installs webpack, webpack-cli, babel-core, babel-loader, babel-preset-env, babel-preset-react, style-loader, css-loader, file-loader, html-webpack-plugin and clean-webpack-plugin as a dev dependency (install is pending) *
* copies a webpack config from some other project *
* creates a babelrc file *
* copies a yarn script called "build:dev" which would launch webpack *
* dev dependencies installed *
* tries to save *
* vscode doesn't save because files differ *
* tries to copy dev dependencies *
* fail *
* tries again *
* saves *
* writes bare-bones index.jsx *
* yarn build:dev *
* opens build/index.html in firefox *
* gets satisfaction *
* writes bare-bones App.jsx which is a react component but it's an entire app *
* yarn build:dev *
* opens build/index.html in firefox *
* gets satisfaction *
-- trim --
* walks out of his room to his mom's room where's sbc is located *
* grandma plays solitare on laptop *
* i ask grandma for a laptop *
* grandma gives me laptop *
* glues all components into App.jsx *
* yarn start:dev (magic of webpack-dev-server) *
* opens localhost:8080 in firefox *
* searches how to update a component prop *
* nothing found *
* registers on devrant and verifies his email *
* writes this rant *

Wow, what a fucking mess this sunday was.
My boss wrote me an email that one route of a RESTful API we wrote for a customer was not working anymore and puking back a status 500 with some error mentioning invalid UTF-8 characters.
Not one single person has had touched nor changed the code on production in some 6 months, so what the fuck could it be?

Phpunit did not give any errors (running only locally), the code had no syntax errors and the DB dump did not contain any invalid bytes (tested with a hex editor).

WHAT THE FUCK?!

OK so I started to comment out lines (all tested directly on production of course) until the error vanished.

Guess what was the culprit?
.
.
.
.
.
.
In the code (PHP) we used strftime(...) to get nice time strings. Of course we set the correct locale on the server, thus having months and days formatted in German.
So, in Geman there is this one mysterious month called "März" which contains an umlaut character.
Calling strftime generated the date with März in it, but the server locale was de_CH.iso-8859-1 and not fucking de_CH.utf8, so the "ä" was returned as 0xE4 instead of 0xC3A4 (valid UTF-8), which json_encode(...) did not want to swallow but instead threw an exception.

So I said I'd rant this yesterday but a long night of server management came in the way!

Yesterday @trogus mentioned in a comment that he thinks everyone deserves a place where they feel like home and this is that place for me along with some sub-places which derived from here.

So in this linux/foss chat yesterday I was trying to get into an IRC chatroom (all people there (or at least a lot) are also like minded on privacy/security). I don't want to use email signup if not absolutely neccesary (don't judge me, everyone there own thing) and I found out very late (after 20 minutes of instructions from a fellow devRanter) that this thing required email signup. I didn't wanna do that so I said that and started typing a whole essay of why I'd rather not do that and what my reasons are (privacy partly) but then the guy said: "haha you got it man".

For one second I forgot that I don't have to explain myself over there on stuff regarding privacy that a lot of people would find paranoid. Man, that feels like being home :).

I'm not angry, mostly sad.

At my workplace we don't use git.
There are constant overwriting, sending code via email or USB stick and forgetting passwords to zip-files shenanigans going on.

I already use git for all my local projects (literally git init in the directory) but my coworker and I thought that it would be a great idea to have a local server with a Gitlab running on it.
So I started looking into running a self-hosted Gitlab (for about 15 minutes) and then our boss who was sitting right next to me almost shouted at us: "Such stuff should be coordinated with the boss! We don't just do something and burn my money because it's _cool_!"

No, git is not cool, it's necessary for crying out loud! Gitlab is cool but at the end of the day also just another tool too.

I guess I have some persuasion to do.

I don't know what version control has done to our boss that he has such a deep dislike for it.

Ok story of my most most recent job search (not sure devRant could handle the load if I was to go through them all)

First a little backstory on why I needed to search for a new job:

Joined a small startup in the blockchain space. They were funded through grants from a non-profit setup by the folks who invented the blockchain and raised funds (they gave those funds out to companies willing to build the various pieces of the network and tools).

We were one of a handful of companies working on the early stages of the network. We built numerous "first"s on the network and spent the majority of our time finding bugs and issues and asking others to fix them so it would become possible, for us to do what we signed up for. We ended up having to build multiple server side applications as middleware to plug massive gaps. All going great, had a lot of success, were told face to face by the foundation not to worry about securing more funds at least for the near term as we were "critical to the success of the network".

1 month later a bug was discovered in our major product, was nasty and we had to take it offline. Nobody lost any funds.

1-2 months later again, the inventor of the blockchain (His majesty, Lord dickhead of cuntinstein) decided to join the foundation as he wasn't happy with the orgs progress and where the network now stood. Immediately says "see that small startup over there ... yeah I hate them. Blackball them from getting anymore money. Use them as an example to others that we are not afraid to cut funds if you fuck up"

Our CEO was informed. He asked for meetings with numerous people, including His royal highness, lord cockbag of never-wrong. The others told our CEO that they didn't agree with the decision, but their hands were tied and they were deeply sorry. Our CEO's pleas with The ghost of Christmas cuntyness, just fell on deaf ears.

CEO broke the news to us, he had 3 weeks of funds left to pay salaries. He'd pay us to keep things going and do whatever we could to reduce server costs, so we could leave everything up long enough for our users to migrate elsewhere. We reduced costs a lot by turning off non essential features, he gave us our last pay check and some great referrals. That was that and we very emotionally closed up shop.

When news got out, we then had to defend ourselves publicly, because the loch ness moron, decided to twist things in his favour. So yeah, AMAZING experience!

So an unemployed and broken man, I did the unthinkable ... I set my linkedin to "open to work". Fuck me every moronic recruiter in a 10,000 mile radius came after me. Didn't matter if I was qualified, didn't matter if I had no experience in that language or type of system, didn't matter if my bio explicitly said "I don't work with X, Y or Z" ... that only made them want me more.

I think I got somewhere around 20 - 30 messages per week, 1 - 2 being actually relevant to what I do. Applied to dozens of jobs myself, only contacted back by 1, who badly fucked up the job description and I wasn't a fit at all.

Got an email from company ABC, who worked on the same blockchain we got kicked off of. They were looking for people with my skills and the skills of one other dev in the preious company. They heard what happened and our CEO gave us a glowing recommendation. They largely offered us the job, but both of us said that we weren't interested in working anywhere near, that kick needing prick, again. We wanted to go elsewhere.

Went back to searching, finding nothing. The other dev got a contract job elsewhere. The guy from ABC message me again to say look, we understand your issues, you got fucked around. We can do out best to promise you'll never have to speak to, the abominable jizz stain, again. We'll also offer you a much bigger role, and a decent salary bump on top of that.

Told them i'd think about it. We ended up having a few more calls where they showed me designs of all the things they wanted to do, and plans on how they would raise money if the same thing was to ever happen to them. Eventually I gave in and signed up.

So far it was absolutely the right call. Haven't had to speak to the scrotum at all. The company is run entirely by engineers. Theres no 14 meetings per week to discuss "where we are" which just involves reading our planning tool tickets, out loud. I'm currently being left alone 99% of the week to get work done. and i'm largely in-charge of everything mobile. It was a fucking hellhole of a trip, but I came out the other side better off

I'm sure there is a thought provoking, meaningful quote I could be writing now about how "things always work out" or that crap. But remembering it all just leaves me with the desire to find him and shove a cactus where the sun don't shine

.... happy job hunting everyone!

When I was 10 years old, all kids at my school got access to school emails. The email address book contained everything in my city, like fire department, all pupils of all schools etc. So I decided to "test" the system by sending out a mass email to everyone in the address book (about 3k) with the question "Hi, how are you?".

The sys admins apparently didn't think very far as I got some responses saying like "You have crashed a server in the capital city" and "I have contacted your local IT admin".

So I went to the IT admin and told him the situation. His face turned red of anger and I remember him almost screaming at me.

Who the fuck doesn't set up protection for this and gives out access to 10 year olds? This was 15 years ago, I really hope sys admins are smarter today!

So...
I'm penetrationtesting a network and the servers on said network
The network administrator and IT security officer knows this, because they hired me..

TL;DR a scan caused the network to crash.

Today I received a very angry email going "Stop scanning NOW!" from one of the IT departments.

Apparently I crashed their login server and thus their entire network...
It happened d the first time I scanned the network from the outside and they had spend an entire day figuring out how and repairing the service they thought was the problem, but then it crashed again, when I scanned from within the network.

Now they want to send me a list of IP's that I'm not allowed to scan and want to know exactly what and when I'm scanning...

How crap can they be at their job, if they weren't able to spot a scan... The only reason they found out it was me was because the NA had whitelistet my IP, so that I could scan in peace...

This just in everyone...

Android Dev: *sent and email to network admin* can you please unblock github for a few mins.

Network admin: *Replied* Can you take a screenshot whats the error your getting.

Android Dev: *Replied with screenshot* "Failed to load resource: the server responded with a status of 503 (Service Unavailable)"

Network admin: that is a known issue. *Replied with Wordpress Links.

Android Dev: why is github working outside our network then?

Network admin: there must be a problem with your code that needs to be tweaked.

Team: *FACE PALM*

Some days I feel like I work in a different universe.

Last night our alerting system sent out a dept. wide email regarding a high number of errors coming from the web site.

Email shows the number of errors and a summary of the error messages.

Ex. 60 errors
59 Object reference not set to an instance of an object
1 The remote server returned an unexpected response: (413) Request Entity Too Large

Web team responds to the email..

"Order processing team's service is returning a 413 error. I'll fill out a corrective action ticket in the morning to address that error in their service. "

Those tickets are taken pretty seriously by upper mgmt, so I thought someone on the order processing team would point out the 1 error vs. 59 (coming from the web team's code).

Two hours go by, nobody responds, so I decide to jump into something that was none of my business.

"Am I missing something? Can everyone see the 59 null reference exceptions? The 413 exception only occurred once. It was the null reference exceptions that triggered the alert. Looking back at the logs, the site has been bleeding null reference exceptions for hours. Not enough for an alert, but there appears to be a bug that needs to be looked into."

After a dept. managers meeting this morning:

MyBoss: "Whoa..you kicked the hornets nest with your response last night."
Me: "Good. What happened?"
<Dan dept VP, Jake web dept mgr>
MyBoss: "Dan asked Jake if they were going to fix the null reference exceptions and Jake got pissed. Said the null reference errors were caused by the 413 error."
Me: "How does he know that? They don't log any stack traces. I don't think those two systems don't even talk to one another."
<boss laughs>
MyBoss:"That's what Dan asked!..oh..then Jake started in on the alert thresholds were too low, and we need to look into fixing your alerting code."
Me: "What!? Good Lord, tell me you chimed in."
MyBoss: "Didn't have to. Dan starting laughing and said there better be a ticket submitted on their service within the next hour. Then Jake walked out of the meeting. Oh boy, he was pissed."
Me: "I don't understand how they operate over there. It's a different universe.
MyBoss: "Since the alert was for their system, nobody looked at the details. I know I didn't. If you didn't respond pointing out the real problem, they would have passed the buck to the other team and wasted hours chasing a non-existent problem. Now they have to take resources away from their main project and answer to the VP for the delay. I'm sure they are prefixing your name right now with 'that asshole'"
Me: "Not the first, won't be the last."

Quick recap of my last two weeks: 15 year old production server is basically dead, boss has taken over calls and claims credit for "resolving" outages (even though my coworker and I did the work, but ultimately the traffic died down enough to where it wasn't an issue anymore).

I go to a meeting to plan migration to a better server, boss bitches about not getting invited, I tell him I invited myself, and then he lectures about how that's not our job.

Different boss says we're migrating a schema for an application that should have been decommissioned 5+ years ago to use as a baseline. I explain what's going on, he says he understands, and proceeds to tell higher bosses it's perfect because there will be no user impact. OF COURSE THERE'S NO FRICKING IMPACT, YA DUNCE! there are no users!!!!

I merge two email threads together, since they discuss the same thing, but with different insight, and get yelled at, even though they requested it.

The two bosses I like are OOO for the next week, too, so I'm just sitting here hoping I don't say something that'll get me fired or sent to sensitivity training.

I'm just starting my on call rotation and don't know that I can do this. I cry when my phone rings, now, because I experience physical pain with how hard I cringe.

I got yelled at today by a guy because SOMEONE I DON'T KNOW assigned a ticket to him directly, rather than to the proper team (not his team). So I had to look into that, which at least had the benefit of preventing a catastrophic outage to our customers world wide, but no one will know because I don't brag at work; I'm too busy doing my job as well as most of my division/section/larger team, whatever the hell it's called. I saved us probably 25+ hours of continuous troubleshooting call from noticing something tiny that the people "smarter" than me missed.

**edit: sorry for typos; got my nails done yesterday but they feel like they're a mile long and I have to relearn how to type**

I was working as a contractor for a client who just got enough funding to hire a full-time dev. I lovingly referred to him as "Mr. Koolaid" because he was obsessed with whatever the newest hotness was and cried constantly about how the 3-year-old code-base didn't use The Next Big Thing(tm). This was my first interaction with him:

Mr. Koolaid: I'd like access to the github repository. My username is xxxx.

Me: We currently aren't hosting the code on github. If you send me your public ssh key, I'll get you access to the private server.

Mr. Koolaid: I'd like to access the github repository.

Me: It's not on github; send me your public key and I'll get you access.

Mr. Koolaid: Can we skype real quick? You don't need my public key to grant me access to the github repo.

*Mr. Koolaid proceeds to forward me github's documentation on adding users to an organization and the documentation for adding users to a private repo. The email is written in a very passive-aggressive tone.*

ಠ_ಠ

*wrestling commentator voice*
"In this weeks episode of encoding hell:
The iiiinnnfamous UTF-8 Byte Order Mark veeeersus PHP!"

For an online shop we developed, there is currently a CSV upload feature in review by our client. Before we developed this feature, we created together with the client a very precise specification, including the file format and encoding (UTF-8).

After the first test day, the client informed us, that there were invalid characters after processing the uploaded file.
We checked the code and compared the customer's file with our template.
The file was encoded in ISO-8859-1 and NOT as specified UTF-8.
But what ever, we had to add an encoding check, thus allowing both encodings from now on.

Well well well welly welly fucking well...

Test day 2: We receive an email from said client, that the CSV is not working, again.
This time: UTF-8 encoding, but some fields had more colums with different values than specified.
Fucking hell.
We tell the customer that.
(I was about to write a nice death threat novel to them, but my boss held me back)

Testing day 3, today:
"The uploading feature is not working with our file, please fix it."
I tried to debug it, but only got misleading errors. After about 30 minutes, at 20 stacks of hatered, I finally had an idea to check the file in a hex editor:

God fucking what!?!!?!11?!1!!!?2!!

The encoding was valid UTF-8, all columns and fields were correct, but this time the file contained somthing different.
Something the world does not need.
Something nearly as wasteful as driving a monster truck in first gear from NYC to LA.

It was the UTF-8 Byte Order Mark.
3 bytes of pure hell.
Fucking 0xEFBBBF.
The archenemy of PHP and sane people.

If the devil had sex with the ethernet port of a rusty Mac OS X Server, then 9 microseconds later a UTF-8 BOM would have been born.

OK, maybe if PHP would actually cope with these bytes of death without crashing, that would be great.

Yesterday the web site started logging an exception “A task was canceled” when making a http call using the .Net HTTPClient class (site calling a REST service).

Emails back n’ forth ..blaming the database…blaming the network..then a senior web developer blamed the logging (the system I’m responsible for).

Under the hood, the logger is sending the exception data to another REST service (which sends emails, generates reports etc.) which I had to quickly re-direct the discussion because if we’re seeing the exception email, the logging didn’t cause the exception, it’s just reporting it. Felt a little sad having to explain it to other IT professionals, but everyone seemed to agree and focused on the server resources.

Last night I get a call about the exceptions occurring again in much larger numbers (from 100 to over 5,000 within a few minutes). I log in, add myself to the large skype group chat going on just to catch the same senior web developer say …
“Here is the APM data that shows logging is causing the http tasks to get canceled.”

FRACK!

Me: “No, that data just shows the logging http traffic of the exception. The exception is occurring before any logging is executed. The task is either being canceled due to a network time out or IIS is running out of threads. The web site is failing to execute the http call to the REST service.”

Several other devs, DBAs, and network admins agree.

The errors only lasted a couple of minutes (exactly 2 minutes, which seemed odd), so everyone agrees to dig into the data further in the morning.

This morning I login to my computer to discover the error(s) occurred again at 6:20AM and an email from the senior web developer saying we (my mgr, her mgr, network admins, DBAs, etc) need to discuss changes to the logging system to prevent this problem from negatively affecting the customer experience...blah blah blah.

FRACKing female dog!

Good news is we never had the meeting. When the senior web dev manager came in, he cancelled the meeting.

Turned out to be a hiccup in a domain controller causing the servers to lose their connection to each other for 2 minutes (1-minute timeout, 1 minute to fully re-sync). The exact two-minute burst of errors explained (and proven via wireshark).

People and their petty office politics piss me off.

So my friend, who owns a restaurant, asked me over 6 months ago, if i could redesign his homepage. I told him "sure why not" and since we're friends i didn't want him to pay me any money.

He told me what his thoughts about the design were and i told him that i needed the menu, some decent pictures of the restaurant, the "about us" story and the credentials to the server.

He didn't know the credentials to his server and i told him to ask the person, who made that page to send me the information i needed, but he kept on saying "could you call her because blah blah". Well, i did but she couldn't give me that info without asking the owner. So i met him and told him "hey i told you so, because it's completely normal not give sensible information to unknown people and besides that she told me to tell you that you should give her a call, because she hasn't got your new phone number". Two months later i got an email with the credentials, but still no menu and no pictures.

Four days ago i made a transition page, because i didn't want to publish the page with stock images and without menu, so i wrote him again whether he wanted design #1 or #2. Got a text at ~21:00 saying "design 2, but you need to publish it at 22:00".

I mean wtf?! He assured me he would call some people he knows to get those things. I told him, that it would be free, because of our friendship, but no support from him and he keeps stressing?! He knows i've got a full-time job and my studies going on, so my time is really limited and he keeps fking around like that?! Man it pisses me really off...

So we hired an intern and his first task was to change a few things in email layout for our client, which is an investment bank.

I told to one of my developers to make his local database dump and setup the project for an intern. When intern completed the task, my developer thought that title "Dow Jones index crashed" was pretty funny title for a test.

What he didn't thought through enough, is that he forgot to configure fake SMTP server and he had production database dump with real email addresses.

I had really awkward 20 minutes conversation with our client. Fuck my life.

I absolutely love the email protocols.

IMAP:
x1 LOGIN user@domain password
x2 LIST "" "*"
x3 SELECT Inbox
x4 LOGOUT

Because a state machine is clearly too hard to implement in server software, clients must instead do the state machine thing and therefore it must be in the IMAP protocol.

SMTP:
I should be careful with this one since there's already more than enough spam on the interwebs, and it's a good thing that the "developers" of these email bombers don't know jack shit about the protocol. But suffice it to say that much like on a real letter, you have an envelope and a letter inside. You know these envelopes with a transparent window so you can print the address information on the letter? Or the "regular" envelopes where you write it on the envelope itself?
Yeah not with SMTP. Both your envelope and your letter have them, and they can be different. That's why you can have an email in your inbox that seemingly came from yourself. The mail server only checks for the envelope headers, and as long as everything checks out domain-wise and such, it will be accepted. Then the mail client checks the headers in the letter itself, the data field as far as the mail server is concerned (and it doesn't look at it). Can be something else, can be nothing at all. Emails can even be sent in the future or the past.

Postfix' main.cf:
You have this property "mynetworks" in /etc/postfix/main.cf where you'd imagine you put your own networks in, right? I dunno, to let Postfix discover what your networks are.. like it says on the tin? Haha, nope. This is a property that defines which networks are allowed no authentication at all to the mail server, and that is exactly what makes an open relay an open relay. If any one of the addresses in your networks (such as a gateway, every network has one) is also where your SMTP traffic flows into the mail server from, congrats the whole internet can now send through your mail server without authentication. And all because it was part of "your networks".

Yeah when it comes to naming things, the protocol designers sure have room for improvement... And fuck email.

Oh, bonus one - STARTTLS:
So SMTP has this thing called STARTTLS where you can.. unlike mynetworks, actually starts a TLS connection like it says on the tin. The problem is that almost every mail server uses self-signed certificates so they're basically meaningless. You don't have a chain of trust. Also not everyone supports it *cough* government *cough*, so if you want to send email to those servers, your TLS policy must be opportunistic, not enforced. And as an icing on the cake, if anything is wrong with the TLS connection (such as an MITM attack), the protocol will actively downgrade to plain. I dunno.. isn't that exactly what the MITM attacker wants? Yeah, great design right there. Are the designers of the email protocols fucking retarded?

The nightmare continues.

Currently dealing with a code review from a “principal” dev (one step above senior), who is unironically called a “legendary dev” by some coworkers. It’s painfully obvious he didn’t read the code, and just started complaining and nitpicking.

It’s full of requests to do things that make absolutely no sense, and would make the code an unmaintainable mess.
• Ex: moving the logic and data collection from the module’s many callers into the module instead of just passing in the data.
• Ex: hiding api endpoint declarations by placing them in the module itself, and using magic instance variables to pass data to it. Basically: using global functions and variables instead of explicit declarations and calls.
• Ex: moving the logic to determine which api endpoint to use, for all callers, into the view.

More comments about methods being “too complex” (barely holds water) right next to comments saying “why are these separate? merge them together!”

Incredulously asking how many times I’m checking permissions and how ridiculous it all is. (The answer? Twice.)

Conflating my “permissions” param and method names with a supposedly forthcoming permissions system overhaul, and saying I shouldn’t use permissions because my code will all have to get rewritten. Even if that were true, and it’s likely not, the ticket still needs to use the current permissions. I can’t just ignore them because they might be rewritten someday.

Requests to revert some code cleanup because the reviewer thought the previous heavily-nested and uncommented versions (with code duplication) were easier to read. Unsurprisingly, he wrote them.

On the same ticket, my boss wants me to remove all styling and clientside validation, debouncing, and error messages from a form. Says “success” and “connection failed” messages are good enough. The form in question sends SMS and email using arbitrary user input for addresses. He also says it shouldn’t be denounced on the server, and doesn’t want me to bother checking permissions. Hello, spam!

Related: the legendary dev reviewer says he can’t think of a reason why we would want to disable the feature for consumers, so I should remove the consumer feature flag.

You can’t make this stuff up.

So I used to do some freelancing in web development last year, nothing too fancy just some simple PHP websites. Comes the worst meeting in my life. So I am from India and we have a lot of long lasting business here being passed on over generations. TL;DR the guy was the owner of a very old business which was actually very huge and the guy was educated too, so I assumed that he'll be sensible as compared to other people.

The meeting was in an expensive cafe and he paid for it, he even told me upfront that meeting is on him. Great, right? So we sit down, order some coffee and then start discussing what he needed.
The guy needed an ecommerce website built with backend and logistics system integrated. We discussed possible designs for the website and stuff too and so far the deal looked promising to both of us.
I explained him the cost estimate and told him that I would email him the final quote from myself once we discussed server cost and shit.
So now comes the bargaining part where he asked me to give him server and domain for free.
At this point, I suspected that he didn't know that servers and domains are not something that you make. You have to purchase and renew them periodically.
So I told that guy that he didn't understand the cost estimation and explained to him that X is the cost of making this fucking thing and Y is its monthly maintenance cost, if he wanted annually could be done too. And this Y did not include server and domain costing.
Now came the fucking tide, the guy straight up turned to his shit and told me I am lying and trying to con him. So I gently asked him if he had ever gotten any website made. To which, he said No, but he knows how the costing works.
I was like "Bitch?". So I calmly tried to explain that that's not how websites are done, delivered and maintained.
He didn't seemed to be understanding and kept on fucking repeating that he knows his shit and blah blah.
At this point, I was like "Okay. Fuck this dude then. I can find another project. " and then I told him that he'll need to find someone according to his needs.
Interestingly enough, the guy called someone and then walked out of the cafe while talking on phone. I waited for 5 minutes and he didn't come back so I decided I would pay for my coffee and leave. Turns out the guy had paid his bill before my arrival and ditched me with the excuse of the call.

But oh well, I think working with such an idiot would have been much worse than paying for that coffee.

This happened quite sometime ago.

I received a client, reputable university in my country. After all the paper work was done, I was emailed access to one of their AWS server, FTP where the username and password were both admin. I didn't say much to them at that moment.. Maybe they had some precautions?

Over night I received another email, around 3am,

"Hi Uzair, we've monitored a breach while leaving FTP access open."

Well, that was sorta expected.
I received SFTP access to the server the following day,

username: admin,
password: @dmin

When I was in college, our email was on a Unix server. We would login via serial connection or telnet over the network, and get a korn shell. The server was poorly secured. Everyone's login device was world writable. So people would just see who was online, see the username of someone they wanted to mess with, take note of the pts(network) or tty(serial) device their connection used, and cat ASCII penises to it.

cat animated_dong.txt > /dev/pts/4

It was a simpler time.

I remember that time, when a pretty big customer bought a tiny server from us and proceeded with setting up their tiny website.

Everything went smooth a smoothsauce until that day they decided to send a newsletter to 3 million subscribers - a complete nice email with alot of html and elements that was hosted on that tiny server.
The fact that most phones do download the emails with the attachment (pictures, html and shit) when the email arrive did they not know about. But sure - they atleast learnt something that day.

That shitty server died.

One Thursday noon,
operation manager: (looking at mobile)what the.....something is wrong i am getting bunch of emails about orders getting confirmed.
Colleague dev: (checks the main email where it gets all email sent/received) holy shit all of our clients getting confirmation email for orders which were already cancelled/incomplete.
Me: imediately contacting bluehost support, asking them to down the server so just that we can stopp it, 600+ emails were already sent and people keep getting it.
*calls head of IT* telling the situation because he's not in the office atm.

CEO: wtf is happening with my business, is it a hacker?

*so we have a intrusion somebody messed the site with a script or something*

All of us(dev) sits on the code finding the vulnerabilities , trying to track the issue that how somebody was able to do that.

*After an hour*

So we have gone through almost easch function written in the code which could possibly cause that but unable to find anything which could break it.

Head asking op when did you started getting it actually?
Op: right after 12 pm.

*an other hour passes*

Head: (checking the logs) so right after the last commit, site got updated too?. And....and.....wtf what da hell who wrote this shit in last commit?
* this fuckin query is missing damn where clause* 🤬

Me: me 😰

*long pause, everyone looking at me and i couldn't look at anyone*
The shame and me that how can i do that.

Head: so its you not any intrudor 😡
Further investigating, what the holy mother of #_/&;=568 why cronjob doesn't check how old the order is. Why why why.

(So basically this happened, because of that query all cancelled/incomplete orders got updated damage done already, helping it the cronjob running on all of them sending clients email and with that function some other values got updated too, inshort the whole db is fucked up.)
and now they know who did it as well.

*Head after some time cooling down, asked me the solution for the mess i create*

Me: i took backup just couple of days before i can restore that with a script and can do manual stuff for the recent 2 days. ( operation manager was already calling people and apologising from our side )

Head: okay do it now.

Me: *in panic* wrote a script to restore the records ( checking what i wrote 100000000 times now ), ran...tested...all working...restored the data.
after that wrote an apology email, because of me staff had to work alot and it becomes so hectic just because of me.

* at the end of the day CEO, head, staff accepted apology and asked me to be careful next time, so it actually teached me a lesson and i always always try to be more careful now especially with quries. People are really good here so that's how it goes* 🙂

DevOps required skillset:

* Frontend engineering
* Backend services
* Database administrator
* Security consultant
* Project management
* 3rd party contract negotiator
* Build system monitor
* Build system hostage negotiator
* Paging, alerting, monitoring
* Search server admin
* Old search server admin
* Old-old-new search server admin
* Redis, ElasticSearch, MySQL, PostGres, owner
* Agile coach
* No you shouldn't do that coach
* Oh, you did that anyway coach
* DNS: (Optional) It'll replicate when it wants, and how it wants to to anyway
* Multi-Cloud deployment strategist
* Must be able to translate Klingon to YAML, and YAML to MySQL
* Cost analyzer, reducer, and justifier
* Complex documentation generation in markdown that we should have done years ago anyway
* Marketing's email went to spam analyzer
* Wordpress is broke fixer
* Where the fuck does Wordpress run anyway?
* Ability to fix MySql running Wordpress on marketing's dusty laptop

Received a urgent email from a business client saying that the application we support is completely broken. Their staff said they used the app to send several submissions that day but they did not come through. This is a major issue as these submissions need to occur daily.

I understand that this is a priority so I immediately check everything. I test the app, the server, check the database. Everything seems fine, but there's no record of these submissions. Maybe it's the specific device that was used. I reply saying that everything seems to be in order. Can I please be provided with more information about what occurred? What time were the submissions sent?

Client replies saying that the submissions were definitely sent and that the staff swear by it.

I now know something is up, so I remote into the the devices in question and check the logs. The app was not even used that day! I've got them! Those liars!

I am now quite pissed off, but remain professional and reply saying that we log all app events and that the logs show that the app had not been used at all that day. Now they have to own up to their lie. Right?

Wrong. Client replies with: The issue has been fixed. Thanks.

Can you believe the bloody nerve? The client doesn't even have the decency to apologise but rather insinuates that it was all our fault.

Well I'm not having that. I reply: It is great that the app is functioning correctly. However, I believe it is important to understand the cause of the issue as to prevent it from occuring again.

Client: No reply.

Well, if you want to waste other people's time, here's the fat bill.

Moral of the story. Don't trust anything that the client says and for any issue, debug the user before doing anything else.

Long rant ahead.. so feel free to refill your cup of coffee and have a seat 🙂

It's completely useless. At least in the school I went to, the teachers were worse than useless. It's a bit of an old story that I've told quite a few times already, but I had a dispute with said teachers at some point after which I wasn't able nor willing to fully do the classes anymore.

So, just to set the stage.. le me, die-hard Linux user, and reasonably initiated in networking and security already, to the point that I really only needed half an ear to follow along with the classes, while most of the time I was just working on my own servers to pass the time instead. I noticed that the Moodle website that the school was using to do a big chunk of the course material with, wasn't TLS-secured. So whenever the class begins and everyone logs in to the Moodle website..? Yeah.. it wouldn't be hard for anyone in that class to steal everyone else's credentials, including the teacher's (as they were using the same network).

So I brought it up a few times in the first year, teacher was like "yeah yeah we'll do it at some point". Shortly before summer break I took the security teacher aside after class and mentioned it another time - please please take the opportunity to do it during summer break.

Coming back in September.. nothing happened. Maybe I needed to bring in more evidence that this is a serious issue, so I asked the security teacher: can I make a proper PoC using my machines in my home network to steal the credentials of my own Moodle account and mail a screencast to you as a private disclosure? She said "yeah sure, that's fine".
Pro tip: make the people involved sign a written contract for this!!! It'll cover your ass when they decide to be dicks.. which spoiler alert, these teachers decided they wanted to be.

So I made the PoC, mailed it to them, yada yada yada... Soon after, next class, and I noticed that my VPN server was blocked. Now I used my personal VPN server at the time mostly to access a file server at home to securely fetch documents I needed in class, without having to carry an external hard drive with me all the time. However it was also used for gateway redirection (i.e. the main purpose of commercial VPN's, le new IP for "le onenumity"). I mean for example, if some douche in that class would've decided to ARP poison the network and steal credentials, my VPN connection would've prevented that.. it was a decent workaround. But now it's for some reason causing Moodle to throw some type of 403.

Asked the teacher for routers and switches I had a class from at the time.. why is my VPN server blocked? He replied with the statement that "yeah we blocked it because you can bypass the firewall with that and watch porn in class".
Alright, fair enough. I can indeed bypass the firewall with that. But watch porn.. in class? I mean I'm a bit of an exhibitionist too, but in a fucking class!? And why right after that PoC, while I've been using that VPN connection for over a year?

Not too long after that, I prematurely left that class out of sheer frustration (I remember browsing devRant with the intent to write about it while the teacher was watching 😂), and left while looking that teacher dead in the eyes.. and never have I been that cold to someone while calling them a fucking idiot.

Shortly after I've also received an email from them in which they stated that they wanted compensation for "the disruption of good service". They actually thought that I had hacked into their servers. Security teachers, ostensibly technical people, if I may add. Never seen anyone more incompetent than those 3 motherfuckers that plotted against me to save their own asses for making such a shitty infrastructure. Regarding that mail, I not so friendly replied to them that they could settle it in court if they wanted to.. but that I already knew who would win that case. Haven't heard of them since.

So yeah. That's why I regard those expensive shitty pieces of paper as such. The only thing they prove is that someone somewhere with some unknown degree of competence confirms that you know something. I think there's far too many unknowns in there.

Nowadays I'm putting my bets on a certification from the Linux Professional Institute - a renowned and well-regarded certification body in sysadmin. Last February at FOSDEM I did half of the LPIC-1 certification exam, next year I'll do the other half. With the amount of reputation the LPI has behind it, I believe that's a far better route to go with than some random school somewhere.

Paranoid Developers - It's a long one
Backstory: I was a freelance web developer when I managed to land a place on a cyber security program with who I consider to be the world leaders in the field (details deliberately withheld; who's paranoid now?). Other than the basic security practices of web dev, my experience with Cyber was limited to the OU introduction course, so I was wholly unprepared for the level of, occasionally hysterical, paranoia that my fellow cohort seemed to perpetually live in. The following is a collection of stories from several of these people, because if I only wrote about one they would accuse me of providing too much data allowing an attacker to aggregate and steal their identity. They do use devrant so if you're reading this, know that I love you and that something is wrong with you.

That time when...

He wrote a social media network with end-to-end encryption before it was cool.

He wrote custom 64kb encryption for his academic HDD.

He removed the 3 HDD from his desktop and stored them in a safe, whenever he left the house.

He set up a pfsense virtualbox with a firewall policy to block the port the student monitoring software used (effectively rendering it useless and definitely in breach of the IT policy).

He used only hashes of passwords as passwords (which isn't actually good).

He kept a drill on the desk ready to destroy his HDD at a moments notice.

He started developing a device to drill through his HDD when he pushed a button. May or may not have finished it.

He set up a new email account for each individual online service.

He hosted a website from his own home server so he didn't have to host the files elsewhere (which is just awful for home network security).

He unplugged the home router and began scanning his devices and manually searching through the process list when his music stopped playing on the laptop several times (turns out he had a wobbly spacebar and the shaking washing machine provided enough jittering for a button press).

He brought his own privacy screen to work (remember, this is a security place, with like background checks and all sorts).

He gave his C programming coursework (a simple messaging program) 2048 bit encryption, which was not required.

He wrote a custom encryption for his other C programming coursework as well as writing out the enigma encryption because there was no library, again not required.

He bought a burner phone to visit the capital city.

He bought a burner phone whenever he left his hometown come to think of it.

He bought a smartphone online, wiped it and installed new firmware (it was Chinese; I'm not saying anything about the Chinese, you're the one thinking it).

He bought a smartphone and installed Kali Linux NetHunter so he could test WiFi networks he connected to before using them on his personal device.

(You might be noticing it's all he's. Maybe it is, maybe it isn't).

He ate a sim card.

He brought a balaclava to pentesting training (it was pretty meme).

He printed out his source code as a manual read-only method.

He made a rule on his academic email to block incoming mail from the academic body (to be fair this is a good spam policy).

He withdraws money from a different cashpoint everytime to avoid patterns in his behaviour (the irony).

He reported someone for hacking the centre's network when they built their own website for practice using XAMMP.

I'm going to stop there. I could tell you so many more stories about these guys, some about them being paranoid and some about the stupid antics Cyber Security and Information Assurance students get up to. Well done for making it this far. Hope you enjoyed it.

So I had a fun week.

It started off with my boss replying to a co-workers email where he sent his new bank account, saying he doesn't need it untill we close off some baddly planned projects, meaning no paycheck.

Needless to say we were working night and days including weekends on it and put our best into it.

For the next part I need to explain a little background. We have this old legacy system I'm working with for the past 3 years. I keet raising the red flag we need a new one. Nothing happened. So every time I worked with it I kept thinking how to improve the parts. Almost two years went into thinking and planning the new system untill I got a green light. It was most satisfying - the day I got to build something good and awesome. I drew all the data structures, laid out the foundations and started building ontop of it. It was amazing and I was really proud of it. Then suddendly client wanted to see something and the decision was made we threw it together quickly with the old legacy system. It was on hold 'till then due to work overload.

Boss wrote me this week if I can put the project from git on a server, where he out sourced the completition into India where they will finish it. On thr question if they can't work on git, he replied: "should they?" -.-

To top it all up, I got a notice at the end of the week if I don't fill his shit time tracking system (that takes me one hour/day to insert all entries) by monday he'll deduct a sizable portion of my paycheck.

I AM WORKING FOR YOU ALL THE FUCKING TIME BECAUSE YOU LACK RESOURCES AND I THOUGHT A TEAM STICKS TOGETHER AND SAVES EACH OTHERS ASS! I DONT HAVE TIME TO ENTER YOUR FUCKING STUPID TIME ENTRIES IN YOUR FUCKING BUGGED SYSTEM EACH DAY ON TASKS THAT DON'T EVEN EXIST BITCH! MAKE IT BETTER FIRST!! OH! AND NO ONE IS MORE QUALIFIED TO FINISH THAT PROJECT THAN ME, I POURED MY FUCKING HEART INTO IT YOU PRICK!

woah.

Ordered a new high storage dedicated server (rented online) today.

The fucker wasn't delivered as soon as I hoped so decided to finally go to bed at 2am.

Aaaaaand then, of fucking-course, I got the email with the deployment notification literally the second before I went to sleep.

No sleep yet anyways, I guess 😅

Spent 20 minutes on the phone to our server center in india spelling letter for letter who I was and what the issue is.
Finally she says: "Send me an email."
Me: "I did, you are calling me remember?"

That feeling when your client connection is more stable than the connection of a fucking game server... Incompetent pieces of shit!!! BEING ABLE TO PUT A COUPLE OF SPRITES DOESN'T MAKE YOU A FUCKING SYSADMIN!!!

Oh and I sent those very incompetent fucks a mail earlier, because my mailers are blocking their servers as per my mailers' security policy. A rant from the old box - their mail servers self-identify a fucking .local!!! Those incompetent shitheads didn't even properly change the values from test into those from prod!! So I sent them an email telling them exactly how they should fix it, as I am running the same MTA on my mailers (Postfix), at some point had to fix my mailers against the exact same issue as well, and clearly noticed in-game that they have deliverability problems (they explicitly mention to unblock their domain). Guess why?! Because their server's shitty configuration triggers fucking security mechanisms that are built against rogue mailers that attempt to spoof themselves as an internal mailer, with that fucking .local! And they STILL DIDN'T CHANGE IT!!!! Your fucking domain has no issues whatsoever, it's your goddamn fucking mail servers that YOU ASOBIMO FUCKERS SHOULD JUST FIX ALREADY!!! MOTHERFUCKERS!!!!!

Trying to setup server monitoring for one server (will deploy it on all my servers once this works well).

Getting these email notifs to work is a bitch 😅

(can't do pushover etc since those require GCM and I've got Google blocked out of my phone)

Me to customer, several years ago:
Your server is over 10 years old, and the support for it will soon be gone together with the possibility to take backups. You should upgrade your website ASAP.

Customer said:
ok, we will look into it.

This day:
Customer:
Hi, our site is down and we need backups from yesterday. Can you please fix?

Me:
Sorry, no. *send copy of the email several years ago*

I sometimes remember the time when I wrote a Email-inbox-exporter-PHP-script-type of application that collects all the emails from an inbox, "copied" it to a database with the attachements and stuff and moves it to a folder..

I just started at the company for like a couple of months, had no privileges to create mailboxes and such and I didn't want to interrupt our programmer to do this for me, so... I decided.. to save time and resources.. to test run it on our global, live 'support' mailbox.. :D Well.. You might guess what happened.. Apparently I mistyped the name of the move-destination folder (because imap-weird-things) that resulted in a completly empty mailbox and an empty database because the inserts failed due to bad encoding and mime-type issues..

The moment I refreshed my Outlook and noticed that all our mails where gone.. I swear, I can't describe that feeling of fear, cold sweat, intense heartbeat... I just stood up, asked if anyone wanted coffee, and just walked out of the office.. When in the hallway, I heard my collegues ask to one another "do you have any issues with outlook, all my mails are gone?". Everyone was stressing out, the chief was stressing out "what happened?!", nobody knew what happened.. :D

They could partially resolve it via one collegue who hadn't refreshed the mailbox and he could forward all the mails back to our support mailbox..

I dropped the project idea and learned to work with dev environments :D A couple of months later, I accidentially forgot a where condition in my SQL UPDATE statement, but that was the last time I seriously f*cked up.. :D Got to learn the hard way I guess.. Now everything I do runs in dev environments, I test everything before publishing,.. When I look back.. I don't even recognize the (inexperienced) guy I was back then ! :D

Ps. No one still knows what happened that day and they blamed it on server issues :D

Best part about the covid19 manufactured crisis?
Liquor stores deliver. Worst part about liquor stores delivering? Needing to use their shoddy websites.

I've been using a particular store (Total Wines) since they're cheaper than the rest and have better selection; it's quite literally a large warehouse made to look like a store.

Their website tries really hard to look professional, too, but it's just not. It took me two days to order, and not just from lack of time -- though from working 14 hour days, that's a factor.

Signing up was difficult. Your username is an email address, but you can't use comments because the server 500s, making the ajax call produce a wonderfully ambiguous error message. It also fades the page out like it's waiting on something, but that fade is on top of the error modal too. Similar error with the password field, though I don't remember how I triggered it.

Signing up also requires agreeing to subscribe to their newsletter. it's technically an opt-in, but not opting-in doesn't allow you to proceed. Same with opting-in to receiving a text notification when your order is ready for pickup -- you also opt-in to reciving SMS spam.

Another issue: After signing up, you start to navigate through the paginated product list. Every page change scrolls you to the exact middle of the next page. Not deliberatly; the UI loads first, and the browser gets as close as it can to your previous position -- which was below that as the pagination is at the bottom -- and then the products populate after. But regardless of why, there is no worse place to start because now you must scroll in both directions to view the products. If it stayed at the very bottom, it would at least mean you only need to scroll upwards to look at everything on the page. Minor, but increasingly irritating.

Also, they have like 198 pages of spirits alone because each size is unique entry. A 50ml, 350ml, 500ml, 750ml, 1000ml, and 1750ml bottle of e.g. Tito's vodka isn't one product, it's six. and they're sorted seemingly randomly. I think it's by available stock, looking back.

If you fancy a product, you can click on it for a detail page. Said detail page lists the various sizes in a dropdown, but they're not sorted correctly either, and changing sizes triggers a page reload, which leads to another problem:

if you navigate to more than a few pages within a 10 or so second window, the site accuses you of using browser automation. No captcha here, just a "click me for five seconds" button. However, it (usually) also triggers the check on every other tab you have open after its next nagivation.

That product page also randomly doesn't work. I haven't narrowed it down, but it will randomly decide to start failing, and won't stop failing for hours. It renders the page just fine, then immediately replaces it with a blank page. When it's failing, the only way to interact with the page is a perfectly-timed [esc], which can (and usually does) break all other page functionality, too. Absolutely great when you need to re-add everything from a stale copy of your signed-out cart living in another tab. More on that later. And don't forget to slow down to bypass the "browser automation" check, too!

Oh, and if you're using container tabs, make sure to open new tabs in the SAME container, as any request from the same IP without the login cookie will usually trigger that "browser automation" response, too.

The site also randomly signs you out, but allows you to continue amassing your cart. You'd think this is a good thing until you choose to sign in again... which empties your cart. It's like they don't want to make a sale at all.

The site also randomly forgets your name, replacing it with "null." My screen currently says "Hello, null". Hello, cruft!

It took me two days to order.
Mostly from lack of time, as i've been pulling 14 hour shifts lately trying to get everything done. but the sheer number of bugs certainly wasted most of what little time i had left. Now I definitely need a drink.

But maybe putting up with all of this is worthwhile because of their loyalty program? Apparently if you spend $500, you can take $5 off your next purchase! Yay! 1%! And your points expire! There are three levels; maybe it gets better. Level zero is for everyone; $0 requirement. There are also levels at $500 and $2500. That last one is seriously 5x more than the first paid level. and what does it earn you? A 'free' magazine subscription, 'free' classes (they're usually like $20-$50 iirc), and a 'free' grab bag (a $2.99 value!) twice per month. All for spending $2500. What a steal. It reminds me of Candy Crush's 3-star system where the first two stars are trivial, and the third is usually a difficult stretch goal. But here it's just thinly-veiled manipulation with no benefit.

I can tell they're employing some "smarketing" people with big ideas (read: stolen mistakes), but it's just such a fail.

The whole thing is a fail.

THE WORST PRANK ATTEMPT

If i remember true, it was 2012. april fool day..

me and my co-worker (we were the founders) decided to fool our members (we had a script's unofficial support forum). so, we did the plan. we register another account on march and wrote a few useful messages with it. help guys with that fake account (named as Root).

on fool day, we move the site to hidden folder (but didnt backup it) and added an index file as "hi, i am Root. you know me who am i. i hacked this site and deleted all dbs. cya later" (in turkish of course)

and we sit our chairs, began the watch our messages from facebook,skype,whatsapp etc..

we act like we are in trouble and we cant solve the problem.
at the same time, one of the our crew, decided to help us :D
so, he contact with our server's management crew. they dont know the fool too :)

server management looked up the situation without try to contact with me or my co. and we got an email from server like that
"hello tilkibey and impack, we just realized your site is hacked. so we delete your all ftp and db for safety. please contact with us asap"

we shocked and contact with them, explain the truths and request the recover our site (because we though they backup site before deleting all things). but they didnt backup it :(

so, we recover our last backup which is got nearly 10 days ago :(

!!oracle

I'm trying to install a minecraft modpack to play with a friend, and I'm super psyced about it. According to the modpack instructions, the first step is to download the java8 jre. Not sure if I actually need it or not, but it can download while I'm doing everything else, so I dutifully go to the download page and find the appropriate version. The download link does point to the file, but redirects to a login page instead. Apparently I need an oracle account to download anything on their site. stupid.

So I make an account. It requires my life story, or at least full name and address and phone number. stupid. So my name is now "fuck off" and I live in Hell, Michigan. My email is also "gofuckyourself" because I'm feeling spiteful. Also, for some reason every character takes about 3/4ths of a second to type, so it's very slow going. Passwords also cannot contain spaces, which makes me think they're doing some stupid "security" shenanigans like custom reversible encryption with some 5th grade math. or they're just stupid. Whatever, I make the stupid account.

Afterwards, I try to log in, but apparently my browser-saved credentials are wrong? I try a few more times, try enabling all of the javascripts, etc. No beans. Okay, maybe I can't use it until I verify the email? That actually makes some sense. Fine, I go check the throwaway inbox. No verification email. It's been like five minutes, but it's oracle so they probably just failed at it like everything else, so I try to have them resend the email. I find the resend link, and try it. Every time I enter my email address, though, it either gives me a validation error or a server error. I try a few mores times, and give up. I try to log in again; no dice. Giving up, I go do something else for awhile.

On a whim later, I check for the verification email again. Apparently it just takes bloody forever, but it did show up. Except instead of the first name "Fuck" I entered, I'm now "Andrew", apparently. okay.... whatever. I click the verify button anyway, and to my surprise it actually works, and says that I'm now allowed to use my account. Yay!

So, I go back to the login page (from the download link) and enter my credentials. A new error appears! I cannot use redirects, apparently, and "must type in the page address I want to visit manually." huh? okay, i go to the page directly, and see the same bloody error because of course i do because oracle fucking sucks. So I close the page, go back to the download list, click the link, wait for the login page redirect (which is so totally not allowed, apparently, except it works and manual navigation does not. yay backwards!), and try to log in.

Instead of being presented with an error because of the redirect, it lets me (try to) log in. But despite using prefilled creds (and also copy/pasting), it tells me they're invalid. I open a new tab container, clear the cache (just to be thorough), and repeat the above steps. This time it redirects me to a single signon server page (their concept of oauth), and presents me with a system error telling me to contact "the Administrator." -.- Any second attempts, refreshes, etc. just display the same error.

Further attempts to log in from the download page fail with the same invalid credentials error as before.

Fucking oracle and their reverse Midas touch.

*In the final weeks of development with a project on a short timeline because the client "needs it".*

Client: "We've hired a consultant we want you to work with."

Me: "Okay, can we push this to after the delivery?"

Client: "Of course"

Wake up to an email from the consultant with a list of scripts he just ran on the production database server for the currently live app.

Get follow-up emails about bugs and app crashes from the client.

My rage is so hot it can keep warm an Eskimo tribe over the winter season.

An important message:
PrOpErLy managing servers is HARD.

I get pissed off at customers with ZERO server knowledge who think they can manage their VPS. “Just get a control panel and a VPS” from some flashy provider that makes server management look way too easy.. Clicking around in their fancy control panel, until:

- they need help with their *self-managed* VPS;
- their email ends up in spam;
- they suffer from performance issues;
- they need to restore a backup;
- something breaks, because YES, things break

Way too little people are able to answer:

- when and how do you make backups?
- how do you monitor your servers and which services?
- how do you keep track of trend analysis?

Then I come by with necessary software. SNMP for trend analysis, Graphite for infrastructure health, Sensu for monitoring, Kibana, Ansible for configuration management..

Things that servers need but that customers have never even heard of.. because they can do everything in their control panel..

Until they come crying to me because it broke and they don’t even know how to get into SSH.

I think the ones to blame are VPS providers that tell the tale of how easy it is to install a control panel and never look at your server again.

Customers become responsible for something *business-critical*! Yet they don’t know how it works.

TLDR: Small family owned finance business woes as the “you-do-everything-now” network/sysadmin intern

Friday my boss, who is currently traveling in Vegas (hmmm), sends me an email asking me to punch a hole in our firewall so he can access our locally hosted Jira server that we use for time logging/task management.

Because of our lack of proper documentation I have to refer to my half completed network map and rely on some acrobatic cable tracing to discover that we use a SonicWall physical firewall. I then realize asking around that I don’t have access to the management interface because no one knows the password.

Using some lucky guesses and documentation I discover on a file share from four years ago, I piece together the username and password to log in only to discover that the enterprise support subscription is two years expired. The pretty and useful interface that I’m expecting has been deactivated and instead of a nice overview of firewall access rules the only thing I can access is an arcane table of network rules using abbreviated notation and five year old custom made objects representing our internal network.

An hour and a half later I have a solid understanding of SonicWallOS, its firewall rules, and our particular configuration and I’m able to direct external traffic from the right port to our internal server running Jira. I even configure a HIDS on the Jira server and throw up an iptables firewall quickly since the machine is now connected to the outside world.

After seeing how many access rules our firewall has, as a precaution I decide to run a quick nmap scan to see what our network looks like to an attacker.

The output doesn’t stop scrolling for a minute. Final count we have 38 ports wide open with a GOLDMINE of information from every web, DNS, and public server flooding my terminal. Our local domain controller has ports directly connected to the Internet. Several un-updated Windows Server 2008 machines with confidential business information have IIS 7.0 running connected directly to the internet (versions with confirmed remote code execution vulnerabilities). I’ve got my work cut out for me.

It looks like someone’s idea of allowing remote access to the office at some point was “port forward everything” instead of setting up a VPN. I learn the owners close personal friend did all their IT until 4 years ago, when the professional documentation stops. He retired and they’ve only invested in low cost students (like me!) to fill the gap. Some kid who port forwarded his home router for League at some point was like “let’s do that with production servers!”

At this point my boss emails me to see what I’ve done. I spit him back a link to use our Jira server. He sends me a reply “You haven’t logged any work in Jira, what have you been doing?”

Facepalm.

I love it when unethical companies' marketing chaps assume that I care about their SMS garbage more than I get pissed off by it. Damart, I didn't even know about your existence or what the hell you do as a business (selling woman clothes apparently.. am I a woman?!) until you shoved your ad crap into my face, and Unigro I really don't have a pending contract with you so kindly fuck off with your reminders (that are in French but am I a Walloon? No!).

Makes me wonder though, with email I usually just give everyone a different email address to deflect spam - since I can reroute email to any particular address straight to /dev/null on the server if so desired. With SMS and SIM cards that isn't really possible.. hence why I'm always so hesitant about sharing my phone number. In email the addresses are scraped off the internet.. but phone numbers? How do such shitcompanies even obtain those?! Their level of pushiness and unethical behavior boggles my mind.

My set up in March, 2003.

On the desk: Titanium PowerBook G4 1Gz
On the floor: custom-built PC running my email and web server on SUSE Linux

I hate asking questions but I need to right now.

Any suggestions for an easy installable email server like mailinabox? Multiple domains is a requirement :)

@netikras since when does proprietary mean bad?

Lemme tell you 3 stories.

CISCO AnyConnect:
- come in to the office
- use internal resources (company newsletter, jira, etc.)
- connect to client's VPN using Cisco AnyConnect
- lose access to my company resources, because AnyConnect overwrites routing table (rather normal for VPN clients)
- issue a route command updating routing table so you could reach confluence page in the intranet
- route command executes successfully, `route -n` shows nothing has changed
- google this whole WTF case
- Cisco AnyConnect constantly overwrites OS routing table to ENFORCE you to use VPN settings and nothing else.

Sooo basically if you want to check your company's email, you have to disconnect from client's VPN, check email and reconnect again. Neat!

Can be easily resolved by using opensource VPN client -- openconnect

CISCO AnyConnect:
- get a server in your company
- connect it to client's VPN and keep the VPN running for data sync. VPN has to be UP at all times
- network glitch [uh-oh]
- VPN is no longer working, AnyConnect still believes everything is peachy. No reconnect attempts.
- service is unable to sync data w/ client's systems. Data gets outdated and eventually corrupted

OpenConnect (OSS alternative to AnyConnect) detects all network glitches, reports them to the log and attempts reconnect immediatelly. Subsequent reconnect attempts getting triggered with longer delays to not to spam network.

SYMANTEC VIP (alleged 2FA?):
- client's portal requires Sym VIP otp code to log in
- open up a browser in your laptop
- navigate to the portal
- enter your credentials
- click on a Sym VIP icon in the systray
- write down the shown otp number
- log in

umm... in what fucking way is that a secure 2FA? Everything is IN the same fucking device, a single click away.

Can be easily solved by opensource alternatives to Sym VIP app: they make HTTP calls to Symantec to register a new token and return you the whole totp url. You can convert that url to a qr code and scan it w/ your phone (e.g. Google's Authenticator). Now you have a true 2FA.

Proprietary is not always bad. There are good propr sw too. But the ones that are core to your BAU and are doing shit -- well these ARE bad. and w/o an oppurtunity to workaround/fix it yourself.

TLDR: There’s truth in the motto “fake it till you make it”

Once upon a time in January 2018 I began work as a part time sysadmin intern for a small financial firm in the rural US. This company is family owned, and the family doesn’t understand or invest in the technology their business is built on. I’m hired on because of my minor background in Cisco networking and Mac repair/administration.

I was the only staff member with vendor certifications and any background in networking / systems administration / computer hardware. There is an overtaxed web developer doing sysadmin/desktop support work and hating it.

I quickly take that part of his job and become the “if it has electricity it’s his job to fix it” guy. I troubleshoot Exchange server and Active Directory problems, configure cloudhosted web servers and DNS records, change lightbulbs and reboot printers in the office.

After realizing that I’m not an intern but actually just a cheap sysadmin I began looking for work that pays appropriately and is full time. I also change my email signature to say “Company Name: Network Administrator”

A few weeks later the “HR” department (we have 30 employees, it’s more like “The accountant who checks hiring paperwork”) sends out an email saying that certain ‘key’ departments have no coverage at inappropriate times. I don’t connect the dots.

Two days later I receive a testy email from one of the owners telling me that she is unhappy with my lack of time spent in the office. That as the Network Administrator I have responsibilities, and I need to be available for her and others 8-5 when problems need troubleshooting. Her son is my “boss” who is rarely in the office and has almost no technical acumen. He neglected to inform her that I’m a part time employee.

I arrange a meeting in which I propose that I be hired on full time as the Network Administrator to alleviate their problems. They agree but wildly underpay me. I continue searching for work but now my resume says Network Administrator.

Two weeks ago I accepted a job offer for double my current salary at a local software development firm as a junior automation engineer. They said they hired me on with so little experience specifically because of my networking background, which their ops dept is weak in. I highlighted my 6 months experience as Network Administrator during my interviews.

My take away: Perception matters more than reality. If you start acting like something, people will treat you like that.

It's finally happened. I've used my mail servers for about a year to give out different email addresses on my domain to things I sign up for online, and only used my "actual" email address that received all this email for the whole domain but the single one that I used outbound for private communications.

This worked well for a long time as I could see when spam comes in, where it came from by looking at the email address I designated it. Each company's email would be sent not only from an email address that they choose, but also to an email address that I choose. It allowed me to easily determine where there were problems. For example, on Freenode IRC my vhost happened to make my username@host there a valid email address. It eventually got blacklisted due to too much incoming spam as crawlers started detecting it. Another one was "nickname"@my.domain as I posted it a few times here. Got crawled as well. But it allowed me to easily blacklist each.

I'd never thought my actual outbound email address, my real one, to get crawled though. That would require the mail server of a company I explicitly communicated with to get hacked. But today that happened. I wonder whose it is, but I can't tell.

Time to make my outgoing email bound to a designated email address as well. I want to know which companies this happens to, even if they don't disclose it.

I just tried to sign up to Instagram. I made a big mistake.

First up with Facebook related stuff is data. Data, data and more data. Initially when you sign up (with a new account, not login with Facebook) you're asked your real name, email address and phone number. And finally the username you'd like to have on the service. I gave them a phone number that I actually own, that is in my iPhone, my daily driver right now (and yes I have 3 Androids which all run custom ROMs, hold your keyboards). The email address is a usual for me, instagram at my domain. I am a postmaster after all, and my mail server is a catch-all one. For a setup like that, this is perfectly reasonable. And here it's no different, devrant at my domain. On Facebook even, I use fb at my domain. I'm sure you're starting to see a pattern here. And on Facebook the username, real name and email domain are actually the same.

So I signed up, with - as far as I'm aware - perfectly valid data. I submitted the data and was told that someone at Instagram will review the data within 24 hours. That's already pretty dystopian to me. It is now how you block bots. It is not how Facebook does it either, at least since last time I checked. But whatever. You'd imagine that regardless of the result, they'd let you know. Cool, you're in, or sorry, you're rejected and here's why. Nope.

Fast-forward to today when I recalled that I wanted to sign up to Instagram to see my girlfriend's pictures. So I opened Chromium again that I already use only for the rancid Facebook shit.. and it was rejected. Apparently the mere act of signing up is a Terms of Service violation. I have read them. I do not know which section I have violated with the heinous act of signing up. But I do have a hunch.

Many times now have I been told by ignorant organizations that I would be "stealing" their intellectual property, or business assets or whatever, just because I sent them an email from their name on my domain. It is fucking retarded. That is MY domain, not yours. Learn how email works before you go educate a postmaster. Always funny to tell them how that works. But I think that in this case, that is what happened.

So I appealed it, using a random link to something on Instagram's help section from a third-party blog. You know it's good when the third-party random blog is better. But I found the form and filled it in. Same shit all over again for info, prefilling be damned I guess. Minor convenience though, whatever.

I get sent an email in German, because apparently browsing through a VPS in Germany acting as a VPN means you're German. Whatever... After translating it, I found that it asks me to upload a picture of myself, holding a paper in my hands, on which I would have a confirmation code, my username, and my email address.. all hand-written. It must not be too dark, it must be clear, it must be in JPEG.. look, I just wanted to fucking sign up.

I sent them an email back asking them to fix all of this. While I was writing it and this rant, I thought to myself that they can shove that piece of paper up their ass. In fact I would gladly do it for them.

Long story short, do not use Instagram. And one final thing I have gripes with every time. You are not being told all the data you'll have to present from the get-go. You're not being told the process. Initially I thought it'd just be email, phone, username, and real name. Once signed up (instantly, not within 24 hours!) I would start setting up my account and adding a profile picture. The right way to ask for a picture of me! And just do it at my own pace, as I please.

And for God's sake, tackle abuse when it actually happens. You'll find out who's a bot and who isn't by their usage patterns soon enough. Do not do any of this at sign-up. Or hell, use a CAPTCHA or whatever, I don't fucking care. There's so many millions of ways to skin this cat.

Facebook and especially Instagram. Both of them are fucking retarded.

Context: we analyzed data from the past 10 years.

So the fuckface who calls himself head of research tried to put blame on me again, what a surprise. He asked for a tool what basically adds a lot of numbers together with some tweakable stuff, doesnt really matter. Now of course all the datanwas available already so i just grabbed it off of our api, and did the math thing. Then this turdnugget spends 4 literal weeks, tryna feed a local csv file into the program, because he 'wanted to change some values'. One, this isnt what we agreed on, he wanted the data from the original. When i told him this, he denied it so i had to dig out a year old email. Two, he never explicitly specified anything so i didnt use a local file because why the fuck would i do that. Three, i clearly told him that it pulls data from the server. Four, what the fuck does he wanna change past values for, getting ready for time travel? Five, he ranted for like 3 pages, when a change can be done by currentVal - changedVal + newVal, even a fucking 10 year old could figure that out. Also, when i allowed changes in a temporary api, he bitched about how the additional info, what was calculated from yet another original dataset, doesnt get updated, when he fucking just randimly changes values in the end set. Pinnacle of professionalism.

Let's setup my email server again! Found an awesome open source solution which installs it through Ansible.

Now i can't get past a fucking error 😫

Here goes my day :(

Often I hear that one should block spam email based on content match rather than IP match. Sometimes even that blocking Chinese ranges in particular is prejudiced and racist. Allow me to debunk that after I've been looking at traffic on port 25 with tcpdump for several weeks now, and got rid of most of my incoming spam too.

There are these spamhausen that communicate with my mail server as much as every minute.
- biz-smtp.com
- mailing-expert.com
- smtp-shop.com

All of them are Chinese. They make up - rough guess - around 90% of the traffic that hits my edge nodes, if not more.

The network ranges I've blocked are apparently as follows:
- 193.106.175.0/24 (Russia)
- 49.64.0.0/11 (China)
- 181.39.88.172 (Ecuador)
- 188.130.160.216 (Russia)
- 106.75.144.0/20 (China)
- 183.227.0.0/16 (China)
- 106.75.32.0/19 (China)
.. apparently I blocked that one twice, heh
- 116.16.0.0/12 (China)
- 123.58.160.0/19 (China)

It's not all China but holy hell, a lot of spam sure comes from there, given how Golden Shield supposedly blocks internet access to the Chinese citizens. A friend of mine who lives in China (how he got past the firewall is beyond me, and he won't tell me either) told me that while incoming information is "regulated", they don't give half a shit about outgoing traffic to foreign countries. Hence all those shitty filter bag suppliers and whatnot. The Chinese government doesn't care.

So what is the alternative like, that would block based on content? Well there are a few solutions out there, namely SpamAssassin, ClamAV and Amavis among others. The problem is that they're all very memory intensive (especially compared to e.g. Postfix and Dovecot themselves) and that they must scan every email, and keep up with evasion techniques (such as putting the content in an image, or using characters from different character sets t̾h̾a̾t̾ ̾l̾o̾o̾k̾ ̾s̾i̾m̾i̾l̾a̾r̾).

But the thing is, all of that traffic comes from a certain few offending IP ranges, and an iptables rule that covers a whole range is very cheap. China (or any country for that matter) has too many IP ranges to block all of them. But the certain few offending IP ranges? I'll take a cheap IP-based filter over expensive content-based filters any day. And I don't want to be shamed for that.

So I fucked up.. I assigned a small wordpress/woocommerce project to myself to avoid my team members from wasting their time on it. I had a two month deadline, which was insane, so I kept postponing it until I forgot about it. Today my client contacted me by email to ask if she could preview the site before our meeting in two weeks..
QUE BULLSHIT EXCUSE:

- “I had to migrate to another server because of some access/permission issues with my current host. They gave me their word that they would be done with the migration thursday or friday, then I have to correct some permissions and database settings, and the DNS update may take up to 24 hours to finish. I will personally make sure that you know as soon as the migration has finished.”

- “Thank you so much! I feel so safe having assigned your company the job! I am really looking forward to our meeting and seeing the site!”

Oh and did I mention that deadline was around 65 days ago? And that I haven’t even started yet? I know what I’ll be doing for the next 6 days..

Today I sent email blast to wrong list of 12k recipients. I was given the list by another employee. I crashed the email server. It was fun.

So yeah, we released our app for 26k people to use two days ago. Due to circumstances not under our control, that was developed way faster than it should've. Today we're two days in and everything is going horrible. 26 thousand people are having trouble using it (not every problem is our fault, but ofc they don't give a shit) and our support team is not big enough to handle every request. We're not able to find any more people to help us handle it, so some developers are being used for support. But that slows the bugfixes a lot. We're at risk of losing our jobs because we will not be able to make it work in due time.

In Italy, at the beginning of the year a particular type of invoice became necessary and to handle that everyone needs a particular kind of email called "certified", you know, because Italy, and today our certified email server went offline for the whole fucking day because the provider was an overload of feces. We were overwhelmed by the amount of people that needed help.

I don't even know what to say, if we don't fix it we're fucking fucked, like literally.

I really hope everyone is having a nice new year. For us, this is going terrible.

Not hacking per se, but I noticed an email floating between ISP and director about radius server login details, promptly saved for future reference.

One day noticed someone downloading mucho dataro... Pissed me off, interrupting my video stream.

Logged into radius server, blocked MAC address.

PM: "so I need you to deploy this new application to some new server. The deadline is in 2 days"

Me: "yeah I can do that, is the application ready and has been tested? Have the servers been set up properly by the IT guy?"

PM: "yep, all is set up and good"

Couple of hours later I try locating the server, only to find it didn't exist.

Me: "the server you mentioned earlier, is doesn't appear to exist?"

PM: "it definitely does, IT guy said he set it all up"

I dig around a little more, but this server definitely doesn't exist. The IT guy was on holiday for a week, so we had to wait for him to get back; delaying the release. On the morning the IT guy got back,

PM: " I though you said you set up that server for the application, we've had to delay it now!"

IT: "I just set it up this morning. Like I said in the email to you before I Ieft, I will have to do it first thing when I get back after holiday"

Turns out the PM had asked the IT guy to spin up the server, but never bothered to read his response. Assuming it was done he told the client he'd have it deployed in a couple of days.

The application was deployed successfully later that day, but not before the PM blamed us two for its delay.

Test server not working as expected - check with test team.
Test team experiencing the same problem - report to dev.
Dev team says the issue is not on their side - check with the service owners.
Service owner says it might be a problem with the request and gives me another point of contact - send an email with all the info.
POC says it doesn't have anything to do with him and copies "the right person".
Next person says to ask a different team.

Get reprimanded for asking too many teams.

Jeez idk, maybe if you stopped passing on the blame like a hot potato and actually helped me out I wouldn't need to.

*looks through code trying to find and fix a bug that crashes server, thinking heavily*

*Coworker comes up*

> Oh hey did you see the email I sent you?

*Forgets everything and has to reread code from top to understand*

repeat indefinitely

For the first time I am feeling like.... I hate my job.

Agile and Scrum can be fucked, but at least there is a work methodology. I was hired by a company being run the old school way.

These guys never heard of git??
- Fuck you. We never used git and neither should you.

Client company does not want to give me push/pull access to their gitlab instance??
- Fuck you, you can use our RDP server for that.

Project planning features be damned, they've got email, Teams and videocalls!

Can I develop in peace? Fuck no, I have to give IT support to the guy who hired me.

Our timeline is defined IN A FUCKING WORD DOCUMENT FOR FUCKS SAKE. I can't connect Issues to milestones in a Word doc

Oh, and the customer is running everything on prem. If there is a need to scale up, FUCK ME. I should have specified 20 machines from the get go or gtfo. We're using 2 machines to run 8 different services that are going to be ingesting and computing data.

They want state of the art on a cheapskate.

And I have nothing else lined up at the moment. Although I am soon to renew the contract... This contract binds me with professional responsibility for a project being ran by people who do not give a single fuck about optimizing the work process.

On friday a colleague reveived an email from one of our biggest customers. The email was about a public repository on github which contains our software. In the code were many emails from employees...

I'm the guy who is actually writing this software and we are in an early stadium of development. So I wrote this emails for a dropdown field plain in the code for testing. I would never do this in a release version!! We have a company bitbucket server where I push all my stuff to.

Two months ago my team leader aquired a student, he will be working during his graduation, and he has many fresh ideas. And he coded some cool stuff for a big conference here germany. But, BUT!! Last tuesday he has the awesome idea to publish our code on github. He didn't ask anyone. This repo was 3 days online, with emails from our customer. I asked him for a reason to do that. He thought they wouldn't find the repo. WTF?!?

I don't know what we can expect, but this is really shitty!

"Want to edit that file? You need to clone [this], then install [x] followed by [y] and [z]. Then telnet into the 80's and curl this page directly from Ceefax. Install the binaries and compile the watchers. Hit ‘enableficate’ and wait whilst it builds the VM mirror. Remote desktop into the vm from a Windows2000 machine and install a UI. Search for [some file]. This is cross-compiled into an reverse ascii hex hashinator so you’ll need to decompile it using fudgeunpacker. Edit the file as required then reverse the entire process to synchronise it with the repo. You can then upload the file to the server (remember that [thisdomain] is on [some obscure remote server] so you’ll have to email them the file in [x localle] office hours)”

Ever want to smack someone in the face with a sea bass? Like left out of the water for 3 days with all kinds of juicy and smelly goodness?
When we get an X number system errors, an email is sent to our team. Couple of hours ago I had to move the alerts from one system to another, re-naming some because I suck at naming things. I guess when I copied, I duplicated one. About an hour ago we get a system alert (as it should, there was a server hiccup) and there were two emails with the same data (just named differently)
DevA: “Why are there two emails?”
Me: “Oh, that’s me. I think I copied the alert instead of move. I’ll fix it tomorrow.”
DevA: “Hmm, but the data is the same. It’s a duplicate.”
Me: “Yea, I know. Networking responded and said it’s fixed. We won’t get another email.”
- 15 minutes later
DevA leans over and says to the boss (who was in a meeting and just sat down)…
DevA: “I looked at the alert, it’s duplicated, but the name is different. I don’t understand why.”
Me: “Like I said, the alert is duplicated. When I migrated, I copied instead of moved. No big deal.”
DevA: “Oh …oh ..yea.…OK.”
- 5 minutes later
DevA: “I looked at the query, we might have to add a filter to prevent duplicate emails. Probably some logic problems in the search.”
Me: “I just deleted the duplicate alert.”
DevA: “Oh…OK…that fixes it too.”

Good lord…as I was typing this, he just told another dev the ‘duplicate’ emails were because of a logic bug in the search. I’m getting my fishing pole.

I might lose my job this week

I'm part of a team of 2 tech people
We were hired as programmers. But over these past 10 months we've done everything from helpdesk to fixing network infrastructure, i setup a backup server for the company, started properly managing the companies passwords,and a host of other things not in my contract.
But my boss is changing the deadline again and she refuses to listen to anyone's concerns, she doesn't understand the complexity of what she wants and since the best we've done so far can be considered at best a prototype in my opinion shes going to be disappointed

So at the next meeting me and my coworker are going to politely list our grivences point out all shes had us do at the same time and the impossible deadlines.

I've seen herpitch a fit for less so I'm fully prepared to be fired in rage in which case I'll compile the documentation and information on what we've done to email her.

But I'm pretty sure she won't find anything long term for the 40k salary shes expecting. Especially with how slow she is to do work herself. I was supposed to be on company health insurance since October 2020

In a way I'm kinda relieved at the potential of being fired.

So I got an email from my employer/prof saying that there's a really important software that needs to be installed on a specific server, and it (the email) had all these additional "highly technical" stuff. So I open the link to this software.
.
.
Had to resist a really strong urge to burn myself to ashes, and then go jump off a cliff.
.
.
The website (and the software) is in Comic Sans. 😲

> Tells client, if content isn't hosted on their server somewhere, we'll have to update the app every time they need new content

> Client refuses saying it will take too much time and effort to maintain

*Several weeks later*

> Get email from client asking to update the app because content is out of date

> I want to reply with 'Sorry no can do, it will take too much time and effort to maintain'

Next 3 days I' ll be working on moving our email server to new server because the old datacenter is closing down. Clients are pissed and they are fucking verbally bashing me left to right when I told them that their emails will be down since most of them host their own DNS.

I am fucked.

Client wanted a site for the 100th anniversary of an important local musician. They wanted to show calendar of events, biography, store, and more. We started the work 8 months before the commemorative date, and after 4 months, the site was 99% complete and waiting final review and approvement. 2 weeks of silence has passed, when I got an email saying their deleted the site from server and all backups, and now they wanted all the work back.

With luck, I could restore a partial backup, and the client didn't want to pay for redo the rest of the work.

10 months later (yes, after all the events has started and the site being off) they contact me again, asking to continue the work.

I was happy to say no.

Had to bring up some issues with my instructor to the school, and forgot to print out emails hosted on their Exchange server that show the issue in detail. When I logged in, every email had mysteriously disappeared from my account, but because I document everything about this horseshit school locally, I could still show the exported emails.

The instructor?
Head of IT.

The bastard tried to hide the evidence, but fuck him, Document Everything.

First rant here, and it's going to be a query to the more professional and experienced members of society (most of you).

I am currently a Sys Admin for a major company, and I develop at night. My primary employment at the moment is the sys admin job (and I code for extra money at nights).

I wanted to start a development department at the company that I am working at, but it was turned turned down. It was stated that we are not branching in development, and that we should stick to our server implementation and support. This was a prompt to me wanting to start studying officially (I wanted to get qualified in JAVA, so that I had some paper behind my name when I looked for another job). HR and my directors outright denied me the ability to study through them (they pay for studies for employees) and I was more than fine with this.

I took a loan and paid for the studies myself. Can't crush a dream, you know?

The director caught wind of me studying, and now has demanded that I develop him a mobile application for the company. I told him that I am not a mobile developer, and that it didn't fall into my key performance areas.

Note, I do my coding on own time, on my own device, and never at work. It's fully my intellectual property. It also in no way interferes with my work during the day, and has NO conflict with my contract this side.

He sent an email yesterday, this is after two months. He is now stating that I WILL do the application, and he has CCd HR and two directors.

I don't want to do the app for this company, I spoke to HR previously about this, and she said that I should try and quote it under my own company name (which I did, but it was denied as it was "too expensive").

Now I am being forced to do something that is COMPLETELY out of my roles and responsibilities, something that this company has ABSOLUTELY no desire to go into further on, and he is basically letting me know that if I don't do it, he is going to start messing with my pay.

I really don't want to do this, and I cannot afford to make my secondary job my primary at the moment. The problem is, too, that I don't have the time during the day to develop AND do my sys admin tasks (I manage more than 300 servers, and 5000 devices).

What can I do in this instance? Or what would you guys recommend, in your experience?

Sorry for the noob question, but I don't know what to do.

I setup an email server a couple of months ago.

The amount of port scans and brute force attacks I've received this month alone is awful.

JUST SOD OFF ALREADY, PLEASE.

A few days ago Aruba Cloud terminated my VPS's without notice (shortly after my previous rant about email spam). The reason behind it is rather mundane - while slightly tipsy I wanted to send some traffic back to those Chinese smtp-shop assholes.

Around half an hour later I found that e1.nixmagic.com had lost its network link. I logged into the admin panel at Aruba and connected to the recovery console. In the kernel log there was a mention of the main network link being unresponsive. Apparently Aruba Cloud's automated systems had cut it off.

Shortly afterwards I got an email about the suspension, requested that I get back to them within 72 hours.. despite the email being from a noreply address. Big brain right there.

Now one server wasn't yet a reason to consider this a major outage. I did have 3 edge nodes, all of which had equal duties and importance in the network. However an hour later I found that Aruba had also shut down the other 2 instances, despite those doing nothing wrong. Another hour later I found my account limited, unable to login to the admin panel. Oh and did I mention that for anything in that admin panel, you have to login to the customer area first? And that the account ID used to login there is more secure than the password? Yeah their password security is that good. Normally my passwords would be 64 random characters.. not there.

So with all my servers now gone, I immediately considered it an emergency. Aruba's employees had already left the office, and wouldn't get back to me until the next day (on-call be damned I guess?). So I had to immediately pull an all-nighter and deploy new servers elsewhere and move my DNS records to those ASAP. For that I chose Hetzner.

Now at Hetzner I was actually very pleasantly surprised at just how clean the interface was, how it puts the project front and center in everything, and just tells you "this is what this is and what it does", nothing else. Despite being a sysadmin myself, I find the hosting part of it insignificant. The project - the application that is to be hosted - that's what's important. Administration of a datacenter on the other hand is background stuff. Aruba's interface is very cluttered, on Hetzner it's super clean. Night and day difference.

Oh and the specs are better for the same price, the password security is actually decent, and the servers are already up despite me not having paid for anything yet. That's incredible if you ask me.. they actually trust a new customer to pay the bills afterwards. How about you Aruba Cloud? Oh yeah.. too much to ask for right. Even the network isn't something you can trust a long-time customer of yours with.

So everything has been set up again now, and there are some things I would like to stress about hosting providers.

You don't own the hardware. While you do have root access, you don't have hardware access at all. Remember that therefore you can't store anything on it that you can't afford to lose, have stolen, or otherwise compromised. This is something I kept in mind when I made my servers. The edge nodes do nothing but reverse proxying the services from my LXC containers at home. Therefore the edge nodes could go down, while the worker nodes still kept running. All that was necessary was a new set of reverse proxies. On the other hand, if e.g. my Gitea server were to be hosted directly on those VPS's, losing that would've been devastating. All my configs, projects, mirrors and shit are hosted there.

Also remember that your hosting provider can terminate you at any time, for any reason. Server redundancy is not enough. If you can afford multiple redundant servers, get them at different hosting providers. I've looked at Aruba Cloud's Terms of Use and this is indeed something they were legally allowed to do. Any reason, any time, no notice. They covered all their bases. Make sure you do too, and hope that you'll never need it.

Oh, right - this is a rant - Aruba Cloud you are a bunch of assholes. Kindly take a 1Gbps DDoS attack up your ass in exchange for that termination without notice, will you?

Still dealing with the web department and their finger pointing after several thousand errors logged.

SeniorWebDev: “Looks like there were 250 database timeout errors at 11:02AM. DBAs might want to take a look.”

I look at the actual exceptions being logged (bulk of the over 1,600 logged errors)..

“Object reference not set to an instance of an object.”

Then I looked the email timestamp…11:00AM. We received the email notification *before* the database timeout errors occurred.

I gather some facts…when the exceptions started, when they ended, and used the stack trace to find the code not checking for null (maybe 10 minutes of junior dev detective work). Send the data to the ‘powers that be’ and carried on with my daily tasks.
I attached what I found (not the actual code, it was changed to protect the innocent)
Couple of hours later another WebDev replied…

WebDev: “These errors look like a database connectivity issue between the web site and the saleitem data service. Appears the logging framework doesn’t allow us to log any information about the database connection.”

FRACK!!...that Fracking lying piece of frack! Our team is responsible for the logging framework. I was typing up my response (having to calm down) then about a minute later the head DBA replies …

DBA: “Do you have any evidence of this? Our logs show no connectivity issues. The logging framework does have the ability to log an extensive amount of data regarding the database transaction. Database name, server, login, command text, and parameter values. Everything we need to troubleshoot. This is the link to the documentation …. If you implement the one line of code to gather the data, it will go a long way in helping us debug performance and connectivity issue. Thank you.”

DBA sends me a skype message “You’re welcome :)”

Ahh..nice to see someone else fed up with their lying bull...stuff.

Just spend the last 2 hours fixing the email server for sites which get next to no visits and no one uses the contact form.

It's Friday evening FFS I don't care if it's a 'Company Priority' I just want to go home.

Anything I (am able to) build myself.
Also, things that are reasonably standardized. So you probably won't see me using a commercial NAS (needing a web browser to navigate and up-/download my files, say what?) nor would I use something like Mega, despite being encrypted. I don't like lock-in into certain clients to speak some proprietary "secure protocol". Same reason why I don't use ProtonMail or that other one.. Tutanota. As a service, use the standards that already exist, implement those well and then come offer it to me.

But yeah. Self-hosted DNS, email (modified iRedMail), Samba file server, a blog where I have unlimited editing capabilities (God I miss that feature here on devRant), ... Don't trust the machines nor the services you don't truly own, or at least make an informed decision about them. That is not to say that any compute task should be kept local such as search engines or AI or whatever that's best suited for centralized use.. but ideally, I do most of my computing locally, in a standardized way, and in a way that I completely control. Most commercial cloud services unfortunately do not offer that.

Edit: Except mail servers. Fuck mail servers. Nastiest things I've ever built, to the point where I'd argue that it was wrong to ever make email in the first place. Such a broken clusterfuck of protocols, add-ons (SPF, DKIM, DMARC etc), reputation to maintain... Fuck mail servers. Bloody soulsuckers those are. If you don't do system administration for a living, by all means do use the likes of ProtonMail and Tutanota, their security features are nonstandard but at least they (claim to) actually respect your privacy.

There you are, fiddling with next.js webpack settings, because your isomorphic JS-in-CSS-in-JS SSR fallback from react-native-web to react-dom throws a runtime error on your SSR prerendering server during isomorphic asynchronous data prefetching from Kubernetes backend-for-frontend edge-server with GraphQL.

You have all that tech to display a landing page with an email form, just to send spam emails with ten tracking links and five tracking beacons per email.

Your product can be replaced by an Excel document made in two days.

It was developed in two years by a team of ten developers crunching every day under twelve project managers that can be replaced with a parrot trained to say “Any updates?”

Your evaluation is $5M+. You have 10,000 dependency security warnings, 1000 likes on Product Hunt, 500 comments on Hacker News, and a popular Twitter account.

Your future looks bright. You finish your coffee, crack your knuckles and carry on writing unit tests.

I am a Windows person. I always argue how great it is.

Well, not today.

I was today years old when I learned that you CANNOT uninstall store app via store ;p You need to go to settings / apps and functionality / your app / uninstall

The photo app (Yes the one bundled with win10) doesn't work if you use Hard drive compression AND it is a symlink for OneDrive (So you don't need to keep all photos on the drive). Fucking Paint works without problems.

Email client : If you alt+tab too fast after hitting 'Send email", there is 50% chances that email won't be send. Basiclly you need to hit "send" and wait until you see it in "sent" folder.

Well, as i'm ranting, here for Linux too :
I have a small ubuntu server VM, worked very well for last 6 months. Now "System in read only mode". Fucking apt-get upgrade fucked with something. I don't want to look, so I'll just rebuild a fresh vm.

And macOS should take sometyhing too : Who the fuck decided "enter" is for editing the name of file ?! really !

Well, ALL os are shit, all have downsides, I need my own OS. But I still want AA games... So windows for me.

So one of my clients had a different company do a penetrationtest on one of my older projects.

So before hand I checked the old project and upgraded a few things on the server. And I thought to myself lets leave something open and see if they will find it.

So I left jquery 1.11.3 in it with a known xss vulnerability in it. Even chrome gives a warning about this issue if you open the audit tab.

Well first round they found that the site was not using a csrf token. And yeah when I build it 8 years ago to my knowledge that was not really a thing yet.

And who is going to make a fake version of this questionair with 200 questions about their farm and then send it to our server again. That's not going to help any hacker because everything that is entered gets checked on the farm again by an inspector. But well csrf is indeed considered the norm so I took an hour out of my day to build one. Because all the ones I found where to complicated for my taste. And added a little extra love by banning any ip that fails the csrf check.

Submitted the new version and asked if I could get a report on what they checked on. Now today few weeks later after hearing nothing yet. I send my client an email asking for the status.

I get a reaction. Everything is perfect now, good job!

In Dutch they said "goed gedaan" but that's like what I say to my puppy when he pisses outside and not in the house. But that might just be me. Not knowing what to do with remarks like that. I'm doing what I'm getting paid for. Saying, good job, your so great, keep up the good work. Are not things I need to hear. It's my job to do it right. I think it feels a bit like somebody clapping for you because you can walk. I'm getting off topic xD

But the xss vulnerability is still there unnoticed, and I still have no report on what they checked. So I have like zero trust in this penetration test.

And after the first round I already mentioned to the security guy in my clients company and my daily contact that they missed things. But they do not seem to care.

Another thing to check of their to do list and reducing their workload. Who cares if it's done well it's no longer their responsibility.

2018 disclaimer: if you can't walk not trying to offend you and I would applaud for you if you could suddenly walk again.

I just got an email that a client changed their DNS zone files to point at a new server. Turns out that they haven't set the server up yet. Client is wondering why that domain's emails aren't working, and why their site is down. They didn't want to give me the Domain's portal login credentials until now, because they "could do it without [me]." Tomorrow morning should be fun.

After about four hours of email server setup I'm sitting here with a citadel setup, not being able to login, Google didn't help, I'm fucking done with this shit

Last Monday I bought an iPhone as a little music player, and just to see how iOS works or doesn't work.. which arguments against Apple are valid, which aren't etc. And at a price point of €60 for a secondhand SE I figured, why not. And needless to say I've jailbroken it shortly after.

Initially setting up the iPhone when coming from fairly unrestricted Android ended up being quite a chore. I just wanted to use this thing as a music player, so how would you do it..?

Well you first have to set up the phone, iCloud account and whatnot, yada yada... Asks for an email address and flat out rejects your email address if it's got "apple" in it, catch-all email servers be damned I guess. So I chose ishit at my domain instead, much better. Address information for billing.. just bullshit that, give it some nulls. Phone number.. well I guess I could just give it a secondary SIM card's number.

So now the phone has been set up, more or less. To get music on it was quite a maze solving experience in its own right. There's some stuff about it on the Debian and Arch Wikis but it's fairly outdated. From the iPhone itself you can install VLC and use its app directory, which I'll get back to later. Then from e.g. Safari, download any music file.. which it downloads to iCloud.. Think Different I guess. Go to your iCloud and pull it into the iPhone for real this time. Now you can share the file to your VLC app, at which point it initializes a database for that particular app.

The databases / app storage can be considered equivalent to the /data directories for applications in Android, minus /sdcard. There is little to no shared storage between apps, most stuff works through sharing from one app to another.

Now you can connect the iPhone to your computer and see a mount point for your pictures, and one for your documents. In that documents mount point, there are directories for each app, which you can just drag files into. For some reason the AFC protocol just hangs up when you try to delete files from your computer however... Think Different?

Anyway, the music has been put on it. Such features, what a nugget! It's less bad than I thought, but still pretty fucked up.

At that point I was fairly dejected and that didn't get better with an update from iOS 14.1 to iOS 14.3. Turns out that Apple in its nannying galore now turns down the volume to 50% every half an hour or so, "for hearing safety" and "EU regulations" that don't exist. Saying that I was fuming and wanting to smack this piece of shit into the wall would be an understatement. And even among the iSheep, I found very few people that thought this is fine. Though despite all that, there were still some. I have no idea what it would take to make those people finally reconsider.. maybe Tim Cook himself shoving an iPhone up their ass, or maybe they'd be honored that Tim Cook noticed them even then... But I digress.

And then, then it really started to take off because I finally ended up jailbreaking the thing. Many people think that it's only third-party apps, but that is far from true. It is equivalent to rooting, and you do get access to a Unix root account by doing it. The way you do it is usually a bootkit, which in a desktop's ring model would be a negative ring. The access level is extremely high.

So you can root it, great. What use is that in a locked down system where there's nothing available..? Aha, that's where the next thing comes in, 2 actually. Cydia has an OpenSSH server in it, and it just binds to port 22 and supports all of OpenSSH's known goodness. All of it, I'm using ed25519 keys and a CA to log into my phone! Fuck yea boi, what a nugget! This is better than Android even! And it doesn't end there.. there's a second thing it has up its sleeve. This thing has an apt package manager in it, which is easily equivalent to what Termux offers, at the system level! You can install not just common CLI applications, but even graphical apps from Cydia over the network!

Without a jailbreak, I would say that iOS is pretty fucking terrible and if you care about modding, you shouldn't use it. But jailbroken, fufu.. this thing trades many blows with Android in the modding scene. I've said it before, but what a nugget!

Here comes the story how I became a DevRanter.
When I was young, I built an expensive gamer-machnine, so I had to crack games. I Got used to computers, so I startet an apprenticeship in IT. I finished with good grades. I left everything and everyone behind and moved in a city, found a parttime job as a PHP developer and started studying CS. After 5 years doing work as developer, studying CS, creeping around as soldier, I finally finished and graduated. After a few months working fulltime (same job), as my life began to settle down and I got bored.
A flatmate (also CS) laughed his ass off about something, then he introduced me to DevRant. It became part of my life to read DevRant, to overcome boredom. But there are not enough new Rants.. I'm f'cked. OK, I resigned my Job, and my flat and signed up for the BS in natural scinces at university in an even bigger city. I will again leave everything behind to begin a new life. Now I'm planing to freelance to pay the bills and challenge me again. Wish me luck :)

So I am beginning this new life with writing this story, how i became a dev. I klick Post, and bang! "please verify your email before ranting.. blah" I got no mail, no span, nothing. Resend.. wait.. nothing. I WAS BORED AGAIN!! FUCK YOU MAIL-SERVER, WHY CAN'T YOU SEND AN EMAIL WITHIN SECONDS OR MINUTES, WE ARE IN 21ST CENTURY AND THE INTERNET CONSISTS MAINLY OF OPTIC FIBER CABLES!!
And this is, dear DevRant community, how i become a Ranter, just then when I wanted to Post my first story.

Server admin: "When do I need to make this config change for you?"

Me (in my head): "You mean the one I put a note in the change request ticket about in ALL CAPS and surrounded by asterisks saying 3pm (aside from the scheduled time field that the ticket requires), and the one we then subsequently chatted about where I reiterated the criticality of the timing about and the one I copied you in the email chain about that said the time in big, bold letters the time? THAT config change?!"

Me (IRL): "3pm, please."

(does not inspire confidence, though better to be asked then they just go off and do it whenever the mood strikes I suppose, which HAS happened)

During my 1st internship (IT support for a big office + maintenance of their server) there was a guy that was refusing to read email from his monitor, he had to print them out and then read... We were so pissed off because he was wasting so much paper and toner.

Stakeholder: Is it possible for you to set up the website to automatically resubmit failed online orders? Last time there were failed orders, we tried submitting manually but a lot failed because they were tickets for the previous day.

Product Manager: What are your thoughts, Developer?

Me: This wouldn’t be worth the labor. It’s something that would rarely be used. There are very few orders that fail. I’d be surprised if it was even once a week. The recent bunch of order failures that SH is talking about happened because the ticketing server (separate from the website) couldn’t handle all the requests. Let’s say you had resubmission logic to try 3x before allowing the fail. It wouldn’t work because the server was overwhelmed already. Let’s say you had a background task to check for failures every ten minutes and resubmit those. It might not be helpful because the customer could have already gone to a ticketing window for help with the failed order.

SH: But what if it happens again???

Me: The solution is to make sure the ticketing server can handle the influx of requests. We can coordinate with that team. Wait. Why did you wait until the next day to resubmit orders in the admin panel? A lot of those failures happened when there were many hours left in the business day. For each order failure, an email notification is sent to the sales support email in real time. Who is monitoring that inbox? Someone must be looking at it because the sales support email is listed multiple times on the ticketing website as the technical assistance email.

SH: I know that email notification goes to the engineering team.

Me: My question is not about the engineering team. I asked who is monitoring the sales support inbox.

SH: That email … gets filtered.

Me and Product Manager: 😧🤯🤬

PM: First, you need to stop filtering that email notification. Second, your team needs to come up with a flow to handle failed orders because you told us you don’t have one. After you tried this and there’s still an issue, then we can revisit.

—-
If you’re wondering why I said no, I’m a team of one and I have a bunch of other development tasks on my plate. I’m not automating a manual task that rarely has to be performed.

I get an email about an hour before I get into work: Our website is 502'ing and our company email addresses are all spammed! I login to the server, test if static files (served separately from site) works (they do). This means that my upstream proxy'd PHP-FPM process was fucked. I killed the daemon, checked the web root for sanity, and ran it again. Then, I set up rate limiting. Who knew such a site would get hit?

Some fucking script kiddie set up a proxy, ran Scrapy behind it, and crawled our site for DDoS-able URLs - even out of forms. I say script kiddie because no real hacker would hit this site (it's minor tourism in New Jersey), and the crawler was too advanced for joe shmoe to write. You're no match for well-tuned rate-limiting, asshole!

Yesterday night, pushed code that work normally to prod server, website down, internal server error, too many connection to MySQL server, tried to fix it for 4 hours, nothing to do, removed the new code, still the same problem, in my head, I told myself that I'm not good at programming (not the first time), send an email to the host, they tell me the problem is from them and they fixed it. And now I know I'm not bad enough.

I was reminded of people's posts about preferred text editors in another post, so I thought I'd do the same, but also add some super old technology that I used along the way.

The first text editor I consistently used was pico. I used it to write my first webpage at school.edu/~username. It was a natural choice, because the it was the default text editor in pine, which is what we would all use for our email after opening a serial connection to the college's Digital Unix server. Or if we were the lucky ones who had a computer in a wired dorm, telnet. My dorm was not wired until my sophomore year.

I got my first job in tech in 2001, working as a night shift tier-one support technician. By this time, most people were using web based email, or POP3, but I wanted to keep using pine (or elm, or mutt) because I was totally in love with the command line by this time, and had been playing with Linux for two or three years by now. I arranged a handshake deal with a guy in my home town who had a couple well-connected NetBSD servers, to let me have an account on one for email and web hosting (a relatively new idea at the time).

I recall telnetting into my shared hosting account from the HP-UX workstations we had in the control room. I would look at webpages on HTML conventions and standards, and I kept seeing references to this thing called vi. I looked into it more deeply, and found that it was a text editor, and was the reason I always had to CTRL-Z out of elm. I was already finding pico to be lacking, so I found a modern implementation of vi called vim that was already installed on the aforementioned NetBSD server, and read through vimtutor on it. I was hooked instantly. The modality massively appealed to me, and I found editing files to be an absolute delight, compared to pico, and its nascent open source offspring/successor, nano.

My position on that hasn't changed in the years that have passed since then.

What's your text editor origin story?

Not really a hack but still worth telling:

I was working in the QA team for a big project. I tried to do some automation when I realized some radio button behaved weird... out of curiosity I checked the source and saw that there was a hidden option for a unimplemented payment option.

I was like: Let’s see how the system behaves if I just submit that form with that hidden value...

Well I was very surprised when I received the email that my order has been processed successfully.

During the investigation we found out that this bug was in prod for over two years. And it requires a one liner executed in the browsers console to skip the payment.

It was kind of a big deal and although I was (and am) still a trainee (in apprenticeship) I got invited to meet up with the client and the bosses.

It was kind of a door opener! After that they trusted me more. I have more responsibility, more interesting tasks and more client contact ever since.

To make a long story short:
Validate everything on the server side ;-)

Developing a notification API, sends emails to subscribers, email API can take only 100 IDs at once, so partitioned the email list and send mails in blocks of 100.

Forgot to reset the list after every block, so each new partition got appended to the existing list and kept going on.

Ran it against a test DB, which was recently refreshed with near-prod data !!! Thousands of emails went out of the app server in one shot and everybody receiving numerous duplicate emails. Especially the ones in the very first partition.

Got an incident raised by the CEO himself reg the flurry of emails. But, things were out of our hands, quite literally. All emails are queued up in the exchange server.

Called up the exchange server team, purged​ the queued emails. No other emails were sent/received during this whole episode.

Thanks to Iterables.partition in the present day.

Back when SharePoint was still foreign to me, and I didn't know the pain of administrating it, I had the idea that files were copied to my local machine. I saw no need to preserve backups from before I started, especially since they already existed on the server, so I got rid of them.

Also hooked up to SharePoint was an email handler. Whenever a case was created or deleted, an email went out to the entire department. Guess what happened when I deleted 250,000 records?

Fortunately, SharePoint has a recycle bin. Unfortunately, restoring those files generated another 250,000 emails. To the whole department.

I bought many donuts to appease the crowd baying for my blood.

Subject: a rant for devRant

Hello,

Not entirely sure why or when exactly it happened, but after I joined mailing lists I have a pet peeve for people who don't use a proper subject line, or don't use email when I literally made a mail server for that purpose (some organizations really prefer calling apparently). Is it really that hard to summarize a message in one sentence? Hell half the time even the message itself is just a few sentences.

Also the greeting and salutation at the beginning and end of email messages. I find them so redundant. Has anyone ever gotten any meaningful information out of "Hello", "Greetings", "Dear", or something like that? Or "Best regards" or whatever. I get that it's just being polite but it's so meaningless! I really don't like using them anymore. Just a message block and who it came from, that's all it needs to me. Instead pour some effort into the damn subject, the title of whatever drivel you're putting out there! Or replying to an email *only* when the subject matter is still related! Or actually replying to the damn email if it's still that subject matter...

I probably sound like an old man, but seriously.. email isn't a hard concept once you "get it". Anyone can write a halfway decent letter, why isn't that the case for email?

Best regards,
Condor

I am so mad, I have no words for how fucking much I hate ever having to work or pass work to other incompetent developers or teams, what a fucking waste of time and resources.

After handing off the frontend - for the client to find some team, that would do it in the short time and budget he needs (multiple developers, more fast, much good), he found a team that seemed to be alright for the job and seemed alright to me too, now maybe a month or two later, the client contacts me, that they fucked something up and if I could talk to them.

The email I then received from them seriously made me speechles, mad and sad, all at same time, I spent multiple upon multiple hours, getting a very good readable documentation up (markdown with TOC, properly rendered headers, bulletpoints, all that shit), with all files, all services used, all credentials, even converted all ssh keys into putty ppk format, in case the developers are using windows and are too dumb to do it themselves, nginx configs, it had seriously everything, even too much to list.

They somehow managed to fuck up the entire server, while attempting to "add ssh keys themselves", EVEN FUCKING THOUGH I have included all the keys they need, all the hosting credentials, everything, yet they decided to fuck with shit themselves and completely annihilate the server in the process (HOW?!), so not even the webserver works anymore.

I am fucking speechless, I made it so fucking easy to gather all info and files they need, all properly put into well named folders, along the documentation in an archive and they somehow managed to nuke the fucking server, while attempting to add ssh keys?!

If you don't know how to config a server, then don't fucking touch it and just use everything, that got served to you on a fucking silver platter.

---

I'll just instantly answer the most annoying comment, that somebody could come up with: "why didn't you do it yourself?"

Because in a perfect world, a fully managed team, can do much more than a single developer can, especially in the same timeframe and from what I heard of said client, atleast they did something in terms of developing the system. (which surprises me, considering it's the same people that nuked a server, while trying to add ssh keys)

- Launch the new version of the system I have been refactoring for 2 years and counting, then ceremoniously burn (literally) the legacy code as well as the cluster fuck of hardware it runs on.

- Decrease my stress + bus factor by bringing another up to speed on my code & the new version (his cluster fuck now).

- Pay attention to & take better care of health, my wrists in patricular.

- Find a mentor and mentor someone else.

- Get out of crisis management mode and find the time to write tuts, experiment and live a little.

- Find & join a local dev meetup, maybe make a local dev friend.

- Book leave and actually take it, preferabbly without having to take my laptop to the beach - actually, preferabbly at least have the choice to take a offline vacation.

- Sort through the drives containing ALL the code I have ever written, migrate the usefull interesting bits to Github.

Phew, that bit of self reflection was intense! I'm adding a cron to my server to sms & email me this rant in a year to remind me what hope looks like.

NewLifeNewHope update No.2 / Day 5

My Server Is finally opened and i installed 2x8 TB WDC Harddrives and SATA 128 SSD. I know this server is freaking garbage, but i got this PC for free, and have somewhat good-ish upgrade path, so heres the spec :
-Gigabyte H110m-S2 LGA1151
-Intel i3-6300
-8GB DDR4 single channel RAM
-128 GB SSD
-2x8TB Harddrives
-TP-Link 1000mbps NIC

so the plan is to make this server as the Main Repository -- yes no offsite backup plan for now -- and also i want to make this server as an Email Server and for hosting my company's website. I've already asked for static IP from my ISP and will take effect tommorow.

I need help for choosing the Operating System (i like centOS) for my server. and to setting this Server to work like what i planned but i don't know where to start, Any help and/or References will be great !

CAN I HAVE YOUR ATTENTION:

Thanks. This is a notice from Yahoo Mail to warm up your IP’s.

Preferably by toaster but only microwave as a last resort.

Background:
Long story short a mail server was used by botnets to send out a few thousand emails over the past few days. We contacted a few email providers in hopes that they remove our IP from their blacklists.

Client asked us to modify site made in some obscure CMS. Authentication on AJAX request is done by sending email and password as plaintext in header and then it would do md5 on server side

Client contacts our company that his site is down, we do some investigating and the only way we can access the site is on a mobile phone. From the office computers the site never loads and times out. Since we don't host the site and I've never logged into it before I don't have a lot of details so I suggest they contact whoever hosts their site. This is where things get weird.

Client tells me that the site is hosted on someone's home server. I tell him that this is quite strange in 2018 and rather unlikely and ask if he was ever given access to the site to log in or if he has access to his domain registration, GoDaddy.

He says he doesn't understand any of this and would rather I just contact his current developer and figure it out with him. We agree that he needs to get access to his site so we are going to migrate it once I get access to it.

I email his current developer letting him know the client has put me in contact with him to troubleshoot the issues with the site. I ask him some standard questions like: where is the site hosted? Can you access it from a computer? Do you have some security measures in place to block certain IP ranges? Can you give me from access to get the files? Will you send me a backup of the site for me to load up on my server?

*2days pass*

Other dev: Tell me the account number and I'll transfer the domain.

Me: I'll have to get back to you on that once I talk to the client and set up his GoDaddy account since we believe the business owner should own their domain, not their developers. In the meantime you didn't answer any of the questions I asked. Transferring the domain won't get the site on my server so I still need the files.

*3 days pass*

OD: You are trying the wrong domain. The correct domain is [redacted].com I'll have my daughter send you the files when she gets in town. We will transfer the domain to you, the client will forget to pay and the site will go down and it'll be your fault.

Me: I appreciate your advice, but the client will own their domain. I'm trying to get the site online and you have no answered any of my questions. It's been a week now and you have not transferred the domain, you have not provided a copy of the site, you have not told me where the site is hosted. The client and I are both getting impatient at this point when will we receive a backup of the site and the transfer of the domain?

OD: Go fuck yourself, tell the client they can sue me.

If the client is that terrible, wouldn't you want to hand them off to anyone willing to take them? I have never understood why developers and agencies try to hold clients hostage by keeping their domain or website and refusing access. From what I can tell this is a freelance developer without a real company so a legal battle likely isn't going to go well since the domain is worthless to him as the copyright to the name is owned by the client. This isn't the first time we've had to help clients through this sort of thing.

1) API Server crash due to Uncaught Error and none of us realised it until 30 mins later (no email alerts, no mechanism in place to notify devs): expensive mistake

2) api server and database server on one frickin vps, with no caching mechanisms or memoisation on frontend: Worst nightmare of my life

I was working on email reporting to business customers and in the test phase was mass sending email to my own account. However it suddenly stopped working and it took me a few minutes to realize I had commented out the hardcoded line with my email address. I had to write to each Customer and apologize for the spam after my error. Also had to get whitelisted our email server after the incident with a few.

Security! I wish clients would listen to me regarding security...

The client has started to ask me to give them access to all the logins I have for the email, domain, server etc.
I created them a new account and gave them admin access.

Now they’re asking for password for all the email accounts (I don’t even store them). So I asked why, she wanted to have them in case some of the employees forgot their password.

I explained to her, deeply and many times, WHY THIS IS A BAD FUCKING IDEA. I also discovered she’s keeping it in a document, clear text.

Why do they pay me for support, when they want to have access to everything...

I’m wondering if they’re planning to find someone else to do their support, or do it themselves.

I didn’t even think 25€ pr month is that expensive for support

I was happily folowing a tutorial earlier on setting up an email server but the site got taken down for maintenance just as I reached the last step...

IT'S BEEN 6 HOURS NOW AND IT'S STILL NOT BACK UP.

I HAVE NO IDEA WHAT I'M DOING AND I'M GOING TO BREAK SOMETHING AND BE SAD.

Just got an email from my company that a http server app I wrote years ago exposed the whole server it runs on because of a misconfig parametered...

Can use it to read any file using server.com/path/to/file

Just got into office and saw a group email about a recently released feature.

Apparently, it creates a lot of connections which it never closes. This could hang the server since it can only handle a large but finite amount of active connections.

Well the dev said: I will optimize later

My thoughts: later = min(never, production blows up)

Not mine, but a colleague puts a script in production which has to sent an email every time a config changes, but in reality sent an email every time the file was accessed. The system sent a good amount of email in a couple of minutes, the remote SMTP bounced them but the connections on port 25 was dropped by the server, the production firewall hits the maximum number of allowed connections... a lot of shit!

Let's talk about the cargo cult of N-factor authentication. It's not some magic security dust you can just sprinkle onto your app "for security purposes".

I once had a client who had a client who I did server maintenance for. Every month I was scheduled to go to the site, stick my fingerprint in their scanner, which would then display my recorded face prominently on their screens, have my name and purpose verified by the contact person, and only then would the guards let me in.

HAHA no of course not. On top of all of that, they ask for a company ID and will not let me in without one.

Because after all, I can easily forge my face, fingerprints, on-site client contact, appointment, and approval. But printing out and laminating a company ID is impossible.

---

With apologies to my "first best friend" in High School, I've forgotten which of the dozens of canonicalisations of which of your nicknames I've put in as my answer to your security question. I've also forgotten if I actually listed you as my first best friend, or my dog - which would actually be more accurate - and actually which dog, as there are times in my High School life that there were more tails than humans in the house.

I have not forgotten these out of spite, but simply because I have also forgotten which of the dozen services of this prominent bullshit computer company I actually signed up for way back in college, which itself has been more than a decade ago. That I actually apparently already signed up for the service before actually eludes me, because in fact, I have no love for their myriad products.

What I have NOT forgotten is my "end of the universe"-grade password, or email, or full legal name and the ability to demonstrate a clear line of continuity of my identity from wherever that was to now.

Because of previous security screwups in the past, this prominent bullshit company has forced its users to activate its second, third, and Nth factors. A possibly decade-old security question; a phone number long lost; whatever - before you can use your account.

Note: not "view sensitive data" about the account, like full name, billing address, and contact info. Not "change settings" of the account, such as changing account info, email, etc. Apparently all those are the lowest tier of security meant to be protected by mere "end of the universe"-grade passwords and a second factor such as email, which itself is likely to be sold by a company that also cargo cults N-factor auth. For REAL hard info, let's ask the guy who we just showed the address to "What street he lived in" and a couple others.

Explaining this to the company's support hotline is an exercise in...

"It's for your security."

"It's not. You're just locking me out of my account. I can show you a government ID corroborating all the other account info."

"But we can't, for security."

"It's not security. Get me your boss."

...

"It's for security."

A long long time ago ( 2007 I think ) I worked for a company that made landing sites, so basically an email campaign would go out, users would be sent to a 1 page website with a form to capture their data, ready to be spammed even more. You know how it was back then.

So I worked with a guy who we had just hired, I didn't do the hiring but his CV checked out, so I gave him one of my tasks. Now most pages were made with js and html, with a PHP backend ( called with Ajax). Now this guy didn't know PHP so I was like all good, ASP works too at the end of the day we don't judge, we do like 2 or 3 of these a day and never look at them again. So he goes of and does is thing.

3 weeks later, the customer calls up to me they still haven't received their landing page. Ok so he probably forgot to email the customer np, I tell him to double check he has emailed the customer. Another week goes by end the customer calls back, same problem. At this point I'm getting worried, because we're days away from the deadline and it was originally my task.

So I go back to the guy and I tell him I want that landing page so I can send it myself, half thinking to myself that we had a freeloader, that guy that comes in to companies for 3 weeks, doesn't work, but still cashes his pay. But no, this was much worse.

So he tells me he has finished yet. I ask him why, what's the blocker ? You had 4 weeks to tell me you were blocked and couldn't progress. And his answer was simply, because I wasn't blocked I have been working on it this whole time. So I tell him to zip his project up and email it to me. We didn't do SVN or git back then, simply wasn't worth it. So he comes back to me and says the email server is telling him attachments can't be bigger then 50mb. At this point I'm thinking he didn't properly sized the art or something, so I give him a flash drive to put it on.

When I then open the flash drive, the archive is 300mb, thinking to myself, the images weren't even that big to begin with.
So I open it up, and I don't even find any images, just a single asp page. About 500mb. When I opened that up and it finally loaded, I saw the most horrendous things ever.

The first 500 lines was just initializing empty vars. Then there was some code that created an empty form with an onChange event that submits the form. After that.. it was just non stop nested if's. No loops, no while, for, foreach, NO elseif's, just nested if's, for every possible combination of the state the form could be in. Abou 5000 of them, in a single file. To make matters worse, all the form ( and page ) layout was hardcoded in the if's. Includes inline css, base64 encoded images, nothing but as dynamic, based on the length of the form he changes the layout, added more background etc. He cut the images up for every possible size of the page and included them in the code.

I showed it to my boss, he fired the guy on the spot. I redid the work from scratch, in under 4 hours. Send it to the client. they had no ammends to make, happy as Larry. Whish I kept the code somewhere.

Morale of the story, allways do a coding test on interviews, even if small things just to sanity check.

So this happened a few days ago.

Me: (chilling like a mo'fucka then suddenly an email alert)

*Opens email and realises it's from a recruiter*

Recruiter: We are a venture-based startup out of LA, funded by a top VC. We have developed the first turnkey Serverless Swift platform for app development.

We are looking at expanding our team and we have a few different openings for remote and contract work.

Simply reply to this email with your resume attached.

Me: Thanks for reaching out and presenting me with this opportunity.

I plan on going back to school this January and for that reason I humbly have to bow out.

I will surely keep an eye out for {company_name}. The idea is rather interesting I should say.

*I go back to chilling like a mo'fucka*

*The next day I'm at work, I get an email from the same recruiter again*

Recruiter: Thank you for submitting your resume.

We are expanding our team and are looking for Swift rockstars to join the movement of bringing server-side Swift to the masses.

We were impressed by your resume and wanted to get to know you better. This survey is the first step in that process.

Please take a moment and complete. It should not take longer than 10 minutes.

Me: ...........................
*Calmly walks away from my desk to the bathroom*

WHAT FUCKING RESUME HAVE I SUBMITTED TO YOU? BITCH, MY EMAIL EXPLICITLY SAID: THANKS BUT NO THANKS...

You can't just force an applicant

If you’re a Russian ux engineer who is present in a Russian ux community and you fucking make your form validate on change event and that leads to the situation when a user starts entering their email and your bouba form immediately throws WRONG EMAIL errors, we don’t call you a bouba.

We call you a ебанок (ebanok) — a small, stupid and miserable creature that you can only feel hatred mixed with disgust towards.

This shit is acceptable if you’re an intern making their first shy steps creating their own personal project, but if you push this to production, you’re a ebanok. If you don’t know how to do ux, just use server-side validation or display errors with alerts on submit.

You fucking ebanok.

I used to be a sysadmin and to some extent I still am. But I absolutely fucking hated the software I had to work with, despite server software having a focus on stability and rigid testing instead of new features *cough* bugs.

After ranting about the "do I really have to do everything myself?!" for long enough, I went ahead and did it. Problem is, the list of stuff to do is years upon years long. Off the top of my head, there's this Android application called DAVx5. It's a CalDAV / CardDAV client. Both of those are extensions to WebDAV which in turn is an extension of HTTP. Should be simple enough. Should be! I paid for that godforsaken piece of software, but don't you dare to delete a calendar entry. Don't you dare to update it in one place and expect it to push that change to another device. And despite "server errors" (the client is fucked, face it you piece of trash app!), just keep on trying, trying and trying some more. Error handling be damned! Notifications be damned! One week that piece of shit lasted for, on 2 Android phones. The Radicale server, that's still running. Both phones however are now out of sync and both of them are complaining about "400 I fucked up my request".

Now that is just a simple example. CalDAV and CardDAV are not complicated protocols. In fact you'd be surprised how easy most protocols are. SMTP email? That's 4 commands and spammers still fuck it up. HTTP GET? That's just 1 command. You may have to do it a few times over to request all the JavaScript shit, but still. None of this is hard. Why do people still keep fucking it up? Is reading a fucking RFC when you're implementing a goddamn protocol so damn hard? Correctness be damned, just like the memory? If you're one of those people, kill yourself.

So yeah. I started writing my own implementations out of pure spite. Because I hated the industry so fucking much. And surprisingly, my software does tend to be lightweight and usually reasonably stable. I wonder why! Maybe it's because I care. Maybe people should care more often about their trade, rather than those filthy 6 figures. There's a reason why you're being paid that much. Writing a steaming pile of dogshit shouldn't be one of them.

I'm writing a devrant like site, so a kind of forum that supports live chat under every article. Login will be just username and password to stay anonymous. Email is optional for password reset. Also it won't have password requirements. Who cares if user uses insecure password. I do like the devrant avatar thing. I will use the ducky generator instead. So everyone on the site is a custom duck. K-SASS prolly never expected his generator to be used anywhere. The requirement of this site is that it scales very well. I have db calls of 0.006s, this is for persistent data only and will be used by all site instances. I expect that it can handle many clients concurrent as long I do not return more than 30 rows or so. Events get handled by a self written pubsub server.

All sounds great and development goes fine. But why is this a rant? Because the same thing as always is biting me, I can't design a site at all. I know how but I don't have any feeling for design at all making me almost incapable of building an attractive site. The only thing I can 'design' is an application in bootstrap or smth. I spend so much time one design while I don't like to do it ironically. But looks of site is almost as important as an good working site. Good working site doesn't get used if looks bad in many casee. This is since the start of my career an issue and it sucks that I appearantly can't deliver a whole site on my own meeting my standards.

My backend work is top notch tho. Btw, this application is not to be an alternative for devrant. I do not think I can attract more users than it already has and I've seen two communities disappearing once because someone decided to make a new one, took half of community with him and both communities died after short while.

End product of this project is a working project, not a live site hosted somewhere. It's pure about mixing mostly self written tech to get the best performance. Reinventing wheel on many levels. I wanted maybe to do the site in C but decided that it's way to much work for the value. I change the site so rapid since I don't have decent plan that python aiohttp is the best choice in amount of writing it yourself and fast. It's very lightweight.

More a story than a rant, sorry

@dfox I sent you an email regarding a better, safer implementation of the third-party notification server! Can you take a look?

Best: chief university lab position, 12 yrs as a 👨‍🏫 system engineer teacher, really need a break, updating me as a pro.

Worst: last chief just left email with CISCO passwords. No F* VLANS reference, no technical manual, deleted all Sh* documents on PC.

So I about 4 days no internet on university, reseted 25+ CISCO switches, reorganizing fibers, all week 💤 6am-11pm or more. VTP server core nice and clean, nice VLans, ClearOS formated an licensed, ubnt portal for Wifi.

December, organizing all the administrative stuff. We are back stable and documenting. Moving and painting office, delegation of staff.

Now in vacations with a “tepache 🍻 “ 🍍

So today was interesting.

I had to extract the domain from an email address and compare the domain to a hard coded whitelist, nothing difficult, fuck takes 2 min really.

Except the project starts throwing 500 errors for no god damn reason, like seriously, I double check syntax, nope looks fine, run pho's syntax checker on the file
# php -l /path/to/file.php

Nope says it's all good.
Checks error log on server -> no log

OoooooooooKay then.

Comments out the few lines, saves, errors gone.

remove comments, error comes back.

Do this a few times, and magically the fucking thing stops throwing errors, now I haven't actually changed anything, and I know this project is so fragile I don't know how it stays running at times but fuck me this is a painful joke.

Guess it's time to ditch ProtonMail as well.

Translation: "[Update Google play services.] ProtonMail won't work until you have updated Google Play Services."

I know I'm almost an anti-Google nazi at this point, but ProtonMail was one of the only remaining apps not needing Google to work, and now it's a "sin equa non" condition. I'm very disappointed. Guess I'll have to switch this address to Tutanota or host my own email server.

Edit: why the fuck would my autocorrect capitalize nazi?

Spam assassin kills most of the spam I get before I see it. It works pretty well. However, I started getting a fuck ton of spam from some asshole on a Turkey server. You cannot forward spam to the gov anymore so what to do (They use a honeypot. Apparently it doesn't catch everything.)? Well I got the abuse email account address for the server. Then I went into my servers spam filter for the email address I am having issues with. Then I redirect the email to this abuse email address. Then I delete it from the server. This makes it so my email client never sees the message and I automagically notify the abuse account. If the abuse account is owned by the spammer then he is just filling up his own server with shit.

Anybody else have fun or interesting ways with dealing with spam the regular filters don't catch?

FUCK YES

The feeling when you and the DBA completely fix an issue that has been fucking up your users and that the third party vendors themselves couldn't fix on your own teamwork is so..... fucking... addicting.

Wrote an email to the hod to let us off a bit late tomorrow morning, least I can do for this fucking server admin, sql class A mastermind, Oracle fucking super pro.

I really pray for all of you mfkers to get the same type of coworker. this dude has taught me a lot and I really jump at the first opportunity I get to work with him. His accomplishments for the institution are many really, its just one of those happy bromances man.

I raise my beer mug, to the best fucking DBA i have ever worked with.

For my next trick, I am going to make sure the dude gets the position for the manager of his department as soon as the current dude retires (should be soon) a great man himself, but short on giving his dba the praise he deserves.

The previous manager of my departament told me "pay attention to <DBA NAME> he is your secret weapon and you will be his" and by heavens sweet momma was right.

So here we go again.
Same "web designer", same me, same website (based on wordpress, completely redone front-end UI, full of ACF to keep their fucking data).

WD: ok, I'll need you to add an information on the project page template in order to show a gallery with slider at the bottom and "film+visuals" at the top if there is a gallery.
Me: ok

1 week later, after like 40 e-mails between us deciding if it was ok or not on DEV server and pushing it to PROD

Me: ok, done
WD: hey, if it's just a film there should be written "FILM" while there's nothing showing, also the gallery must have same height as the film above

Me, internally: why the fuck didn't you fucking tell me before pushing to PROD? are you fucking dumb or something?
Me, via email: ok, i'll check it now...

Global variables destroyed my day

Of course you can call me a bad programmer and all. When you have the idea, sometimes globals make it seem easy, accessible and "saving resources". They are devils.

The app was connected to a suite of applications. So I ended up silently destroying it neighbors. I committed and pushed the shit. Just when the testers made their weed high smoke tests, a server stopped working.

I got an email that boss wanted the latest version, I reverted the wrong branch, which had code unrelated to its name, pushed shit again and voila.

I went to the bathroom and laughed. I had to take a smoke. I'm still laughing while typing this. The damage is too much and I can't help it. I'll go home, leave this pc on and work remotely through the night. It might be the hysterics speaking now, but I messed up, and I need magic by friday morning.