Wow, what a fucking mess this sunday was.
My boss wrote me an email that one route of a RESTful API we wrote for a customer was not working anymore and puking back a status 500 with some error mentioning invalid UTF-8 characters.
Not one single person has had touched nor changed the code on production in some 6 months, so what the fuck could it be?

Phpunit did not give any errors (running only locally), the code had no syntax errors and the DB dump did not contain any invalid bytes (tested with a hex editor).

WHAT THE FUCK?!

OK so I started to comment out lines (all tested directly on production of course) until the error vanished.

Guess what was the culprit?
.
.
.
.
.
.
In the code (PHP) we used strftime(...) to get nice time strings. Of course we set the correct locale on the server, thus having months and days formatted in German.
So, in Geman there is this one mysterious month called "März" which contains an umlaut character.
Calling strftime generated the date with März in it, but the server locale was de_CH.iso-8859-1 and not fucking de_CH.utf8, so the "ä" was returned as 0xE4 instead of 0xC3A4 (valid UTF-8), which json_encode(...) did not want to swallow but instead threw an exception.

*wrestling commentator voice*
"In this weeks episode of encoding hell:
The iiiinnnfamous UTF-8 Byte Order Mark veeeersus PHP!"

For an online shop we developed, there is currently a CSV upload feature in review by our client. Before we developed this feature, we created together with the client a very precise specification, including the file format and encoding (UTF-8).

After the first test day, the client informed us, that there were invalid characters after processing the uploaded file.
We checked the code and compared the customer's file with our template.
The file was encoded in ISO-8859-1 and NOT as specified UTF-8.
But what ever, we had to add an encoding check, thus allowing both encodings from now on.

Well well well welly welly fucking well...

Test day 2: We receive an email from said client, that the CSV is not working, again.
This time: UTF-8 encoding, but some fields had more colums with different values than specified.
Fucking hell.
We tell the customer that.
(I was about to write a nice death threat novel to them, but my boss held me back)

Testing day 3, today:
"The uploading feature is not working with our file, please fix it."
I tried to debug it, but only got misleading errors. After about 30 minutes, at 20 stacks of hatered, I finally had an idea to check the file in a hex editor:

God fucking what!?!!?!11?!1!!!?2!!

The encoding was valid UTF-8, all columns and fields were correct, but this time the file contained somthing different.
Something the world does not need.
Something nearly as wasteful as driving a monster truck in first gear from NYC to LA.

It was the UTF-8 Byte Order Mark.
3 bytes of pure hell.
Fucking 0xEFBBBF.
The archenemy of PHP and sane people.

If the devil had sex with the ethernet port of a rusty Mac OS X Server, then 9 microseconds later a UTF-8 BOM would have been born.

OK, maybe if PHP would actually cope with these bytes of death without crashing, that would be great.

For fuck's sake! I just spent almost an entire workday trying to fix a bug which didn't even exist in the first place.
Well, at least now I got this to cheer me up:

Yesterday a strange bug appeared in Chrome: In a small web app we have some umlaut characters like äöü. Strangely said characters were displayed as cyrillic, but only on my pc... On every other device it worked. I spent about 5 hours of checking encodings (everything was in UTF-8), reading posts in the almighty stackoverflow.
Finally i figured out, the font was broken. After reinstalling it, everything was peaceful again in my head.

REDIS: Great for cloud, will fuck up your local disk if too many write operations per second.

DynamoDB: WTF 10Mb should not be "too large for a single record"!!

SPARK: NEVER CONNECT IT TO A DATABASE! Wasted A LOT of cluster time. Also, can you be LESS specific on exactly what are the bugs in my code? 'cause I don't think it's possible.

NPM: can't install a package for shit. tried it waaaay to many times.

Makefiles: Just fuck you.

WSL1: breaks more often than a glass hammer.

Python >= 3.6: FUCK ENCODINGS!!

Jupyter: STOP MESSING UP WHILE SAVING!

Living is to collet bugs, it seems.

Hmm...recently I've seen an increase in the idea of raising security awareness at a user level...but really now , it gets me thinking , why not raise security awareness at a coding level ? Just having one guy do encryption and encoding most certainly isn't enough for an app to be considered secure . In this day an age where most apps are web based and even open source some of them , I think that first of all it should be our duty to protect the customer/consumer rather than make him protect himself . Most of everyone knows how to get user input from the UI but how many out here actually think that the normal dummy user might actually type unintentional malicious code which would break the app or give him access to something he shouldn't be allowed into ? I've seen very few developers/software architects/engineers actually take the blame for insecure code . I've seen people build apps starting on an unacceptable idea security wise and then in the end thinking of patching in filters , encryptions , encodings , tokens and days before release realise that their app is half broken because they didn't start the whole project in a more secure way for the user .

Just my two cents...we as devs should be more aware of coding in a way that makes apps more secure from and for the user rather than saying that we had some epic mythical hackers pull all the user tables that also contained unhashed unencrypted passwords by using magix . It certainly isn't magic , it's just our bad coding that lets outside code interact with our own code .

Co-worker: "I'm getting files from customers which are in multiple types and encodings. Is there a way to force them to send only us-ascii so my tool gets only one kind which is easier to parse?"

Me: "Welcome to programming with arbitrary sources; you'll just have to build your tools to figure it out and not break (just rely on dozens of existing tools which you refuse to use)"

It's always fucking file encodings breaking these stupid fucking legacy projects. FUCK

Fuck this I'm going to bed. I'll fix it in the morning it's a little past midnight

FUCK YOU MICROSOFT
GO FIX YOUR FUCKIN C# METHODS

Language felt good but jesus fuckin christ.

HOW YOUR File.Exists() can be so retarded jesus fuckin christ
I mean god, how retarded can it be when i obtain the current directory with your builtin method (System.Environment.CurrentDirectory) attach to it the directory name with the images i need and I ALWAYS GET FALSE ABOUT ITEMS THAT ARE FUCKIN THERE.

Fix your fuckin encodings too, suckers.

Microsoft certsrv is returning UTF-8 on the authorization error page but UTF-16 when logging in via basic auth...

Debugged this for 2 hours today to parse the response correctly. Thanks Microsoft

What is the problem with nowadays colleges? So many of the programmers I know know absolutely nothing regarding encodings and such.
It's pretty nice to see people resorting to hexadecimals to show their Latin characters in HTML instead of simply learning about bloody UTF8

I was in my IDE writing up a new program, but my compiler didn't accept the encoding format of the files. Because I used special characters, it flipped a lot of encodings around. Then I changed my projects master encoding, then all of the source files they all messed up. The all the files were composed of "????????" I went to a backup and they were messed up too. I exploded... Control Z couldn't even be my hero this time.

Encodings.

Fuck powershell.

How in the world can one lang have so much problems with date conversion, and chars like üöä. Its so funny how they get through just fine sometime when u specify encoding utf8 but other times it has no ides wha5 these chars are.
Honestly fuck powershell