Just saw how Git confirms UTF8-Support... with an UTF8-Character. 👍

UTF-8 gone wrong

When we found out MySQL utf8 isn't actually utf8..... it's a proprietary subset of utf8 that only includes up to Unicode 255..... and there is a separate "utf8mtb4" that is actual utf8.

So apparently using utf8 with LaTeX isn't enough to get € symbol working, I have to include two more packages for that (two, because one handles the character, and the other one makes it look "not weird"). How hard can it be to just use a single utf8 font, and have it work with whatever utf8 character you type in? And apparently there are at least 3 packages specifically for euro symbol - a certain XKCD comic comes to mind...

Wow, what a fucking mess this sunday was.
My boss wrote me an email that one route of a RESTful API we wrote for a customer was not working anymore and puking back a status 500 with some error mentioning invalid UTF-8 characters.
Not one single person has had touched nor changed the code on production in some 6 months, so what the fuck could it be?

Phpunit did not give any errors (running only locally), the code had no syntax errors and the DB dump did not contain any invalid bytes (tested with a hex editor).

WHAT THE FUCK?!

OK so I started to comment out lines (all tested directly on production of course) until the error vanished.

Guess what was the culprit?
.
.
.
.
.
.
In the code (PHP) we used strftime(...) to get nice time strings. Of course we set the correct locale on the server, thus having months and days formatted in German.
So, in Geman there is this one mysterious month called "März" which contains an umlaut character.
Calling strftime generated the date with März in it, but the server locale was de_CH.iso-8859-1 and not fucking de_CH.utf8, so the "ä" was returned as 0xE4 instead of 0xC3A4 (valid UTF-8), which json_encode(...) did not want to swallow but instead threw an exception.

I like when american companies adopt Unicode ;)

Yesterday, after six months of work, a small side project ran to completion, a search engine written in django.

It's a thing of beauty, which took many trials, including discovering utf8 in mysql isn't the full utf8 spec, dealing with files that have wrong date metadata, or even none at all, a new it backup policy that stores backups along side real data.

Nevertheless, it is a pretty complete product. Beaming with pride I began to get myself a drink, and collapsed onto the floor, this caused me to accidentally hibernate my computer, which interrupted the network connection, which in turn caused an OSError exception in one of my threads, which caused a critical part of code not to run, which left a thread suspended, doing nothing.

From the floor I looked at my error and realised my hubris and meditated on my assumptions that in theory nothing should interrupt a specific block of code, but in reality something might, like someone falling over...

Fuck mysql charset shit. Pretty easy to forgot setting charset in server side config file. And after creating several databases/tables, boom! All in latin1. That's just so convenient. are you kidding me? Why would any modern database have their default charset to be latin1??? rather than utf8? Oh, just forgot again, that's utf8fuckingmb4 on mysql land.

Shoutout to MySQL's shitty support for utf8

2008: Back in the days when Vodafone used emojis for their contracts. Cool.

Screw MySQL/MariaDB. Who the fuck thought not to document that utf8 is only 3 bytes and not unicode capable. You have to use utf8mb4 for it to work. Fuck those idiots that don't want to admit they made a mistake and put this info in a footnote on a pricy documentation piece.

Yesterday a strange bug appeared in Chrome: In a small web app we have some umlaut characters like äöü. Strangely said characters were displayed as cyrillic, but only on my pc... On every other device it worked. I spent about 5 hours of checking encodings (everything was in UTF-8), reading posts in the almighty stackoverflow.
Finally i figured out, the font was broken. After reinstalling it, everything was peaceful again in my head.

I was working on this mustachio fork called Morestachio and changed the Default Encoding from "Encoding.Default" to UTF8 while creating a .NetStandard version of it. After i have created everything i thought "Hey just for the lolz run my tests again" ...

Another Unicode rant. How is it still not the default?!?

For fucks sake! It's 2018 and MS™ Excel™ is still not able to store a file in UTF-8...
And neither can you choose the separators when opening a CSV.

Go eat a bag of corporate dicks and greedily choke on it to an agonizing death.

So I enventually spent 2 years working for that company with a strong b2b market. Everything from the checkouts in their 6 b2c stores to the softwares used by the 30-people sales team was dependant on the main ERP shit home-built with this monstruosity we call Windev here in France. If you don't know it just google and have some laugh : this is a proprieteray FRENCH language. Not french like made by french people, well that too, but mostly french like the fucking language is un fucking french ! Instructions are on french, everything. Hey that's my natural language okay, but for code, really ?

The php website was using the ERP database too, even all the software/hardware of the massive logistic installation they had (like a tiny Amazon depot), and of course the emails of all employees. Everything was just handled by this unique shitty and so sloooooow fucking app. When there was to many clients on the website or even too many salespeople connected to the ERP at the same time, every-fuckin-piece of the company was slowing down, and even worse facing critical bugs. So they installed a monitor in the corner of a desk constantly showing the live report page of Google analytics and they started panic attacks everytime it was counting more than 30 sessions on the website. That was at the time fun and sad to observe.

The whole shit was created 12 years ago and is since maintened locally by one unique old-fashion-microsoft dev who also have to maintain all the hardware of all the fucking 150+ people business. You know, when the keyboard of anyone is "broken" cause it's unplugged... That's his job too. The poor guy was totally overstressed on a daily basis and his tech knowledge just saddly losts themeselves somewhere in the way. He was my n+1 in a tech team of 3 people : him, a young and inexperimented so-called "php developer" who was in charge of the website (btw full of security holes I discovered and dealed with when I first arrive at the job), and myself.

The database was a hell of 100+ tables of business and marketing data with a ton of specific logic added on-the-go during years. No consistent data model or naming. No utf8. Fucked up relations that ends with queries long enough to fill books. And that's not all, all the customers passwords was just stored there uncrypted. Several very big companies and administrations were some of these clients. I was insisting on the passwords point litterally all the time, that was an easy security fix and a good start... But no, in two years of discussions on the subject I never achieved to have them focusing on other considerations than "our customers like that we can remind them their password by a simple phone call if they lost it". What. The. Fuck. WHATTHEFUCK!

Eventually I ran myself out of this nightmare. I had a few bad jobs already, and worked on shitty software already. But that one really blows my mind (and motivation for a time too). Happy it's over.

Whenever I want to type the shrug emoji, I copy it from a Google note ¯\_(ツ)_/¯

I can type emojis just fine, enjoy a 🍓, yet there is a clear lack of accessibility for ASCII-emojis input on any device.

How is this fair!?

PS: I do know that this shrug is actually utf8 and not ascii. But that's beside the point. Also try saying “utf8 emoji” three time in a row.

Todays rant is about me trying to add some long text into my database. I tried it all day long, but the text was inserted partially all the time. I changed the collumns data type to BLOB, this felt false, but it seemed to work. The bad feeling triggered me to search further, so I rewrote my code and found the source of this behavior. I used utf8-decode-function on my text and that triggered some problems when inserting the text. I don't completely understand it, but I solved the mystery, that fucked up the day. I will sleep good now.

How it started:

Need to replace in a lot of SQL files certain stuff...

find . -type f -iname '*.sql' -exec sed -i 's|new|old|g' {} \;

12 hours later that find executed a shell script containing roughly 120 lines of text pipelining.

The jolly of inconsistent workflows.

Different SQL format stylings... Makes fun when single line string replace needs to be extended to multiline RegEx handling. Or matching SQL comment configuration..

Different line endings. MacOS, Windows, Unix, Bukkake.

Different charsets / collations. Anyone wants latin1_swedish_ci... utf8... utf16... :/

Realizing some people even left sensitive data inside the SQL files (e.g. API Tokens..... Yayyyyyyy).

...

Ugh. It's never a one liner. It's never easy. -.-

I hate cleaning up messy shit.

Saving files in UTF-8
Korean becomes a mess when I save in ANSI

I spent hours trying to support \n, \r and \r\n in my algorithms to convert between utf16 line/col and utf8 absolute indices to comply with the LSP, before realizing that

- Orchid itself doesn't support \r
- There is no established user base

so I'm in the best position to reject all files that contain \r and offer to convert them instead.

I've got 99 problems but UTF8'nt one. 😀 Best I could come up with.

In some cases only emojis are not encoded properly - dont know how to replicate - dont know why its happening (made sure im setting to utf8)

a lot of dev have a miss concept about Unicode/utf8 including me but I believe my understanding get Better and this my last version.
For a project i was developing a rest api for mobile app
when an ios dev asked me
"I send you Unicode string but it appears as ????? in admin web panel "
OMG!!!😨😨😨
Unicode is not an encoding nor an algorithm. it's a standerd which just map a glyph to a codepiont .
but utf8 is the encoding of Unicode and how it's stored or transferred ,
the string you send must be a utf-8 encoded string as the rest of the json you sent .

What is the problem with nowadays colleges? So many of the programmers I know know absolutely nothing regarding encodings and such.
It's pretty nice to see people resorting to hexadecimals to show their Latin characters in HTML instead of simply learning about bloody UTF8

I remember playing games like wolfenstein3D, supaplex, sokoban etc. on our family computer 486 which had as I remember around 100mhz processor. (120mhz in TURBO mode)
Yeah and I did created a few levels in wolfenstein, there was a simple editor.

From programming view I did code my first website only using html and inline css in early 2000s. However internet was a thing of a rich people back then (in my country), so my brother downloaded the whole website with docs and basics of html/css/js for me in collage. My first website was coded on 300mhz pentium2 (or 3?) with no internet connection, took me about a two months to complete and was total mess. But was graphically satisfying with nice gifs which took tens od seconds to download. Main container had 600px width and looked pretty good on my 800x600 resolution.
I still remember messing with BOM signature because of notepad could not save a file without BOM. Leaving all utf8 chars as mess after saving.

Good old times.

Please help me before I get mad,
First day with Linux Mint.
Objective: Make a 3Tb Hdd Read and Write, Right now I can use it only to Read.
Finally Installed Linux after some bumps (bad ISO).
I have 2 HDDs, the SSD with Linux and a 3Tb HDD
Right now the 3T has 4 partitions, one for windows, 3 for personal use with lots of personal stuff I can't lose.
I've been looking for videos, tutorials and the maximum I got was to had one partition mounted as a folder
<code>
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda1 during installation
UUID=f0a65631-ccec-4aec-bbf5-393f83e230db / ext4 errors=remount-ro 0 1
/swapfile none swap sw 0 0
UUID=F8F07052F07018D8 /mnt/3T_Rodrigo ntfs-3g rw,auto,users,uid=1000,gid=100,dmask=027,fmask=137,utf8 0 0
</code>

What am I missing?
PS.: Next: Make fingerprint work in Linux

I had to program a webservice in PHP (never touched it before). First of all, i get why this language is so much hated. Second i had to debug one function for 5h because of 'é' in my DB.... UTF8 FTW i guess.

Let's finish this by a pun

PHP is really a rich language.

So, spammers found the special character catalog on their machines.

Want a simple but terrible annoying prank?
Change the keyboard map from UTF8 to ASCII or vice versa and set the system font to something funky like a Greek or Cyrillic variant... :)

The one thing I need, a "LATIN SMALL LETTER REVERSED C WITH DIAERESIS" doesn't seem to exist but "LATIN SMALL LETTER C WITH DIAERESIS" the one thing that I don't need, does D:

Like there's a god dang sideways u with an umlaut (or diaeresis I guess) but not a reversed c̈? pls

Came across a book by Clinton L. Jeffery that details in programming language design, a topic that has always fascinated me. So I went ahead and bought the book knowing full well it uses an obscure language called Unicon (cool fucking name) devised in order to mimic the Icon programming language (obscure as well) which are languages that detail goal oriented programming. While I do not mind the language itself, seems pretty good for my taste, does not use curly braces or semicolons and a lot of other scripty things, gets compiled to bytecode and works well, but shit man, trying to find documentation for this outside of its own (I don't like it) book is a pain in the ass. To give some perspetive: you know you are dealing with some obscure shit when there ain't any youtube videos on the language. It has some interesting notions, but I just fucking hate the "documentation standard" book that it has for it, and yes, this is because the language has not taken any actual traction from the masses, there are some things that it does not have such as full utf8 support among other things, it really is a nice tech but I hate the lack of proper documentation/tutorials on it.

rant off

"Values for NLS_LANG differs between the system (UTF8) and the database (UTF-8)" really? REALLY?

I spent fuckin whole day on a defect..
I made a cma content using fatwire in which I added &pound; on a html snippet and it converts to (£) and when I use it in page it shows £ even it's in utf8..
Can anybody help?

How's that character encoding going McDonalds?

Fuck powershell.

How in the world can one lang have so much problems with date conversion, and chars like üöä. Its so funny how they get through just fine sometime when u specify encoding utf8 but other times it has no ides wha5 these chars are.
Honestly fuck powershell