Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "call graph"
-
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
Had 2 days of vacation. Theoretically (plus weekend, plus 2 days) 6 days.
Worked today… At Saturday.
Some administrators forgot to properly check bandwidth limitations....
*rolls eyes*
We had a major version upgrade of some server software at Monday.
Guess why I got called...
Of course it MUST be the software upgrade.
It couldn't be the new hardware that was setup 2 weeks ago and on which a lot of "important" VMs were migrated.
*eyes roll inside till only white is visible*
The even more annoying thing is that it wasn't that hard to figure out.
Looking at monitoring, we had spikes on 20 Gbit/s (roughly 2.x Gigabyte/sec - Ethernet) connection of some server at roughly 1.9 plus Gigabyte/sec.
IO latency spikes that made the graph look like a heartbeat EKG with severe tachycardia...
*additionally to white eyes starts cursing in reverse latin*
Incompetent admin answer: Booboo that can only be your fault - the developers must investigate.
Me (just a tad more polite): Meep Meep mother fucker, get your shit together. If the software would eat that much, the network would be a niece chunk of charcoal. Plus the time (sending instead of links to monitoring pictures… guess the lazy fucktard who's brain is a vacuum didn't even bother to check it)...
NOTICE SOMETHING?!
Incompetent admin: It starts at the same time. Always.
After wasting roughly another hour of time discussing with him, I just hanged up the video call.
Called someone I knew from the admin department and turns out that - drumrolls please - the incompetent admin was someone who got recruited 3 months ago…
*turning into antichrist*
I then had a not so polite discussion about how the only competent people could take days off (all except incompetent admin were on vacation) and the seemingly incompetent fresh recruit - who by the way NEVER mentioned this - was the only one left of the admin department. Which would be bad alone, but no - he even got the 24/7 emergency support role for the whole weekend.
Sometimes this company and HR especially notoriously drive me insane...
Guess next week there will be some HR barbecue.
But yeah. After a lot of raging around we nailed it down to the traffic of backups and could fix it.
Roughly 4 hours of analysis, communication, raging and hatred.
Just one hour implementing shit.
*goozfraba*11 -
It’s now day 4 into handing in my notice. Here's a recap of day 1&2. Here's the recap of day 0: https://www.devrant.io/rants/871145
I handed in my notice on Wednesday with a leaving date of 10/27/17:
> format_date('27/10/17', 'short', 'muurcan');
Thursday, I had an appointment outside of the office... I was called by a marketing guy at [popular graph database company] to try and wiggle his way into my org. I forget his name, so we'll call him Derek:
Derek: 'Hi James, it’s marketer at [graph co] here; I know you downloaded our free book two months ago and we reserved the right to call you constantly since. I just wanted to...'
Me: 'Hol up Derek! I don’t want to waste your time, thank you guys for the book.
I’d have happily paid to avoid these phone calls.
I’ve resigned from [company] before getting a chance to introduce [most popular graph database platform on google, for real, go check now].
Again thanks, but I’m no longer a useful lead.'
Life lesson learned: free doesn’t mean free, free books aren’t worth shit. Marketing people are lovely... but have an job to do so they’re also basically all cunts.
If you want to learn graph DB best practices from oreilly, pay the £7 and be done with it.
Don’t download that book! Derek will take your number and use it like you’re a young naive college girl with a golden pička.
Aside: I’ve met a new girl! I’ve rapidly learned Slovenian swear words. She’s a beautiful Slovenian girl and has the mouth of a sailor. Peace out to any of my eastern euro buddies on here. Privyet, serbus, stay frigging awesome.
I'll be following up on the tag 'jct resigns' for anyone interested.4 -
After a few weeks of being insanely busy, I decided to log onto Steam and maybe relax with a few people and play some games. I enjoy playing a few sandbox games and do freelance development for those games (Anywhere from a simple script to a full on server setup) on the side. It just so happened that I had an 'urgent' request from one of my old staff member from an old community I use to own. This staff member decided to run his own community after I sold mine off since I didn't have the passion anymore to deal with the community on a daily basis.
O: Owner (Former staff member/friend)
D: Other Dev
O: Hey, I need urgent help man! Got a few things developed for my server, and now the server won't stay stable and crashes randomly. I really need help, my developer can't figure it out.
Me: Uhm, sure. Just remember, if it's small I'll do it for free since you're an old friend, but if it's a bigger issue or needs a full recode or whatever, you're gonna have to pay. Another option is, I tell you what's wrong and you can have your developer fix it.
O: Sounds good, I'll give you owner access to everything so you can check it out.
Me: Sounds good
*An hour passes by*
O: Sorry it took so long, had to deal with some crap. *Insert credentials, etc*
Me: Ok, give me a few minutes to do some basic tests. What was that new feature or whatever you added?
O: *Explains long feature, and where it's located*
Me: *Begins to review the files* *Internal rage wondering what fucking developer could code such trash* *Tests a few methods, and watches CPU/RAM and an internal graph for usage*
Me: Who coded this module?
O: My developer.
Me: *Calm tone, with a mix of some anger* So, you know what, I'm just gonna do some simple math for ya. You're running 33 ticks a second for the server, with an average of about 40ish players. 33x60 = 1980 cycles a minute, now lets times that by the 40 players on average, you have 79,200 cycles per minute or nearly 4.8 fucking cycles an hour (If you maxed the server at 64 players, it's going to run an amazing fucking 7.6 million cycles an hour, like holy fuck). You're also running a MySQLite query every cycle while transferring useless data to the server, you're clusterfucking the server and overloading it for no fucking reason and that's why you're crashing it. Another question, who the fuck wrote the security of this? I can literally send commands to the server with this insecure method and delete all of your files... If you actually want your fucking server stable and secure, I'm gonna have to recode this entire module to reduce your developer's clusterfuck of 4.8 million cycles to about 400 every hour... it's gonna be $50.
D: *Angered* You're wrong, this is the best way to do it, I did stress testing! *Insert other defensive comments* You're just a shitty developer (This one got me)
Me: *Calm* You're calling me a shitty developer? You're the person that doesn't understand a timer, I get that you're new to this world, but reading the wiki or even using the game's forums would've ripped this code to shreds and you to shreds. You're not even a developer, cause most of this is so disorganized it looks like you copy and pasted it. *Get's angered here and starts some light screaming* You're wasting CPU usage, the game can't use more than 1 physical core, and after a quick test, you're stupid 'amazing' module is using about 40% of the CPU. You need to fucking realize the 40ish average players, use less than this... THEY SHOULD BE MORE INTENSIVE THAN YOUR CODE, NOT THE OPPOSITE.
O: Hey don't be rude to Venom, he's an amazing coder. You're still new, you don't know as much as him. Ok, I'll pay you the money to get it recoded.
Me: Sounds good. *Angered tone* Also you developer boy, learn to listen to feedback and maybe learn to improve your shitty code. Cause you'll never go anywhere if you don't even understand who bad this garbage is, and that you can't even use the fucking wiki for this game. The only fucking way you're gonna improve is to use some of my suggestions.
D: *Leaves call without saying anything*
TL;DR: Shitty developer ran some shitty XP system code for a game nearly 4.8 million times an hour (average) or just above 7.6 million times an hour (if maxed), plus running MySQLite when it could've been done within about like 400 an hour at max. Tried calling me a shitty developer, and got sorta yelled at while I was trying to keep calm.
Still pissed he tried calling me a shitty developer... -
Spaghetti Code, Spaghetti Code.
Flush it right down the commode,
Spins a call graph, unreadable size,
Anyone who sees it cries.
Look out, here comes the Spaghetti Code!2 -
Can someone explain me AI/ML/DL in traditional algorithmic way without AI jargons?
What I currently understand is that they convert the training data to numbers based on a complex black boxed mathematical algorithm and then when a new data comes in, the same conversion is done and a decision is taken based on where the the new number fits in within the geometry/graph plot of the old numbers from training. The numbers are then updated. Is this what they call AI? Nearest number/decision search?
Kindly try to avoid critic, I am having a difficult time understanding the already trending AI stuff. People say that the algo exists from long back but only now we have the compute power.22 -
Working on a project, got stuck, thought to call it a night but nooo. Fucking brain decided to come up with a solution 0.0000001 ms right after I shut my eyes. Now I am way too tired to open my laptop back again and I am forced to use Facebook Graph API on my cellphone. FML
-
I had a pretty good year! I've gone from being a totally unknown passionate web dev to a respected full stack dev. This will be a bit lengthy rant...
Best:
- Got my first full time employment dev role at a company after being self-taught for 8+ years at the start of the year. Finally got someone to take the risk of hiring someone who's "untested" and only done small and odd jobs professionally. This kickstarted my career, super grateful for that!
- Started my own programming consulting company.
- Gained enough confidence to apply to other jobs, snatched a few consulting jobs, nailed the interviews even though I never practiced any leet code.
- Currently work as a 99% remote dev (only meet up in person during the initialization of some projects.) I never thought working remotely could actually work this well. I am able to stay productive and actually focus on the work instead of living up to the 9-5 standard. If I want to go for a walk to think I can do that, I can be as social and asocial as I want. I like to sleep in and work during the night with a cup of tea in the dark and it's not an issue! I really like the freedom and I feel like I've never been more productive.
- Ended up with very happy customers and now got a steady amount of jobs rolling in and contracts are being extended.
- I learned a lot, specialized in graph databases, no more db modelling hell. Loving it!
- Got a job where I can use my favorite tools and actually create something from scratch which includes a lot of different fields. I am really happy I can use all my skills and learn new things along the way, like data analysis, databricks, hadoop, data ingesting, centralised auth like promerium and centralised logging.
- I also learned how important softskills are, I've learned to understand my clients needs and how to both communicate both as a developer and an entrepeneur.
Worst:
- First job had a manager which just gave me the specifications solo project and didn't check in or meet me for 8 weeks with vague specifications. Turns out the manager was super biased on how to write code and wanted to micromanage every aspect while still being totally absent. They got mad that I had used AJAX for requests as that was a "waste of time".
- I learned the harsh reality of working as a contractor in the US from a foreign country. Worked on an "indefinite" contract, suddenly got a 2 day notification to sum up my work (not related to my performance) after being there for 7+ months.
- I really don't like the current industry standard when it comes to developing websites (I mostly work in node.js), I like working with static websites (with static website generators like what the Svelte.js driver) and use a REST API for dynamic content. When working on the backend there's a library for everything and I've wasted so many hours this year to fix bugs and create workarounds related to dependencies. You need to dive into a rabbit hole for every tool and do something which may work or break something later. I've had so many issues with CICD and deployment to the cloud. There's a library for everything but there's so many that it's impossible to learn about the edge cases of everything. Doesn't help that everything is abstracted away, which works 90% of the time but I use 15 times the time to debug things when a bug appears. I work against a black box which may or may not have an up to date documentation and it's so complex that it will require you to yell incantations from the F#$K
era and sacrifice a goat for it to work properly.
- Learned that a lot of companies call their complex services "microservices". Ah yes, the microservice with 20 endpoints which all do completely unrelated tasks? -
What do you call a function based on (w*x) / (y*z)
where w and y remain the same, and x always increases linearly with z?
For example
at one point on the graph it might look like
(60*1) / (25*1)
and on the next it equals
(60*2) / (25*2)17 -
Rant
Fucking async tasks in c# are the bane of my existence right now.
It's running one to call the ms graph API and then if you click at the wrong moment it throws an exception and crashes to desktop but I can't bloody reproduce it because the API call is so fast3