Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "server pool"
-
Contrary to most people I really love to receive email related to jobs when I'm in holiday. I keep important alerts on.
It's like:
email: ***urgent, server down***
me (sipping mojito by the pool): fuck them. let's them deal with that
email: ***requirements all wrong, must develop the feature again***
me (enjoying a dinner): oh, I told them 100 times!, fuck all of them, work for me now, stupid moron.
email: I destroyed by mistake the db with an update..."
me (dancing like crazy): ahahaha I told you that support guys should not have access to production db, fuckfuck you, fix it yourself!!!
and so on..... I don't know, it just boost my pleasure during holiday.9 -
Dev checked in code (I suspect purposely not inviting me on the code review invite) saying he "fixed" the authentication bug in the web service.
Um no, like I told you last week, the authentication error is because the load balancer wasn't passing the user's authentication to IIS.
If I didn't overhear him telling a user "Still getting the error? I don't know, we might have to re-write that service", he might have gotten away with it.
Me: "Wait, that doesn't sound right. If I hit the server directly, authentication works. Its an issue with the load balancer, not the service"
Dev: "Admin said the load balancer is fine and it has to be the service."
Me: "I don't buy it. IIS is returning the authentication error, not the service."
Dev: "I added exception handling and nothing is being logged. Must be something in the service configuration."
Me: "No, IIS performs the authentication, not the service. I explained that last week, remember?"
Dev: "Oh yea. What changes do we need to make to the service?"
<my blood pressure starts to spike>
Me: "None. Give me a sec.."
<we have other apps on the same server farm that work just fine, so I re-configure the service pool settings to match theirs>
Me: "See, now going through the load balancer, the service works fine. For some reason, the admin had our service set up differently."
Dev: "OK, I'll let the users know the service is fixed."
Me: "Service was never broke and I'm not leaving it in its current state. In the morning I'll talk to the admin and see what he can do to fix."6 -
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
My Sunday Morning until afternoon. FML. So I was experiencing nightly reboots of my home server for three days now. Always at 3:12am strange thing. Sunday morning (10am ca) I thought I'd investigate because the reboots affected my backups as well. All the logs and the security mails said was that some processes received signal 11. Strange. Checked the periodics tasks and executed every task manually. Nothing special. Strange. Checked smart status for all disks. Two disks where having CRC errors. Not many but a couple. Oh well. Changing sata cables again 🙄. But those CRC errors cannot be the reason for the reboots at precisely the same time each night. I noticed that all my zpools got scrubbed except my root-pool which hasn't been scrubbed since the error first occured. Well, let's do it by hand: zpool scrub zroot....Freeze. dafuq. Walked over to the server and resetted. Waited 10 minutes. System not up yet. Fuuu...that was when I first guessed that Sunday won't be that sunny after all. Connected monitor. Reset. Black screen?!?! Disconnected all disks aso. Reset. Black screen. Oh c'moooon! CMOS reset. Black screen. Sigh. CMOS reset with a 5 minute battery removal. And new sata cable just in cable. Yes, boots again. Mood lightened... Now the system segfaults when importing zroot. Good damnit. Pulled out the FreeBSD bootstick. zpool import -R /tmp zroot...segfault. reboot. Read-only zroot import. Manually triggering checksum test with the zdb command. "Invalid blckptr type". Deep breath now. Destroyed pool, recreated it. Zfs send/recv from backup. Some more config. Reboot. Boots yeah ... Doesn't find files??? Reboot. Other error? Undefined symbols???? Now I need another coffee. Maybe I did something wrong during recovery? Not very likely but let's do it again...recover-recover. different but same horrible errors. What in the name...? Pulled out a really old disk. Put it in, boots fine. So it must be the disks. Walked around the house and searched for some new disks for a new 2 disk zfs root mirror to replace the obviously broken disks. Found some new ones even. Recovery boot, minimal FreeBSD Install for bootloader aso. Deleted and recreated zroot, zfs send/recv from backup. Set bootfs attribute, reboot........
It works again. Fuckit, now it is 6pm, I still haven't showered. Put both disks through extensive tests and checked every single block. These disks aren't faulty. But for some reason they froze my system in a way so that I had to reset my BIOS and they had really low level data errors....? I Wonder if those disks have a firmware problem? So that was most of my Sunday. Nice, isn't it? But hey: calm sea won't make a good sailor, right?3 -
At work we have a....err....pool of usefulness awareness situation.
You see. The school does not know all that me and the lead developer does. Our head of department does. Our manager used to do(fore she left) but as far as the rest of the 2 fucking campuses are concerned, all of our shit works through magical dwarves that live inside the servers in our server rooms.
Normally, we are fine with this. But whenever one of us misses, or decides to take the day off and some shit hits the fan...THEN and only then do our users know what we do.
So we have a "pool" of items that seem to be picked every time one of us is not there. The more it gets picked, the more people are aware of us, the more they figure out what we do...the more they start to ask for customization and shit like that.
Such a funny situation. Wonder if this will get in the way of friday morning coffee and magic the Gathering at the office.
Or me and the cms admin playing videogames together.
Oh well -
At work, my closest relation is with the DBA. Dude is a genius when it comes to proper database management as well as having a very high level of understanding concerning server administration, how he got that good at that I have no clue, he just says that he likes to fuck around with servers, Linux in particular although he also knows a lot about Windows servers.
Thing is, the dude used to work as a dev way back when VB pre VB.NET was all the rage and has been generating different small tools for his team of analysts(I used to be a part of his team) to use with only him maintaining them. He mentioned how he did not like how Microsoft just said fk u to VB6 developers, but that he was happy as long as he could use VB. He relearned how to do most of the GUI stuff he was used to do with VB6 into VB.NEt and all was good with the world. I have seen his code, proper OOP practices and architectural decisions, etc etc. Nothing to complain about his code, seems easy enough to extend, properly documented as well.
Then he got with me in order to figure out how to breach the gap between building GUI applications into web form, so that we could just host those apps in one of our servers and his users go from there, boy was he not prepared to see the amount of fuckery that we do in the web development world. Last time my dude touched web development there was still Classic ASP with JScript and VBScript(we actually had the same employer at one point in the past in which I had to deal with said technology, not bad, but definitely not something I recommend for the current state of web development) and decided that the closest thing to what he was used was either PHP(which he did not enjoy, no problem with that really, he just didn't click with the language) and WebForms using VB.NET, which he also did not like on account of them basically being on support mode since Microsoft is really pushing for people to adopt dotnet core.
After came ASP.NET with MVC, now, he did like it, but still had that lil bug in his head that told him that sticking to core was probably a better idea since he was just starting, why not start with the newest and greatest? Then in hit(both of us actually) that to this day Microsoft still not has command line templates for building web applications in .net core using VB.NET. I thought it was weird, so I decided to look into. Turns out, that without using Razor, you can actually build Web APIs with VB.NET just fine if you just convert a C# template into VB.NET, the process was...err....tricky, and not something we would want to do for other projects, with that in we decided to look into Microsoft's reasons to not have VB.NET. We discovered how Microsoft is not keeping the same language features between both languages, having crown C# as the language of choice for everything Microsoft, to this point, it seems that Microsoft was much more focused in developing features for the excellent F# way more than it ever had for VB.NET at this point and that it was not a major strategy for them to adapt most of the .net core functionality inside of VB, we found articles when the very same Microsoft team stated of how they will be slowly adding the required support for VB and that on version 5 we would definitely have proper support for VB.NET ALTHOUGH they will not be adding any new development into the language.
Past experience with Microsoft seems to point at them getting more and more ready to completely drop the language, it does not matter how many people use it, they would still kill it :P I personally would rather keep it, or open source the language's features so that people can keep adding support to it(if they can of course) because of its historical significance rather than them just completely dropping the language. I prefer using C#, and most of my .net core applications use C#, its very similar to Java on a lot of things(although very much different in others) and I am fine with it being the main language. I just think that it sucks to leave such a large developer pool in the shadows with their preferred tool of choice and force them to use something else just like that.
My boy is currently looking at how I developed a sample api with validation, user management, mediatR and a custom project structure as well as a client side application using React and typescript swappable with another one built using Angular(i wanted to test the differences to see which one I prefer, React with Typescript is beautiful, would not want to use it without it) and he is hating every minute of it on account of how complex frontend development has become :V
Just wanted to vent a little about a non bothersome situation.6 -
I am currently doing my first ever internship. It is a medium-ish sized IT company and I basically do stuff in networking and software development. I sit on my chair, wheel it back and forth the small space behind my desk for like and hour. Then I go to the cafeteria, eat whatever is there (it’s absolutely free? Hopefully). There is a pool table which is always occupied. Then I sit in the lab and configure routers till 8 in the night. Boring.
I am developing a management system side by side so I break my head over server side routes (seriously, they are confusing) over the night.
So coffee is my mantra and boring is my life.2 -
I am building a website inspired by devrant but have never built a server network before, and as im still a student I have no industry experience to base a design on, so was hoping for any advice on what is important/ what I have fucked up in my plan.
The attached image is my currently planned design. Blue is for the main site, and is a cluster of app servers to handle any incoming requests.
Green is a subdomain to handle images, as I figured it would help with performance to have image uploads/downloads separated from the main webpage content. It also means I can keep cache servers and app servers separated.
Pink is internal stuff for logging and backups and probably some monitoring stuff too.
Purple is databases. One is dedicated for images, that way I can easily back them up or load them to a cache server, and the other is for normal user data and posts etc.
The brown proxy in the middle is sorta an internal proxy which the servers need to authenticate with to connect to, that way I can just open the database to the internal proxy, and deny all other requests, and then I can have as many app servers as I want and as long as they authenticate with the proxy, they can access the database without me changing any firewall rules. The other 2 proxies just distribute requests between the available servers in the pool.
Any advice would be greatly appreciated! Thanks in advanced :D13 -
How the Common Lisp Community will eventually die soon:
Clojure is the only main Lisp dialect having some sort of heavy presence in today's modern development world. Yes, I am aware of other(if not all) environments in which Lisp or a dialect of it is being used for multiple things, CADLisp, Guile Scheme, Racket, etc etc whatever. I know.
Not only is Clojure present in the JVM(I give 0 fucks about whether you like it or not also) but also has compilation targets for Javascript via Clojurescript. This means that i can effectively target backend server operations, damn near everything inside of the JVM and also the browser.
Yet, there is no real point in using Lisp or Clojure other than for pure academic endeavours, for which it is not even a pure functional programming language, you would be better served learning something else if you want true functional purity. But also because examples for one of the major areas in software development, mainly web, are really lacking, like, lacking bad, as in, so bad most examples are few in between and there is no interest in making it target complete beginners or anything of the like.
But my biggest fucking gripe with Lisp as a whole, specifically Common Lisp, is how monstrously outdated the documentation you can find available for it is.
Say for example, aesthetics, these play a large role, a developer(web mostly) used to the attention to detail placed by the Rails community, the Laravel community, django, etc etc would find on documentation that came straight from the 90s. There is no passion for design, no attention to detail, it makes it look hacky and abandoned. Everything in Lisp looks so severely abandoned for which the most abundant pool of resources are not even made present on a fully general purpose language constrained as a scripting environment for a text editor: Emacs with Emacs Lisp which I reckon is about the most used Lisp dialect in the planet, even more so than Clojure or Common Lisp.
I just want the language to be made popular again y'know? To have a killer app or framework for it much like there is Rails for Ruby, Phoenix for Elixir, etc etc. But unless I get some serious hacking done to bring about the level of maturity of those frameworks(which I won't nor I believe I can) then it will always remain a niche language with funny syntax.
To be honest I am phasing away my use of Clojure in place of Pharo. I just hate seeing how much the Lisp community does in an effort to keep shit as obscure and far away from the reach of new developers as possible. I also DESPISE reading other Lisp developer's code. Far too fucking dense and clever for anyone other than the original developer to read and add to. The idea that Lisp allows for read only code is far too real man.
Lisp has been DED for a while, and the zombies that remain will soon disappear because the community was too busy playing circle jerks for anything real to be done with it. Even as the original language of AI it has been severely outshined by the likes of Python, R and Scala, shit, even Javascript has more presence in AI than Lisp does now a days.9 -
Damn it XenCenter! Why won't you reboot the VMs like I tell you!? Force reboot also didn't work. Now I have to shut it down and restore a back up VM. This has been going on for the past 2 weeks!!
-
Update:
I've been trying to leave DoD for a couple of months now. Translating my 10 year's experience with complex Intelligence enterprise level systems to something relatable to the civilian IT world. Grabbed a few certs to help out A+, network+ and security+ with Linux+ as my next target. Photos of me working on unclassified systems, radios, cell towers and servers. I'm a teacher for military UAS so this shouldn't be to hard to get even a basic job in IT right.
No one will hire...
Linux admin: Nope
Network admin: Nope
Assistant Network admin: Nope
IT call service: Nope
Pool cleaner fucking nope
Many interviews and nothing
I'm broke and sold all of my personal valuables. I can't hold out much longer and really looking at becoming homeless. But I'm kinda ok with it, one last payment on my apartment and car is all I can do now. My parents think I'm in Afghanistan working a six figure job lol
DoD: we see you're trying to leave we'll pay you alot to teach A+, Network+ and Security+ traveling all across the country and staying at hotels with all expenses paid.
FU FU FU I want out please tell me someone has a job, I'll be a janitor of a server room Idc I just want out. Fuck the pay
I start Tuesday...4 -
Hello, I have a question for anyone familiar with multithreading!
I just started working with threading for the first time, I mostly write powershell scripts 😅, I found that certain conditions make using multithreading an absolute time saver. And of course in some tasks it's not such a big deal.
I am currently working on a project that runs multiple threads and each thread might invoke one of my functions that also threads the work.
I'm a total newbhat when it comes to this stuff, but if my main process is 4 threads, and I can spin up, up-to 4 more threads to run one of my functions, does the math equate to a possible total number of threads of 16 or is it possible to have the threading go ape-shit bananas and utterly thrash the cpu with rampant threads getting created?
I've looked online and based on some of the info that I've managed to come across on my own, the answers elude towards being safe because I'm creating pools for running the threads first and the pool is responsible for maintaining min/max threads, but I can't seem to find good info on running a pool+threads inside another thread.
Just to let you in on what the function does that requires threading in the first place, I need to basically query CloudTrail based on ARN's to find events, but I can only pass a single ARN to the find-ctevent cmdlet. So I'm essentially making 1500-ish really really small calls to AWS just to get back event data for the ARN.
Serially, this takes like almost 20 mins, on my laptop using stupid settings like 24 threads, it completes in about 95seconds. On the actual server that will be running this code, I'm going to limit it to 4 threads and try to figure out a way to cache the info locally and update the info on a cron or schedule so only the initial scrape takes forever and then the updates can be done nightly or something.
thank you in advance for your help, I'm not too sure if the question is dumb but please let me know either way!8