Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "server terminated"
-
Last september:
I get an email from a customer who want to end the contract. I inform him that the contract runs to the end of March and it will be terminated the last of march.
Customer agrees and I do not hear a word from him. Until now when I closed down all his services.
Angry email:
Why did you close the server and email? are you **** stupid!?
I reply:
21/9 2016, you wanted to terminate all services you had with us. That means that your domains, DNS-services, VPS, email and Office 365 was going to be closed today.
Why is this not uncommon?!5 -
I've had many, but this is one of my favorite "OK, I'm getting fired for this" moments.
A new team in charge of source control and development standards came up with a 20 page work-instruction document for the new TFS source control structure.
The source control kingpin came from semi-large military contract company where taking a piss was probably outlined somewhere.
Maybe twice, I merged down from a release branch when I should have merged down from a dev branch, which "messed up" the flow of code that one team was working on.
Each time I was 'coached' and reminded on page 13, paragraph 5, sub-section C ... "When merging down from release, you must verify no other teams are working
on branches...blah blah blah..and if they have pending changes, use a shelfset and document the changes using Document A234-B..."
A fellow dev overheard the kingpin and the department manager in the breakroom saying if I messed up TFS one more time, I was gone.
Wasn't two days later I needed to merge up some new files to Main, and 'something' happened in TFS and a couple of files didn't get merged up. No errors, nothing.
Another team was waiting on me, so I simply added the files directly into Main. Unknown to me, the kingpin had a specific alert in TFS to notify him when someone added
files directly into Main, and I get a visit.
KP: "Did you add a couple of files directly into Main?"
Me:"Yes, I don't what happened, but the files never made it from my branch, to dev, to the review shelfset, and then to Main. I never got an error, but since
they were new files and adding a new feature, they never broke a build. Adding the files directly allowed the Web team to finish their project and deploy the
site this morning."
KP: "That is in direct violation of the standard. Didn't you read the documentation?"
Me: "Uh...well...um..yes, but that is an oddly specific case. I didn't think I hurt any.."
KP: "Ha ha...hurt? That's why we have standards. The document clearly states on page 18, paragraph 9, no files may ever be created in Main."
Me: "Really? I don't remember reading that."
<I navigate to the document, page 18, paragraph 9>
Me: "Um...no, it doesn't say that. The document only talks about merging process from a lower branch to Main."
KP: "Exactly. It is forbidden to create files directly in Main."
Me: "No, doesn't say that anywhere."
KP: "That is the spirit of the document. You violated the spirit of what we're trying to accomplish here."
Me: "You gotta be fracking kidding me."
KP grumbles something, goes back to his desk. Maybe a minute later he leaves the IS office, and the department manager leaves his office.
It was after 5:00PM, they never came back, so I headed home worried if I had a job in the morning.
I decided to come in a little early to snoop around, I knew where HR kept their terminated employee documents, and my badge wouldn't let me in the building.
Oh crap.
It was a shift change, so was able to walk in with the warehouse workers in another part of the building (many knew me, so nothing seemed that odd), and to my desk.
I tried to log into my computer...account locked. Oh crap..this was it. I'm done. I fill my computer backpack with as much personal items as I could, and started down the hallway when I meet one of our FS accountants.
L: "Hey, did your card let you in the building this morning? Mine didn't work. I had to walk around to the warehouse entrance and my computer account is locked. None of us can get into the system."
*whew!* is an understatement. Found out later the user account server crashed, which locked out everybody.
Never found out what kingpin and the dev manager left to talk about, but I at least still had a job.13 -
This is going to be a long rant, coz this is the only way to vent out my frustration against our tech head.
Yesterday, while our fucking twat tech head was playing around in company aws account, he terminated the production server. By mistake, apparently. Coz he doesn't know shit about server management. But that egoist ass won't admit and fucked the production server.
And then ran away. We developers sprang into action. Updated dns to point to staging server, setup virtual hosts, env files, point to prod database, force flush dns cache. All systems were up and running in 30 mins. And since it was staging server, it had lot of untested features and codes, and we spent rest of the day fixing the bugs.
And that tech head, who ran away hiding his tail between his legs, after he fucked the server, came back after systems were up. And started cracking jokes, that "so many features got released in 1 day" . "We cut server cost by shutting down 1 server."
We were struggling and working in full throttle to make the services running again. And that fuckity fucker was cracking jokes.
And I don't even know what excuse he gave to ceo for the downtime. I am pretty sure he would have made up some crappy excuse to hide his fucking mistake. That ass never admits his mistake. I am thinking to go to ceo today and tell the real story and get that faggot head fired or at least a strict warning.4 -
A few days ago Aruba Cloud terminated my VPS's without notice (shortly after my previous rant about email spam). The reason behind it is rather mundane - while slightly tipsy I wanted to send some traffic back to those Chinese smtp-shop assholes.
Around half an hour later I found that e1.nixmagic.com had lost its network link. I logged into the admin panel at Aruba and connected to the recovery console. In the kernel log there was a mention of the main network link being unresponsive. Apparently Aruba Cloud's automated systems had cut it off.
Shortly afterwards I got an email about the suspension, requested that I get back to them within 72 hours.. despite the email being from a noreply address. Big brain right there.
Now one server wasn't yet a reason to consider this a major outage. I did have 3 edge nodes, all of which had equal duties and importance in the network. However an hour later I found that Aruba had also shut down the other 2 instances, despite those doing nothing wrong. Another hour later I found my account limited, unable to login to the admin panel. Oh and did I mention that for anything in that admin panel, you have to login to the customer area first? And that the account ID used to login there is more secure than the password? Yeah their password security is that good. Normally my passwords would be 64 random characters.. not there.
So with all my servers now gone, I immediately considered it an emergency. Aruba's employees had already left the office, and wouldn't get back to me until the next day (on-call be damned I guess?). So I had to immediately pull an all-nighter and deploy new servers elsewhere and move my DNS records to those ASAP. For that I chose Hetzner.
Now at Hetzner I was actually very pleasantly surprised at just how clean the interface was, how it puts the project front and center in everything, and just tells you "this is what this is and what it does", nothing else. Despite being a sysadmin myself, I find the hosting part of it insignificant. The project - the application that is to be hosted - that's what's important. Administration of a datacenter on the other hand is background stuff. Aruba's interface is very cluttered, on Hetzner it's super clean. Night and day difference.
Oh and the specs are better for the same price, the password security is actually decent, and the servers are already up despite me not having paid for anything yet. That's incredible if you ask me.. they actually trust a new customer to pay the bills afterwards. How about you Aruba Cloud? Oh yeah.. too much to ask for right. Even the network isn't something you can trust a long-time customer of yours with.
So everything has been set up again now, and there are some things I would like to stress about hosting providers.
You don't own the hardware. While you do have root access, you don't have hardware access at all. Remember that therefore you can't store anything on it that you can't afford to lose, have stolen, or otherwise compromised. This is something I kept in mind when I made my servers. The edge nodes do nothing but reverse proxying the services from my LXC containers at home. Therefore the edge nodes could go down, while the worker nodes still kept running. All that was necessary was a new set of reverse proxies. On the other hand, if e.g. my Gitea server were to be hosted directly on those VPS's, losing that would've been devastating. All my configs, projects, mirrors and shit are hosted there.
Also remember that your hosting provider can terminate you at any time, for any reason. Server redundancy is not enough. If you can afford multiple redundant servers, get them at different hosting providers. I've looked at Aruba Cloud's Terms of Use and this is indeed something they were legally allowed to do. Any reason, any time, no notice. They covered all their bases. Make sure you do too, and hope that you'll never need it.
Oh, right - this is a rant - Aruba Cloud you are a bunch of assholes. Kindly take a 1Gbps DDoS attack up your ass in exchange for that termination without notice, will you?5 -
Man I fucking love debugging Windows applications... OpenVPN dun shit the bed because the management interface is locked (on the Windows client I presume?) - so poke that error message into the Gargler along with "openvpn windows"... First result, OpenVPN forums. Excellent. ... Some dickhead in the forums: "this is the wrong forum, this is for Access-Server users, and you the user MUST have terminated the process".
Come fucking on! If only I could replace this fucking device with a proper OS already (and no I can't). Windows itself being a clusterfuck is one thing but the goddamn support around it. Atrocious!4 -
This happened to me sometime back.
I want to try out a WordPress plugin in my local machine before installing on a production server. It is an Ubuntu machine. Downloaded and installed Xampp, then setup WordPress with MySQL. Now tried uploading the plugin zip file, it throws some permission error, asking to fix permissions or use FTP. I thought of just chmod 777 recursively for the WordPress directory to fix this easily.
Ran the command, looks like it is hung. Terminated using Ctrl+C and then ran the same command. Again it is taking much time. It should not take so much time to recursively change the permission of just a WordPress directory. Thought something was wrong. Before I realized the damage is already done.
Looks like I ran the command
sudo chmod -R 777 /
instead of
sudo chmod -R 777 ./
Fuck, I missed a dot in the command and it is changing permissions of everything in my machine. Saw the System monitor, CPU usage spiked to 100%. I can't close or open any program. Force shutdown the machine using the power key. It didn't boot again. Recovery mode didn't help. Looks like there is no easy way to restore back from this damage. Most of the files I need are backed up in the cloud, still, need a few more personal files so that I can format and reinstall Ubuntu. Realised I have Windows in dual booting. Boot into Windows and used some ext4 reader to recover the files, formatted and reinstalled the OS. Took a few hours to get back to my previous setup.
Lesson Learned: Don't use sudo unnecessarily.
Double check the command while executing.
Running a wrong command with root permission can fuckup your entire machine.