Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
So I got the job. Here's a story, never let anyone stop you from accomplishing your dreams!
It all started in 2010. Windows just crashed unrecoverably for the 3rd time in two years. Back then I wasn't good with computers yet so we got our tech guy to look at it and he said: "either pay for a windows license again (we nearly spend 1K on licenses already) or try another operating system which is free: Ubuntu. If you don't like it anyways, we can always switch back to Windows!"
Oh well, fair enough, not much to lose, right! So we went with Ubuntu. Within about 2 hours I could find everything. From the software installer to OpenOffice, browsers, email things and so on. Also I already got the basics of the Linux terminal (bash in this case) like ls, cd, mkdir and a few more.
My parents found it very easy to work with as well so we decided to stick with it.
I already started to experiment with some html/css code because the thought of being able to write my own websites was awesome! Within about a week or so I figured out a simple html site.
Then I started to experiment more and more.
After about a year of trial and error (repeat about 1000+ times) I finally got my first Apache server setup on a VirtualBox running Ubuntu server. Damn, it felt awesome to see my own shit working!
From that moment on I continued to try everything I could with Linux because I found the principle that I basically could do everything I wanted (possible with software solutions) without any limitations (like with Windows/Mac) very fucking awesome. I owned the fucking system.
Then, after some years, I got my first shared hosting plan! It was awesome to see my own (with subdomain) website online, functioning very well!
I started to learn stuff like FTP, SSH and so on.
Went on with trial and error for a while and then the thought occured to me: what if I'd have a little server ONLINE which I could use myself to experiment around?
First rented VPS was there! Couldn't get enough of it and kept experimenting with server thingies, linux in general aaand so on.
Started learning about rsa key based login, firewalls (iptables), brute force prevention (fail2ban), vhosts (apache2 still), SSL (damn this was an interesting one, how the fuck do you do this yourself?!), PHP and many other things.
Then, after a while, the thought came to mind: what if I'd have a dedicated server!?!?!?!
I ordered my first fucking dedicated server. Damn, this was awesome! Already knew some stuff about defending myself from brute force bots and so on so it went pretty well.
Finally made the jump to NginX and CentOS!
Made multiple VPS's for shitloads of purposes and just to learn. Started working with reverse proxies (nginx), proxy servers, SSL for everything (because fuck basic http WITHOUT SSL), vhosts and so on.
Started with simple, one screen linux setup with ubuntu 10.04.
Running a five monitor setup now with many distro's, running about 20 servers with proxies/nginx/apache2/multiple db engines, as much security as I can integrate and this fucking passion just got me my first Linux job!
It's not just an operating system for me, it's a way of life. And with that I don't just mean the operating system, but also the idea behind it :).20 -
New Guy Day 2: He has deleted the git repo on the project he was assigned to 4 times, written a recursion formula that crashed a server, & knocked my coffee cup onto the floor.
I messaged my boss telling him I am going to hide the body in his trunk.8 -
Navy story time, and this one is lengthy.
As a Lieutenant Jr. I served for a year on a large (>100m) ship, with the duties of assistant navigation officer, and of course, unofficial computer guy. When I first entered the ship (carrying my trusty laptop), I had to wait for 2 hours at the officer's wardroom... where I noticed an ethernet plug. After 15 minutes of waiting, I got bored. Like, really bored. What on TCP/IP could possibly go wrong?
So, scanning the network it is. Besides the usual security holes I came to expect in ""military secure networks"" (Windows XP SP2 unpatched and Windows 2003 Servers, also unpatched) I came along a variety of interesting computers with interesting things... that I cannot name. The aggressive scan also crashed the SMB service on the server causing no end of cute reactions, until I restarted it remotely.
But me and my big mouth... I actually talked about it with the ship's CO and the electronics officer, and promptly got the unofficial duty of computer guy, aka helldesk, technical support and I-try-to-explain-you-that-it-is-impossible-given-my-resources guy. I seriously think that this was their punishment for me messing around. At one time I received a call, that a certain PC was disconnected. I repeatedly told them to look if the ethernet cable was on. "Yes, of course it's on, I am not an idiot." (yea, right)
So I went to that room, 4 decks down and 3 sections aft. Just to push in the half-popped out ethernet jack. I would swear it was on purpose, but reality showed me I was wrong, oh so dead wrong.
For the full year of my commission, I kept pestering the CO to assign me with an assistant to teach them, and to give approval for some serious upgrades, patching and documenting. No good.
I set up some little things to get them interested, like some NMEA relays and installed navigation software on certain computers, re-enabled the server's webmail and patched the server itself, tried to clean the malware (aka. Sisyphus' rock), and tried to enforce a security policy. I also tried to convince the CO to install a document management system, to his utter horror and refusal (he was the hard copy type, as were most officers in the ship). I gave up on almost all besides the assistant thing, because I knew that once I left, everything would go to the high-entropy status of carrying papers around, but the CO kept telling me that would be unnecessary.
"You'll always be our man, you'll fix it (sic)".
What could go wrong?
I got my transfer with 1 week's notice. Panic struck. The CO was... well, he was less shocked than I expected, but still shocked (I learned later that he knew beforehand, but decided not to tell anybody anything). So came the most rediculous request of all:
To put down, within 1 A4 sheet, and in simple instructions, the things one had to do in order to fulfil the duties of the computer guy.
I. SHIT. YOU. NOT.
My answer:
"What I can do is write: 'Please read the following:', followed by the list of books one must read in order to get some introductory understanding of network and server management, with most accompanying skills."
I was so glad I got out of that hellhole.6 -
A client’s server crashed so they called us. When we checked the logs we found a user was logged in 200,000+ times. We told them the user and a few minutes later we get a picture of a cupcake that was on the enter key. They got cupcake DOS attacked. My team isn’t in charge of the login functionality but I can say the team that is got a fairly aggressive internal support ticket.5
-
passionately making love on Friday morning.
Cellphone buzzes for WhatsApp notification, ignored.
Call ignored, SMS ignored, again call, ignored.
She : whose that bitch calling you?
Me : it's no one important.
She : Checks WhatsApp, message from CTO, server crashed.
Me : Umm, darling, I need to fix this.
It's been 2 days she hasn't answered my call after she left.15 -
Worst thing you've seen another dev do? Long one, but has a happy ending.
Classic 'Dev deploys to production at 5:00PM on a Friday, and goes home.' story.
The web department was managed under the the Marketing department, so they were not required to adhere to any type of coding standards and for months we fought with them on logging. Pre-Splunk, we rolled our own logging/alerting solution and they hated being the #1 reason for phone calls/texts/emails every night.
Wanting to "get it done", 'Tony' decided to bypass the default logging and send himself an email if an exception occurred in his code.
At 5:00PM on a Friday, deploys, goes home.
Around 11:00AM on Sunday (a lot folks are still in church at this time), the VP of IS gets a call from the CEO (who does not go to church) about unable to log into his email. VP has to leave church..drive home and find out he cannot remote access the exchange server. He starts making other phone calls..forcing the entire networking department to drive in and get email back up (you can imagine not a group of happy people)
After some network-admin voodoo, by 12:00, they discover/fix the issue (know it was Tony's email that was the problem)
We find out Monday that not only did Tony deploy at 5:00 on a Friday, the deployment wasn't approved, had features no one asked for, wasn't checked into version control, and the exception during checkout cost the company over $50,000 in lost sales.
Was Tony fired? Noooo. The web is our cash cow and Tony was considered a top web developer (and he knew that), Tony decided to blame logging. While in the discovery meeting, Tony told the bosses that it wasn't his fault logging was so buggy and caused so many phone calls/texts/emails every night, if he had been trained properly, this problem could have been avoided.
Well, since I was responsible for logging, I was next in the hot seat.
For almost 30 minutes I listened to every terrible thing I had done to Tony ever since he started. I was a terrible mentor, I was mean, I was degrading, etc..etc.
Me: "Where is this coming from? I barely know Tony. We're not even in the same building. I met him once when he started, maybe saw him a couple of times in meetings."
Andrew: "Aren't you responsible for this logging fiasco?"
Me: "Good Lord no, why am I here?"
Andrew: "I'll rephrase so you'll understand, aren't you are responsible for the proper training of how developers log errors in their code? This disaster is clearly a consequence of your failure. What do you have to say for yourself?"
Me: "Nothing. Developers are responsible for their own choices. Tony made the choice to bypass our logging and send errors to himself, causing Exchange to lockup and losing sales."
Andrew: "A choice he made because he was not properly informed of the consequences? Again, that is a failure in the proper use of logging, and why you are here."
Me: "I'm done with this. Does John know I'm in here? How about you get John and you talk to him like that."
'John' was the department head at the time.
Andrew:"John, have you spoken to Tony?"
John: "Yes, and I'm very sorry and very disappointed. This won't happen again."
Me: "Um...What?"
John: "You know what. Did you even fucking talk to Tony? You just sit in your ivory tower and think your actions don't matter?"
Me: "Whoa!! What are you talking about!? My responsibility for logging stops with the work instructions. After that if Tony decides to do something else, that is on him."
John: "That is not how Tony tells it. He said he's been struggling with your logging system everyday since he's started and you've done nothing to help. This behavior ends today. We're a fucking team. Get off your damn high horse and help the little guy every once in a while."
Me: "I don't know what Tony has been telling you, but I barely know the guy. If he has been having trouble with the one line of code to log, this is the first I've heard of it."
John: "Like I said, this ends today. You are going to come up with a proper training class and learn to get out and talk to other people."
Over the next couple of weeks I become a powerpoint wizard and 'train' anyone/everyone on the proper use of logging. The one line of code to log. One line of code.
A friend 'Scott' sits close to Tony (I mean I do get out and know people) told me that Tony poured out the crocodile tears. Like cried and cried, apologizing, calling me everything but a kitchen sink,...etc. It was so bad, his manager 'Sally' was crying, her boss 'Andrew', was red in the face, when 'John' heard 'Sally' was crying, you can imagine the high levels of alpha-male 'gotta look like I'm protecting the females' hormones flowing.
Took almost another year, Tony released a change on a Friday, went home, web site crashed (losses were in the thousands of $ per minute this time), and Tony was not let back into the building on Monday (one of the best days of my life).10 -
What an awful day :(
The server where I host my 4 clients websites crashed.
Unable to reboot from the console.
I contact the support. 15 minutes later: "we'll look at this"
No news for 1 week despite my messages.
Then... 1st ticket escalation... 2nd ticket escalation... 3rd ticket escalation...
Answer: "Sorry, your server is down and cannot be repaired."
Fuck.
I ask "is there any way to get my data back?". Answer: "No, because we would shutdown the whole bay and all our clients would be impacted".
Fuck.
I subscribe to another server, at another provider.
I look at my backups... shit, the last one is 4 month ago!!
I restore the first website: OK
I restore the second website: OK
I restore the third website: My new server is "too recent" and not compatible. with this old Wordpress. Fuck! I'll look at this later...
I restore the fourth website: database is empty!! What??? I look at the SQL backup for this site... it failed...
I lost ALL my 4th client data!!!
I'm sooooo piece of crap!14 -
I've had many, but this is one of my favorite "OK, I'm getting fired for this" moments.
A new team in charge of source control and development standards came up with a 20 page work-instruction document for the new TFS source control structure.
The source control kingpin came from semi-large military contract company where taking a piss was probably outlined somewhere.
Maybe twice, I merged down from a release branch when I should have merged down from a dev branch, which "messed up" the flow of code that one team was working on.
Each time I was 'coached' and reminded on page 13, paragraph 5, sub-section C ... "When merging down from release, you must verify no other teams are working
on branches...blah blah blah..and if they have pending changes, use a shelfset and document the changes using Document A234-B..."
A fellow dev overheard the kingpin and the department manager in the breakroom saying if I messed up TFS one more time, I was gone.
Wasn't two days later I needed to merge up some new files to Main, and 'something' happened in TFS and a couple of files didn't get merged up. No errors, nothing.
Another team was waiting on me, so I simply added the files directly into Main. Unknown to me, the kingpin had a specific alert in TFS to notify him when someone added
files directly into Main, and I get a visit.
KP: "Did you add a couple of files directly into Main?"
Me:"Yes, I don't what happened, but the files never made it from my branch, to dev, to the review shelfset, and then to Main. I never got an error, but since
they were new files and adding a new feature, they never broke a build. Adding the files directly allowed the Web team to finish their project and deploy the
site this morning."
KP: "That is in direct violation of the standard. Didn't you read the documentation?"
Me: "Uh...well...um..yes, but that is an oddly specific case. I didn't think I hurt any.."
KP: "Ha ha...hurt? That's why we have standards. The document clearly states on page 18, paragraph 9, no files may ever be created in Main."
Me: "Really? I don't remember reading that."
<I navigate to the document, page 18, paragraph 9>
Me: "Um...no, it doesn't say that. The document only talks about merging process from a lower branch to Main."
KP: "Exactly. It is forbidden to create files directly in Main."
Me: "No, doesn't say that anywhere."
KP: "That is the spirit of the document. You violated the spirit of what we're trying to accomplish here."
Me: "You gotta be fracking kidding me."
KP grumbles something, goes back to his desk. Maybe a minute later he leaves the IS office, and the department manager leaves his office.
It was after 5:00PM, they never came back, so I headed home worried if I had a job in the morning.
I decided to come in a little early to snoop around, I knew where HR kept their terminated employee documents, and my badge wouldn't let me in the building.
Oh crap.
It was a shift change, so was able to walk in with the warehouse workers in another part of the building (many knew me, so nothing seemed that odd), and to my desk.
I tried to log into my computer...account locked. Oh crap..this was it. I'm done. I fill my computer backpack with as much personal items as I could, and started down the hallway when I meet one of our FS accountants.
L: "Hey, did your card let you in the building this morning? Mine didn't work. I had to walk around to the warehouse entrance and my computer account is locked. None of us can get into the system."
*whew!* is an understatement. Found out later the user account server crashed, which locked out everybody.
Never found out what kingpin and the dev manager left to talk about, but I at least still had a job.13 -
When I was 10 years old, all kids at my school got access to school emails. The email address book contained everything in my city, like fire department, all pupils of all schools etc. So I decided to "test" the system by sending out a mass email to everyone in the address book (about 3k) with the question "Hi, how are you?".
The sys admins apparently didn't think very far as I got some responses saying like "You have crashed a server in the capital city" and "I have contacted your local IT admin".
So I went to the IT admin and told him the situation. His face turned red of anger and I remember him almost screaming at me.
Who the fuck doesn't set up protection for this and gives out access to 10 year olds? This was 15 years ago, I really hope sys admins are smarter today!6 -
Today was bad day.
- only had 3 hours of sleep
- 1.5h exam in the morning
- work in the afternoon until 8pm
- 1 drive crashed in a RAID5 array
- wasted hours of data copying
- my hands and arms got really dirty from all that nasty trump-face-colored dust in the server room
- nothing new in the west
- I have to get up in 4 hours again to start a new copying task
- I only knew it was friday today because the devRant meme game was reaching the weekly peak
- lists can save lives
- good night 😴2 -
So...
I'm penetrationtesting a network and the servers on said network
The network administrator and IT security officer knows this, because they hired me..
TL;DR a scan caused the network to crash.
Today I received a very angry email going "Stop scanning NOW!" from one of the IT departments.
Apparently I crashed their login server and thus their entire network...
It happened d the first time I scanned the network from the outside and they had spend an entire day figuring out how and repairing the service they thought was the problem, but then it crashed again, when I scanned from within the network.
Now they want to send me a list of IP's that I'm not allowed to scan and want to know exactly what and when I'm scanning...
How crap can they be at their job, if they weren't able to spot a scan... The only reason they found out it was me was because the NA had whitelistet my IP, so that I could scan in peace...5 -
"Almond, I thought you said the cause of the outage the other week was that our server crashed?"
"The Tomcat server crashed, yeah. Not the physical server." (And you won't give me the time or budget to spin up any kind of redundant one, but that's besides the point...)
"Ok, but I've spoken to ops and they say none of the servers have gone offline in the last month?"
"Yup, the physical server was fine, it was the Tomcat server running on it that crashed."
"...so the server didn't crash?"
"We're mixing terms here. There's two things that can be referred to as the server. One is the physical machine, and one is an application running on it. The physical machine was fine, but
the application running on it crashed."
"What?! It's a very simple question. Did the server crash, or didn't it?!"
🤦♂️13 -
When pandemic hit in 2020 I found myself out of work. Until then I used to have a java based pirate gameserver of a MMORPG as a hobby.
When pandemic hit I noticed that online players count increased from like 70 to 200 without much advertising because purely of people being stuck in home. So i decided to scale and spent 2 years with that. What a wild ride it was.
So i invested a bit in ads, managed to reach around 500 online players, opened my own company and launched a couple other successful spinoffs of that gameserver.
First year it was a goldmine but I was doing 10-14 hour days because I had to take care of everything (web, advertising, payment integrations, player support and also developing the server itself, ddos protections and etc.). I made quite a bit of money, saved for a downpayment for mortgage and got an apartment.
Second year I noticed that there was a lot of competition and online players count dropped, but I double downed on this and invested a lot into the product itself and spent most of the time developing a perfect gameserver that would be the big bang while also maintaining existing ones. Clasic overengineering mistake. As you can guess, I crashed and burned on all levels, never even managed to launch my final project because simply the scope was too big and I had trouble finding decent devs to outsource it to, since it was a very niche gameserver.
In the end I learned a lot especially about my own limits and ownership, now Im back to being a dev but working as a contractor.
I believe having actual business owner experience allows me to have different perspective and I can bring more to the table rather than focusing on crunching tasks.6 -
Sometimes I think back to all the funny shit that happened and how simple stuff fucks everyone
- tired Database engineer deleting (not dropping, literally rm -rf) the database files on the wrong server
- Microsoft delivering viruses through updates
- Pissed and stubborn dev deleting his one line library repo which does something like removing a char left side of string fucking an unmeasurable amount of other projects
- Adobe getting hacked and exposed for storing passwords in plain texts
- a doubled line causing a bug called heartbleed in a fuckton of webservers
- a Tutorial Company getting kicked from github because their repo got so big github staff had to maintain the repo manually
- and an old one: bad code crashed a space shuttle16 -
So we hired an intern and his first task was to change a few things in email layout for our client, which is an investment bank.
I told to one of my developers to make his local database dump and setup the project for an intern. When intern completed the task, my developer thought that title "Dow Jones index crashed" was pretty funny title for a test.
What he didn't thought through enough, is that he forgot to configure fake SMTP server and he had production database dump with real email addresses.
I had really awkward 20 minutes conversation with our client. Fuck my life.4 -
We had a server that crashed every single Christmas four years a row. Last year I put a cross and garlic on it for fun. Everybody laughed at me... but it didn't crash!
Soo... should I protect it from the Christmas ghost again, this year too? 🤔👻1 -
So today I accidentally wrote a non-termating for loop that sent POST requests en masse to our server and likely crashed it while I was peer programming with my team lead, how's y'all's day goin?4
-
Gather around folks, I'll paint you a nice picture based on a true story, back from my sysadmin days. Listen up.
It's about HP and their Solaris 5.4/6 support.
- Yet another Prod Solaris dinosaur crashed
- Connected to console, found a dead system disk; for some reason it was not booting on the remaining redundant disk...
- Logged an HP vendor case. Sev1. SLA for response is 30 minutes, SLA for a fix is <24 hours
- It took them 2 days to respond to our Prod server outage due to failed system disks (responses "we are looking into this" do not count)
- it took another day for them to find an engineer who could attend the server in the DC
- The field engineer came to the DC 4 hours before the agreed time, so he had to wait (DC was 4-5 hours of driving away from HP centre)
- Turns out, he came to the wrong datacentre and was not let in even when the time came
- We had to reschedule for two days later. Prod is still down
- The engg came to the DC on time. He confirmed he had the FRU on him. Looks promising
- He entered the Hall
- He replaced the disk on the Solaris server
- It was the wrong disk he replaced. So now the server is beyond rebuild. It has to be built anew... but only after he comes back and replaces the actually faulty disk.
- He replaced that disk on the wrong Solaris server2 -
That time I thought I crashed a prod server which I had no access to physically turn it back on. Turned out it was doing its weekly reboot. Scary 3 minutes.
-
Today I sent email blast to wrong list of 12k recipients. I was given the list by another employee. I crashed the email server. It was fun.2
-
I call this one the tester than knew too much.
Note: The server the tester is running on has a hard drive that is breaking down...
Tester: Remember the error I talked to you about yesterday?
Me: Yeah, what about it?
Tester: Well the server hasn't recovered yet and I haven't restarted anything...
Me: Well the application itself hasn't crashed so our monitoring application doesn't seem to notice that the service is in a bad state. The error seems only to have brought down certain threads within the application.
Tester: No, I think there is a different issue here and has nothing to do with that error, the application is still doing things.
*tails the log*
Tester: See?
Me: As I said some things are still running and are unaffected by the error.
Tester: NO! It has to be caused by the other error I had a week ago where my file got corrupted. As we said I removed the file, restarted it and it worked again, but had the same problem a day later...
Note: The problem is not related, this time the application is running out of file descriptors
Me: Well... If the problem is the same it would have complained about the file descriptors then aswell, not an I/O error.
Tester: Nope, I think you are wrong!
Me: ¯\_(ツ)_/¯
FML
3 -
Fucking fuck you 'executive' sales director.
You come into my office when I'm not here, and then force people to prioritise your shitty fucking customer above a high priority one that I need to go to. I get phoned and told, "You will go there".
Basically, he just prioritised a shitty fucking issue of 3 users over a high priority SLA customer of 140 users.
Getting shat out as I speak about why the high priority is not being done (I need to do DR there, because their server crashed).
Basically said - I am fucked. I either choose to go to the SLA call and then the 'director' fucking loses his shit and hits me, or I go to the call and the MD loses his shit and fucks me up (again).
Fuck this.
Fuck.
Time to get to work! -
remember when our whole dev team was drunk but we went back to work anyway? turns out, someone rm -rf a whole folder of js scripts!! it took us some time before we could find the back up files which were in the old server (that crashed) because recently we changed servers (so we were transferring files) and we are now using amazon.
our boss got so pissed this morning but oh well, it was his idea to drink and feast in the first place!7 -
Not been a good day so far:
1. Woke up to my Synology in a 'Volume crashed' state. Tried to contact support via web page; support web page not loading.
2. Ancient software at work stops working. As the last remaining C++ dev, I gotta troubleshoot. Original developer wrote test program...in VB6.
3. Server config file changed, but all the admins swear up and down nobody's made any changes.
4. Client calls account rep and wants to know about our security policies, so he schedules a meeting with me and client and forgets to mention until he's emailing me asking where the hell I am. From the tone of the conversation between the rep and the client, it's clear that somehow I'm to blame for being late.
Sigh.
Well, hey, at least it's Friday, right? Right?1 -
Why WordPress is not very good:
I wrote a quick 230 line python script that uses the power of urllib, ebooklib and 12 regular expressions that would make any italian proud to download webnovels from virlyce.com and turn them into .epub files for me.
The chapters are all individual WordPress pages, and after sequentially downloading only 202 of them I got an internal server error.
Why, WordPress?
Of course, I saw this coming and put mitmproxy to good use caching everything, so even though my python script with terrible error handling crashed I don't have to do it all again (yay)
4 -
A server application pulled off some sort of listings as table. Problem was, it crashed with some thousand data files after one and a half hours. I looked into that, and couldn't stop WTFing.
A stupid server side script fetched the data in XML (WTF!) and then inserted shit node-wise (WTF!!), which was O(n^2) - in PHP and on XML! Then it converted the whole shebang into HTML for browser display although users would finally copy/paste the result into Excel anyway.
The original developer even had written a note on the application page that pulling the data "could take long". Yeah because it's so fucking STUPID that Clippy is an Einstein in comparison, that's why!
So I pulled the raw data via batch file without XML wrapping and wrote a little C program for merging the dumped stuff client-side in O(n), spitting out a final CSV for Excel import.
Instead of fucking the server for 1.5 hours and then crashing, shit is done after 7 seconds, out of which the actual data processing takes 40 bloody milliseconds!4 -
I don't know what you did yesterday, but i did make my company throw away 2 months of progress.
It all started in the beginning, since that i've made numerous complaints about the workflow or code and how to improve it. I've been told off every time, and every time i either told the boss who agreed in the end or wrote code to prove myself. Everything was a hassle and my tasks weren't better.
Team lead: you'll do X now, please do that by making Y.
Me: but Y is insecure, we should do Z.
Team lead: please do Y
Later it turns out Y is impossible and we do Z in the end...
Team lead: please do W now
Me, a few days later: i've tried and their server doesn't give http cors headers, doing W in the browser is impossible
Team lead, a few days later: have you made progress on W?
Me: * tells again it's impossible and uploads code to prove it *
Team lead: * no response *
After that i had enough. Technically i still was assigned to do W, but i used my time to look over the application and list all the things wrong with it. We had everything, giant commits, commented out code, unnecessary packages, a new commit introduced packages that crashed npm install on non-macs, angularjs-packages even though we use angular, weird logic, a security bug, all css in one file even though you can use component-specific css files...
I sent that to my boss, telling him to let the backend-guys have a look at it too and we had a meeting about this. I couldn't attend but they agreed with me completely. They decided to throw away what we have already and to let one of the backend-guys supervise our team. I guess there will be another talk with the team lead, but time will tell.
It feels so good having hope to finally escape this hellish development cycle of badly defined task, bad communication and headache-inducing merges. -
A (work-)project i spent a year on will finally be released soon. That's the perfect opportunity to vent out all the rage i built up during dealing with what is the javascript version of a zodiac letter.
Everything went wrong with the beginning. 3 people were assigned to rewrite an old flash-application. Me, A and B. B suggested a javascript framework, even though me and A never worked with more than jquery. In the end we chose react/redux with rest on the server, a classic.
After some time i got the hang of time, around that time B left and a new guy, C, was hired soon after that. He didn't know about react/redux either. The perfect start off to a burning pile of smelly code.
Today this burning pile turned into a wasteland of code quality, a house of cards with a storm approaching, a rocket with leaks ready to launch, you get the idea.
We got 2 dozen files with 200-500 loc, each in the same directory and each with the same 2 word prefix which makes finding the right one a nightmare on its on. We have an i18n-library used only for ~10 textfields, copy-pasted code you never know if it's used or not, fetch-calls with no error-handling, and many other code smells that turn this fire into a garbage fire. An eternal fire. 3 months ago i reduced the linter-warnings on this project to 1, now i can't keep count anymore.
We use the reactabular-module which gives us headaches because IT DOESN'T DO WHAT IT'S SUPPOSED TO DO AND WE CANT USE IT WELL EITHER. All because the client cant be bothered to have the table header scroll along with the body. We have methods which do two things because passing another callback somehow crashed in the browser. And the only thing about indentation is that it exists. Copy pasting from websites, other files and indentation wars give the files the unique look that make you wonder if some of the devs hides his whitespace code in the files.
All of this is the result of missing time, results over quality and the worst approach of all, used by A: if A wants an ui-component similar to an existing one, he copies the original and edits he copy until it does what he wants. A knows about classes, modules, components, etc. Still, he can't bring himself to spend his time on creating superclasses... his approach gives results much faster
Things got worse when A tried redux, luckily A prefers the components local state. WHICH IS ANOTHER PROBLEM. He doesn't understand redux and loads all of the data directly from the server and puts it into the local state. The point of redux is that you don't have to do this. But there are only 1 or 2 examples of how this practice hurt us yet, so i'm gonna have to let this slide. IF HE AT LEAST WOULD UPDATE THE DATA PROPERLY. Changes are just sent to the server and then all of the data is re-fetched. I programmed the rest-endpoints to return the updated objects for a very reason. But no, fuck me.
I've heard A decided (A is the teamleader) to use less redux on the next project and use a dedicated rest-endpoints for every little comoutation you COULD DO WITH REDUX INSTEAD. My will is broken and just don't want to work with this anymore.
There are still various subpages that cant f5 because the components cant handle an empty redux state in the beginning, but to be honest i don't care anymore. Lets hope the client will never find out, along with the "on error nothing happens"-bugs. The product should've been shipped last week, but thanks to mandatory bugfixes the release was postponed to next week. Then the next project starts...
Please give me some tips to keep up code quality over time, i cant take this once more.
I'm also aware that i could've done more, talking A and C about code style, prettifying the code, etc. Etc. But i was busy putting out my out fires, i couldn't kill much of the other fires which in the end became a burning building (a perfect metaphor for this software)4 -
I work with content. More specifically I work on content migration and improvement.
We connect to many platforms and pull and push documents into it. This one time we had to connect to some outrageously expensive (6 figures) system which we obviously couldn't afford to buy just for testing. The client wouldn't give us a testing server either.
My literal warning: "We need a testing server because we're gonna push it until it breaks. Then we know the limit." Client: "nah it will be fine." Us: "I promise you the server will go down..." Client: "It's a stable system. You can test in your own folder on our server"
10 minutes later we had an angry client because the server crashed due to overload.
I'm not sure if I'm annoyed or amused :p -
✓ running server on windows 10
✓ running postgresql server
✓ running ngrok server
✓ running android studio
✓ running 15 chrome tabs
✓ running ubuntu virtualbox
✓ scraping a website with python in ubuntu
✓ laptop freezes
✓ gently slam the laptop so it can go to sleep mode
✓ try to login so it can unfreeze
✓ get a purple screen of death
✓ system crashed and has to restart
✓ get a blue screen of death for memory diagnostic tool
✓ all unsaved work is lost
✓ gtx 1060
✓ i7
✓ ddr4
✓ 8gb ram
✓ acer laptop of $2400
✓ regret buying acer laptop
11 -
Pro tip: always make sure your methods return the correct variable.
I’m currently working with deep neural networks using tensorflow. I needed to generate some test data and wrote a program to create it. I had two text files which each consisted of approximately 5000 lines of text.
I wrote a method that should sort out some words, and make my final data shorter. When I executed the program first time on our server, it spent about 25 minutes, then crashed due to MemoryError (which in Python means that the server didn’t have enough ram). That seemed quite weird since I only had about 10k lines of text, and I even sorted out a bunch of it, and the server has 128gb ram, and nothing’s using it.
Apparently I returned the wrong variable. That meant that my program tried to save 750 quadrillion lines of text rather than just a few thousand.
Always make sure to return the correct variables!1 -
Three days ago my focus was shifted from a development role to a support role. I was shifted to replace another support guy who had used fraud to get the position. I have no experience with this role but there was decent KT and I'm catching on fine. During onboarding and KT I'm serving as the first contact for new tickets and whatnot...
Today I got a ticket with an error on our production instance that no one had ever seen before. It prevented the guy from using our service entirely. I tried to reproduce it and... I couldn't use the service either. No one could. Everything was down. I could see the sweat building on my manager's forehead.
Thankfully another member on my team has done a bit of support before, so we collaborated with each other and other teams throughout the day to figure out what's wrong and how to fix it. I'm listening to them chat remotely as we speak - so far I've been working on it 9 hours straight.
This service is used by everyone - it's a business critical service with due dates on actions and escalations to managers... Imagine if the support ticketing service for your company crashed. That means a lot of people are asking what's wrong, requiring extensions, etc. I've been answering to managers and seniors in the business throughout the day.
The best part? We figured out why the server went down, and the reason is fantastic: someone updated the server's code without telling anyone, and all they had done was remove critical parsing code. Just took it right out, pushed, redeployed. We don't know who did it or who even has access to do that. I guess I have some detective work cut out for me after we've fixed everything that was broken by that.
I miss coding already.1 -
A few years ago I worked at company specialized in Magento(eCommerce) and Magento was changing their licensing model. At the time they had 3 Versions. Community(free), Pro and Enterprise.
They decided to ditch Pro and either make all migrate to Enterprise(with a discount) or go community which wasn’t really compatible. So some shops were in need of a more or less complete rewrite.
My hdd crashed literally the day before but hey no big deal everything is 99% done and on staging. So I had a Trainee at that Time and thought the last few crappy things could be done in pair programming so he can learn a few things.
But fuck him! That motherfucker! He managed to WIPE the staging server and no that was at a time without gut and no SVN. That dipshit just deleted 2 months of work because he thought it was a good idea to SYNC his empty project to the staging system.
Oh god I nearly stabbed him. He did that shit out of his own mind even though I told him a dozen times what would happen... we had to do the whole thing again with me sitting next to him watching every stroke he made.
Guess he learned something while inward silently raging the next weeks.1 -
I spent 2 weeks at work building a dashbord (not a feature but...) wich provided an overview of projects and tasks managed in redmine (Kanban tool) by using its API. After i finished it we started using it till my boss found out that it was completly "useless" for him - it had all the features he asked for! -.- ...
three days later the redmine server crashed and we changed the provider, nobody missed my dashboard and so it got abandoned :( sometimes i miss it, it looked fancy af and stuff!!
But at least i learned a lot of js and API stuff. I was verry new to js back then :)
Boss asked -> I deliver -> and *pooff* down the river it goes. seems like my tasks have not changed much since then. -
Requests to a soap server were failing randomly. In order to contact the API provider, I tried to provide an curl example with the same payload and the error response. Yet when sending the payload over curl, the request worked just fine. When my application was building the request, it failed.
What. The. Fuck.
I checked and double-checked the request body and headers. They were identical.
Of course, no error response was returned by the API provider and, of course, they could not tell me how what error I caused in my request.
So I created a basic dummy server, installed wireshark and compared the payload when sending a request from my application and from curl to my dummy server.
It turns out: curl, if called in a certain way, automagically strips out newlines. The soap client kept them.
So that that shitty soap server crashed due to newlines in the message body!
Stripping out the newlines was rather easy.
Shame on you, your house, and entire family for letting it crash due to them!1 -
So I developed this proxy server that will throttle down API calls to one of our providers so we don't get blocked for TOS violation...
Some dude had a tool running all day long which crashed 2 minutes before I left work.
This literally ruined my day until I recalled it's all cached!!!!
Mood is back again and I deserve my beer! -
Was slightly panicking at work today when I couldn't reach my server at home anymore, though it might have crashed or something. Turns out my isp changed my fucking IP address! It hasn't done that in years! WHY NOW? Ugh, time to update *all* the dns records I suppose...2
-
That time you where asked to start a vm that would over allocate the servers resources, said I don't think that a good idea and where told to just do it.
Well as expected the server slowed to a crawl then crashed. Guess I'll be getting a bigger server now2 -
So it's a bank holiday today. Would be a shame if I crashed and you'd need to come in anyway...
- VPN Server -
Back when I was a freshman in high school a friend of mine put an emulator on the shared drive, so we could play NES games while in the computer lab. Didn't know better/didn't care. One day I get pulled out of class and walked into the computer guys office. In there is also the principal of the school and the Chief of police.
The computer guy tells me there was an issue last week that caused the school server to crash and it caused damage. I asked what happened and the he said one of the emulators we were playing had a script that crashed the server and caused damage. I asked how much damage and they informed me it was over 3 thousand dollars. At this point I'm very skeptical that the damage was worth about the cost of a new workstation (the old one sitting on his desk, buried in boxes), and afterwards none of the faculty knew of any kind of an outage. I asked for him to show me what broke and what had to be done to fix/replace the damaged equipment but all I got was a simple, "I'm sorry. I can't show you that at this time."
They threatened legal action for a felony of damaging a school property. Myself and the other tech savvy kids talked about it over the next couple of days wondering what would happen. They threatened expulsion for myself and a couple of other kids, but ultimately just got a talking to about keeping personal information safe.
What I got out of it was if they think I'm good with computers I must be doing something right. Now I'm in IT. This is where it went wrong. -
Hey guys, I have a question.
If you ever had to deal with parity (Ethereum node software) and ever ran archive node, you perfectly know how long this bastard synchronizes. For our server it took almost month. Well, today or yesterday parity decided to spontaniously blow up (or crash) corruptiong database manifest file which greatly undercut us.
Anyone knows any viable way to rebuild manifest file withoud doing full sync from a scratfch?
If anyone has any suggestions other than what 95% of internet say "well, delete your database and sync from scratch" Im happy to test if it will solve our burning burning issue.
I am sure there is some way to rebuild database, especially where it's manifest file that's corrupt (Ive checked it, for wtf reason parity decided to truncate the file when it crashed).
Database backend is written in rust, and is called rocks db.
EDIT: if helps, its archive fat database (fat db means it should be easier to recover?)4 -
so we are having a campus recruitment at college. So there's a company that came in campus . The first step was to register on their website however their website is so shitty that they have specified that their Website is working just on Chrome and IE and not on Firefox . I was like what kind of developer support IE over Firefox really on top my friends who were applying at this company spend over 5 hrs just to register and the worst part @ 9 their server crashed . I would rather drink bleach than joining such a company1
-
While I was browsing the server through a ftp client, it froze for a sec and the next thing i notice is that a folder is missing. It was then when i found out the "drag'n'drop" feature.. Ofcourse it took me a few minutes to figure it out and ofcourse everything crashed. Btw, it was an asp.net web application I fucked up..on public..with more than 100 clients...o.o
-
MySQL Innodb easily get crashed, bullshit, I just restarted my server now all databases get corrupted. F*ck you OVH3
-
I tried Appgyver over christmas, since it promised easy front-end (no-)coding I was looking forward to getting rudimentary frontends done faster.
Well, the first real project that I wanted to start didn't compile anymore (internal error from the service), the page told me to reload and try again.
It failed again... And again.
Fine with me, I only spent 10 minutes on the project at this point.
I then searched for the bugreporting page and found it. The sad thing is that when I wanted to open a ticket the server crashed. It didn't even return a HTTP error, just a JSON saying there is a error and a GUID.
I have to say, if a Dev decided to have holidays without new issues that's one way of getting that done.3
Top Tags
Weekly Rant
View