Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "production-issues"
-
Dear DEVS,
chmod -R 777 /
is not the right way to fix your application permissions issues!!!!!!
Yours truly,
sys ad who is not fixing your production server.23 -
Finally did it. Quit my job.
The full story:
Just came back from vacation to find out that pretty much all the work I put at place has been either destroyed by "temporary fixes" or wiped clean in favour of buggy older versions. The reason, and this is a direct quote "Ari left the code riddled with bugs prior to leaving".
Oh no. Oh no I did not you fucker.
Some background:
My boss wrote a piece of major software with another coder (over the course of month and a balf). This software was very fragile as its intention was to demo specific features we want to adopt for a version 2 of it.
I was then handed over this software (which was vanilajs with angular) and was told to "clean it up" introduce a typing system, introduce a build system, add webpack for better module and dependency management, learn cordova (because its essential and I had no idea of how it works). As well as fix the billion of issues with data storage in the software. Add a webgui and setup multiple databses for data exports from the app. Ensure that transmission of the data is clean and valid.
What else. This software had ZERO documentation. And I had to sit my boss for a solid 3hrs plus some occasional questions as I was developing to get a clear idea of whats going on.
Took a bit over 3 weeks. But I had the damn thing ported over. Cleaned up. And partially documented.
During this period, I was suppose to work with another 2 other coders "my team". But they were always pulled into other things by my Boss.
During this period, I kept asking for code reviews (as I was handling a very large code base on my own).
During this period, I was asking for help from my boss to make sure that the visual aspect of the software meets the requirements (there are LOTS of windows, screens, panels etc, which I just could not possibly get to checking on my own).
At the end of this period. I went on vacation (booked by my brothers for my bday <3 ).
I come back. My work is null. The Boss only looked at it on the friday night leading up to my return. And decided to go back to v1 and fix whatever he didnt like there.
So this guy calls me. Calls me on a friggin SUNDAY. I like just got off the plane. Was heading to dinner with my family.
He and another coder have basically nuked my work. And in an extremely hacky way tied some things together to sort of work. Moreever, the webguis that I setup for the database viewing. They were EDITED ON THE PRODUCTION SERVER without git tracking!!
So monday. I get bombarded with over 20 emails. Claiming that I left things in an usuable state with no documentation. As well as I get yelled at by my boss for introducing "unnecessary complicated shit".
For fuck sakes. I was the one to bring the word documentation into the vocabulary of this company. There are literally ZERO documentated projects here. While all of mine are at least partially documented (due to lack of time).
For fuck sakes, during my time here I have been basically begging to pull the coder who made the admin views for our software and clean up some of the views so that no one will ever have to touch any database directly.
To say this story is the only reason I am done is so not true.
I dedicated over a year to this company. During this time I saw aspects of this behaviour attacking other coders as well as me. But never to this level.
I am so friggin happy that I quit. Never gonna look back.14 -
A wild Darwin Award nominee appears.
Background: Admins report that a legacy nightly update process isn't working. Ticket actually states problem is obviously in "the codes."
Scene: Meeting with about 20 people to triage the issue (blamestorming)
"Senior" Admin: "update process not working, the file is not present"
Moi: "which file?"
SAdmin: "file that is in ticket, EPN-1003"
Moi: "..." *grumbles, plans murder, opens ticket*
...
Moi: "The config dotfile is missing?"
SAdmin: "Yes, file no there. Can you fix?"
Moi: "Engineers don't have access to the production system. Please share your screen"
SAdmin: "ok"
*time passes, screen appears*
Moi: "ls the configuration dir"
SAdmin: *fails in bash* > ls
*computer prints*
> ls
_.legacyjobrc
Moi: *sees issues, blood pressure rises* "Please run list all long"
SAdmin: *fails in bash, again* > ls ?
Moi: *shakes* "ls -la"
SAdmin: *shonorable mention* > ls -la
*computer prints*
> ls -la
total 1300
drwxrwxrwx- 18 SAdmin {Today} -- _.legacyjobrc
Moi: "Why did you rename the config file?"
SAdmin: "Nothing changed"
Moi: "... are you sure?"
SAdmin: "No, changed nothing."
Moi: "Is the job running as your account for some reason?"
SAdmin: "No, job is root"
Moi: *shares screenshot of previous ls* This suggests your account was likely used to rename the dotfile, did you share your account with anyone?
SAdmin: "No, I rename file because could not see"
Moi: *heavy seething* so, just to make sure I understand, you renamed a dotfile because you couldn't see it in the terminal with ls?
SAdmin: "No, I rename file because it was not visible, now is visible"
Moi: "and then you filed a ticket because the application stopped working after you renamed the configuration file? You didn't think there might be a correlation between those two things?"
SAdmin: "yes, it no work"
Interjecting Director: "How did no one catch this? Why were there no checks, and why is there no user interface to configure this application? When I was writing applications I cared about quality"
Moi: *heavy seething*
IDjit: "Well? Anyone? How are we going to fix this"
Moi: "The administrative team will need to rename the file back to its original name"
IDjit: "can't the engineering team do this?!"
Moi: "We could, but it's corporate policy that we have no access to those environments"
IDjit: "Ok, what caused this issue in the first place? How did it get this way?!"
TFW you think you've hit the bottom of idiocy barrel, and the director says, "hold my mango lassi."27 -
toxic workplace; leaving
I haven't wanted to write this rant. I haven't even wanted to talk to anyone (save my gf, ofc). I've just been silently fuming.
I wrote a much longer rant going into far too much detail, but none of that is relevant, so I deleted it and wrote this shorter (believe it or not) version instead. And then added in more details because details.
------
On Tuesday, as every Tuesday, I had a conference call with the rest of the company. For various, mostly stupid reasons, the boss yelled at and insulted me for twenty minutes straight in front of everyone, telling me how i'm disorganized, forgetful, how can't manage my time, can't manage myself let alone others, how I don't have my priorities straight, etc. He told the sales team to get off the call, and then proceeded to yell and chew at me for another twenty minutes in front of the frontend contractor about basically the same things. The call was 53 minutes, and he spent 40 minutes of it telling me how terrible I've been. No exaggeration, no spin. The issues? I didn't respond to an email (it got lost in my ever-filling inbox), and I didn't push a very minor update last week (untested and straight to prod, ofc). (Side note: he's yelled at me for ~15 minutes before for being horribly disorganized and unable to keep up on Trello -- because I had a single card in the wrong column. One card, out of 60+ over two boards. Never mind that most have time estimates, project tags, details, linked to cards on his boards, columns for project/qa/released, labels for deferred, released to / rejected from qa, finished, in production, are ordered by priority, .... Yep. I'm totes disorganized.)
Anyway, I spent most of conference call writing "Go fuck yourself," "Choke on a cat and die asshole," "Shit code, low pay, and broken promises. what a prize position," etc. or flipping him off under the camera on our conference-turn-video-call (switched due to connection issues, because ofc video is more stable than audio-only in his mind).
I'm just.
so, so done.
I did nothing the rest of the day on Tuesday, and basically just played games on Wednesday. I did one small ticket -- a cert replacement since that was to expire the next day -- but the rest was just playing CrossCode. (fun game, fyi; totally recommend.)
Today? It's 3:30pm and I can't be bothered to do anything. I have an "urgent" project to finish by Monday, literally "to give [random third party sales guy] a small win". Total actual wording. I was to drop all other tasks (even the expiring cert lol) and give this guy his small win. fucking whatever. But the project deals with decent code -- it's a minor extension to the first project I did for the company (see my much earlier rants), back when I was actually applying myself and learning something (everything) new, enjoying myself, and architecting+writing my own code. So I might actually do the project, but It's been two days and I haven't even opened single file yet.
But yeah. This place is total and complete shit. Dealing with the asshole reminds me of dealing with my parents while growing up, and that's a subject I don't want to broach -- far too many toxic memories.
So, I'm quitting as soon as I find something new.
and with luck, this will be before assface hires my replacement-to-be, and who will hopefully quit as soon as s/he sees the abysmal codebase. With even more luck, the asshole king himself will get to watch his company die due to horrible mismanagement. (though ofc he'll never attribute it to himself. whatever.)
I just never want to see or think about him again.
(nor this fetid landfill of a codebase. bleh.)
With luck, this will be one of my last rants about this toxic waste dump and its king of the pile.
Fourty fucking minutes, what the fuck.34 -
Seven months ago:
===============
Project Manager: - "Guys, we need to make this brand new ProjectX, here are the specs. What do you think?"
Bored Old Lead: - "I was going to resign this week but you've convinced me, this is a challenge, I never worked with this stack, I'm staying! I'll gladly play with this framework I never used before, it seems to work with this libA I can use here and this libB that I can use here! Such fun!"
Project Manager: - "Awesome! I'm counting on you!"
Six months ago:
====================
Cprn: - "So this part you asked me to implement is tons of work due to the way you're using libA. I really don't think we need it here. We could use a more common approach."
Bored Old Lead: - "No, I already rewrote parts of libB to work with libA, we're keeping it. Just do what's needed."
Cprn: - "Really? Oh, I see. It solves this one issue I'm having at least. Did you push the changes upstream?"
Bored Old Lead: - "No, nobody uses it like that, people don't need it."
Cprn: - "Wait... What? Then why did you even *think* about using those two libs together? It makes no sense."
Bored Old Lead: - "Come on, it's a challenge! Read it! Understand it! It'll make you a better coder!"
Four months ago:
==============
Cprn: - "That version of the framework you used is loosing support next month. We really should update."
Bored Old Lead: - "Yeah, we can't. I changed some core framework mechanics and the patches won't work with the new version. I'd have to rewrite these."
Cprn: - "Please do?"
Bored Old Lead: - "Nah, it's a waste of time! We're not updating!"
Three months ago:
===============
Bored Old Lead: - "The code you committed doesn't pass the tests."
Cprn: - "I just run it on my working copy and everything passes."
Bored Old Lead: - "Doesn't work on mine."
Cprn: - "Let me take a look... Ah! Here you go! You've misused these two options in the framework config for your dev environment."
Bored Old Lead: - "No, I had to hack them like that to work with libB."
Cprn: - "But the new framework version already brings everything we need from libB. We could just update and drop it."
Bored Old Lead: - "No! Can't update, remember?"
Last Friday:
=========
Bored Old Lead: - "You need to rewrite these tests. They work really slow. Two hours to pass all."
Cprn: - "What..? How come? I just run them on revision from this morning and all passed in a minute."
Bored Old Lead: - "Pull the changes and try again. I changed few input dataset objects and then copied results from error messages to assertions to make the tests pass and now it takes two hours. I've narrowed it to those weird tests here."
Cprn: - "Yeah, all of those use ORM. Maybe it's something with the model?"
Bored Old Lead: - "No, all is fine with the model. I was just there rewriting the way framework maps data types to accommodate for my new type that's really just an enum but I made it into a special custom object that needs special custom handling in the ORM. I haven't noticed any issues."
Cprn: - "What!? This makes *zero* sense! You're rewriting vendor code and expect everything to just work!? You're using libs that aren't designed to work together in production code because you wanted a challenge!?? And when everything blows up you're blaming my test code that you're feeding with incorrect dataset!??? See you on Monday, I'm going home! *door slam*"
Today:
=====
Project Manager: - "Cprn, Bored Old Lead left on Friday. He said he can't work with you. You're responsible for Project X now."24 -
🎶 Fixing production issues 🎶
🎶 Fixing production issues 🎶
🎶 In other people’s code! 🎶
Seriously, how am I still in a good mood when I have to deal with this?14 -
Worst dev team failure I've experienced?
One of several.
Around 2012, a team of devs were tasked to convert a ASPX service to WCF that had one responsibility, returning product data (description, price, availability, etc...simple stuff)
No complex searching, just pass the ID, you get the response.
I was the original developer of the ASPX service, which API was an XML request and returned an XML response. The 'powers-that-be' decided anything XML was evil and had to be purged from the planet. If this thought bubble popped up over your head "Wait a sec...doesn't WCF transmit everything via SOAP, which is XML?", yes, but in their minds SOAP wasn't XML. That's not the worst WTF of this story.
The team, 3 developers, 2 DBAs, network administrators, several web developers, worked on the conversion for about 9 months using the Waterfall method (3~5 months was mostly in meetings and very basic prototyping) and using a test-first approach (their own flavor of TDD). The 'go live' day was to occur at 3:00AM and mandatory that nearly the entire department be on-sight (including the department VP) and available to help troubleshoot any system issues.
3:00AM - Teams start their deployments
3:05AM - Thousands and thousands of errors from all kinds of sources (web exceptions, database exceptions, server exceptions, etc), site goes down, teams roll everything back.
3:30AM - The primary developer remembered he made a last minute change to a stored procedure parameter that hadn't been pushed to production, which caused a side-affect across several layers of their stack.
4:00AM - The developer found his bug, but the manager decided it would be better if everyone went home and get a fresh look at the problem at 8:00AM (yes, he expected everyone to be back in the office at 8:00AM).
About a month later, the team scheduled another 3:00AM deployment (VP was present again), confident that introducing mocking into their testing pipeline would fix any database related errors.
3:00AM - Team starts their deployments.
3:30AM - No major errors, things seem to be going well. High fives, cheers..manager tells everyone to head home.
3:35AM - Site crashes, like white page, no response from the servers kind of crash. Resetting IIS on the servers works, but only for around 10 minutes or so.
4:00AM - Team rolls back, manager is clearly pissed at this point, "Nobody is going fucking home until we figure this out!!"
6:00AM - Diagnostics found the WCF client was causing the server to run out of resources, with a mix of clogging up server bandwidth, and a sprinkle of N+1 scaling problem. Manager lets everyone go home, but be back in the office at 8:00AM to develop a plan so this *never* happens again.
About 2 months later, a 'real' development+integration environment (previously, any+all integration tests were on the developer's machine) and the team scheduled a 6:00AM deployment, but at a much, much smaller scale with just the 3 development team members.
Why? Because the manager 'froze' changes to the ASPX service, the web team still needed various enhancements, so they bypassed the service (not using the ASPX service at all) and wrote their own SQL scripts that hit the database directly and utilized AppFabric/Velocity caching to allow the site to scale. There were only a couple client application using the ASPX service that needed to be converted, so deploying at 6:00AM gave everyone a couple of hours before users got into the office. Service deployed, worked like a champ.
A week later the VP schedules a celebration for the successful migration to WCF. Pizza, cake, the works. The 3 team members received awards (and a envelope, which probably equaled some $$$) and the entire team received a custom Benchmade pocket knife to remember this project's success. Myself and several others just stared at each other, not knowing what to say.
Later, my manager pulls several of us into a conference room
Me: "What the hell? This is one of the biggest failures I've been apart of. We got rewarded for thousands and thousands of dollars of wasted time."
<others expressed the same and expletive sediments>
Mgr: "I know..I know...but that's the story we have to stick with. If the company realizes what a fucking mess this is, we could all be fired."
Me: "What?!! All of us?!"
Mgr: "Well, shit rolls downhill. Dept-Mgr-John is ready to fire anyone he felt could make him look bad, which is why I pulled you guys in here. The other sheep out there will go along with anything he says and more than happy to throw you under the bus. Keep your head down until this blows over. Say nothing."11 -
IF LIVES DEPEND ON A SYSTEM
1. Code review, collaboration, and knowledge sharing (each hour of code review saves 33 hours of maintenance)
2. TDD (40% — 80% reduction in production bug density)
3. Daily continuous integration (large code merges are a major source of bugs)
4. Minimize developer interruptions (an interrupted task takes twice as long and contains twice as many defects)
5. Linting (catches many typo and undefined variable bugs that static types could catch, as well as a host of stylistic issues that correlate with bug creation, such as accidentally assigning when you meant to compare)
6. Reduce complexity & improve modularity -- complex code is harder to understand, test, and maintain
-Eric Elliott12 -
A while back a co-worker of mine fucked up by leaving some debug code and pushing to production.
He quickly repaired it, redeployed and everything was good again before the customer experienced any issues.
Later that day, management showed up by his desk to ask what happened, how it happened and stating that he was not "angry enough" about his fuck up, long after it has been repaired.
Up to this day i regret not asking in what unit of measurement we could determine if we were angry enough; decibels? gray hairs? grams of shit in my underwear?4 -
We had issues with lack of disk space on our production SQL server. Another developer decided to delete the databases he thought weren't in use to clear some space.
Ever think about checking first?!
Production chaos!7 -
PM (on slack): "we’re about to deploy to production".
Me: "ok"
… I keep on working on a task / remain available for any post deployment issues …
PM (5 minutes later on slack): "deployment broke production! We need to handle this NOW!"
My dev colleague has already called it a day, but I’m still online
Me: "ok I don’t have access to prod, can you describe what’s going on? I can’t reproduce on any other environment"
PM: …
10 minutes go by
Me: "anybody there?"
PM: …
45 minutes later, I realize PM is offline
The following day:
PM: "ok we got prod running again" (turns out it was client’s fault for not updating a config we as devs can’t access)
PM: "but we’re REALLY UPSET! You guys need to be available to intervene for any issues following deployment to production! At least one of you should be available!"
Me: "but, but…" 🫠14 -
The world makes no fucking sense.
In 2013 I had a manager approve a couple days' leave coz my son was having medical issues.
He was super nice about it and told me I could take as much time as I needed. I said, a couple days is enough. I took Thursday and Friday off. I took two days.
On Monday, an emergency meeting was held with the CTO (it was a small company, it went me -> manager -> C suite). I was told that a production deployment happened on Friday that fucked up a few clients' systems and that it had cost said clients hundreds of thousands dollars and are now suing the company.
Turns out on Friday, lead developer was also given the day off for whatever reason and I was being scolded because as the next senior developer, it was my responsibility to review code and make sure shit like this doesn't happen.
I agreed (and still agree) but also explained I had already filed leave weeks prior and I wasn't informed about dev lead's absence. Sure I could've checked my messages but my kid was in the hospital and I was busy. Still I couldn't help but feel a little guilty.
Manager holds a separate meeting with me and talks me into just writing an apology note in the email chain and he'll do the rest of the talking for me and make sure I get minimal punishment. I trusted him, he was the one who found me and brought me into the company (I know, I was naive).
So I wrote the email. It was a small note. I apologized for not checking messages and explained my situation again and mentioned I would've definitely checked if I was informed that the lead dev would be away.
Another meeting was held the next day and after pleasantries the Manager started with this, "Ok so we've all seen the email and understand that this was all Angry's fault right?".
Now, we're not native English speakers and Manager doesn't really do well with grammar. I was alarmed by what he said but wasn't angry because I was pretty sure that's not what he meant. I'm sure he meant to say that "Angry feel's guilty but his actions were understandable given the circumstance" or that he forgot a "not" in there and really meant "not Angry's fault". Surely this is what he meant to say. Right?
But then the rest of the meeting went on and I was unceremoniously let go. Immediately for "failing to accomplish my tasks and costing the client 100Ks of dollars". I wasn't even given a chance to say anything else.
The meeting ended and since we were both in the office, Manager approached me with exit papers and a check (~1200 USD)--it was my month's pay. I was asked to leave that day and was told I didn't need to come back. No handovers, no knowledge transfers, not a even a documentation of open projects I was handling.
I realized I just was made the scapegoat by a management screwup that costed our clients a lot of money.
Of course, I wrote the CEO multiple emails the next couple days. I also cc'd the CTO. No response.
A couple of weeks pass, I get another job at a cool company and i promptly move on.
I write this story now because I just found out today that in 2016, Manager was let go by the company for **sexual harassment**. Apparently, he actually did it too according to friends I still had within the company.
Here's where it gets fucked up. He turns and sues the company for unlawful termination and I guess to avoid a long legal battle? the company settled. They fucking settled and handed this man 2 Million PHP (at the time about 40k USD).
2 fucking million. Life changing money around here. And he got it by being a slimy piece of shit.
The world makes no fucking sense.10 -
Be me, new dev on a team. Taking a look through source code to get up to speed.
Dev: **thinking to self** why is there no package lock.. let me bring this up to boss man
Dev: hey boss man, you’ve got no package lock, did we forget to commit it?
Manager: no I don’t like package locks.
Dev: ...why?
Manager: they fuck up computer. The project never ran with a package lock.
Dev: ..how will you make sure that every dev has the same packages while developing?
Manager: don’t worry, I’ve done this before, we haven’t had any issues.
**couple weeks goes by**
Dev: pushes code
Manager: hey your feature is not working on my machine
Dev: it’s working on mine, and the dev servers. Let’s take a look and see
**finds out he deletes his package lock every time he does npm install, so therefore he literally has the latest of like a 50 packages with no testing**
Dev: well you see you have some packages here that updates, and have broken some of the features.
Manager: >=|, fix it.
Dev: commit a working package lock so we’re all on the same.
Manager: just set the package version to whatever works.
Dev: okay
**more weeks go by**
Manager: why are we having so many issues between devs, why are things working on some computers and not others??? We can’t be having this it’s wasting time.
Dev: **takes a look at everyone’s packages** we all have different packages.
Manager: that’s it, no one can use Mac computers. You must use these windows computers, and you must install npm v6.0 and node v15.11. Everyone must have the same system and software install to guarantee we’re all on the same page
Dev: so can we also commit package lock so we’re all having the same packages as well?
Manager: No, package locks don’t work.
**few days go by**
Manager: GUYS WHY IS THE CODE DEPLOYING TO PRODUCTION NOT WORKING. IT WAS WORKING IN DEV
DEV: **looks at packages**, when the project was built on dev on 9/1 package x was on version 1.1, when it was approved and moved to prod on 9/3 package x was now on version 1.2 which was a change that broke our code.
Manager: CHANGE THE DEPLOYMENT SCRIPTS THEN. MAKE PROD RSYNC NODE_MODULES WITH DEV
Dev: okay
Manager: just trust me, I’ve been doing this for years
Who the fuck put this man in charge.11 -
Summary: Burnout, and everything's broken.
I don't feel like doing a damn thing today. I look at the code and cringe. I look at Slack and think "ugh. i can't." Mental capitals are even too much work.
(I've started reading "Zen and the Art of Motorcycle Maintenance" to try and combat burnout. I'll write a rant/story about it here if I find it helpful. but all I want to do today is drink tea and read.)
But onto the story:
Heroku is deprecating support for and will automatically upgrade any old verisons of Postgres running on its platform after August something (like five days from now).
I performed the upgrade to PG10 on Sunday (and late into the night), provisioning a new follower, blah blah blah.
However, the version of Rails we're using (4.2.x) doesn't support PG10 sequences, so I manually added in support via a monkeypatch. I did this on our QA servers first, obviously, and everything worked as expected. After half a day of no issues, I did the same on production, and again: everything worked as expected.
But today? I keep hearing about new things that are broken. One specific type of alert doesn't work for one specific person (wat). Can't send [redacted] at all. Can't update merchants! Yet there are magically no errors logged.
That last one (well, two) are just great; let me explain: when there's an error concerning merchants, the error gets caught, isn't logged or recorded anywhere so it just disappears, and the rescue block triggers a json response instead and happily exits. This is for an internal admin tool, so returning a user-friendly error is kinda stupid anyway, but masking what actually happened? fuck that dev with an obelisk made from spikes and solidified pain. That json response is also lovely: it's a 200 OK returning {status: 1, data: "[generic message containing incorrect IT jargon]"}. Doesn't even say "error" anywhere. Bloody everything about this pattern is absolutely wrong. Even the friggin' text.
Fucking hell. I want to pipe the entire codebase into shred and walk out the door.
But I digress. So many things are broken, my motivation is wanning to a sliver, and I have a conference call today where I'll undoubtedly be asked why everything is on smoking and/or on fire, and my huge and overly productive week last week will ofc mean nothing by contrast.
Ugh.
`shred ~/dev/work -zfu -n 32 &; ./brew tea --hot && wine ~/takeabreak.exe`rant zen and the art of motorcycle maintenance postgres heroku ship's sinking and the fixer's all fixed out burnout21 -
Ooof.
In a meeting with my client today, about issues with their staging and production environments.
They pull in the lead dev working on the project. He's a 🤡 who freelanced for my previous company where I was CTO.
I fired him for being plain bad.
Today he doesn't recognize me and proceeds to patronize me in server administration...
The same 🤡 that checks production secrets into git, builds projects directly in the production vm.
Buckle up... Deploys *both* staging and production to the *same* vm...
Doesn't even assign a static IP to the VM and is puzzled when its IP has changed after a relaunch...
Stores long term aws credentials instead of using instance roles.
Claims there are "memory leaks", in a js project. (There may be memory misuse by project or its dependencies, an actual memory leak in v8 that somehow only he finds...? Don't think so.)
Didn't even set up pm2 in systemd so his services didn't even relaunch after a reboot...
You know, I'm keeping my mouth shut and make the clown work all weekend to fix his own hubris.11 -
A colleague named Sam was really pissed off today at an out sourcing firm from India.
My Boss outsourced an application to India based firm. Sam was the one handling the project after the handover. Sam coded a feature 2 weeks ago and moved to staging server for approval. After the sign off from the lead developer of the outsourcing firm, he moved the feature to production. For the past 2 days the application was crashing over and over again so Sam went to check and found out that the feature he coded was causing the issue. When he pulled the feature to his computer and had a look at the code, it wasn’t his code. The code he wrote was commented out and the lead developer of the outsourcing firm wrote new code.
When Sam emailed to him regarding this he replied that he re-wrote his code to fix issues with the feature. Sam and outsourcing firm lead developer had heated argument about this. It’s turns out that the outsourcing developer re-wrote the code without anyone’s approval and on production server.
The lead developer of the outsourcing firm was fired.7 -
Other PM: We must fix the database performance issues now.
Me: We can't. We're still only halfway on the dependency chain to tackle this and honestly, even if the dependency chain would be fulfilled, I'd leave at least 2 weeks monitoring the production after the changes were rolled out before we further poke around.
Other PM: This is taking far too long. And whaddya mean by dependency chain? Why was I not informed about this?
Me: *sigh* like in every meeting in the last weeks: the dependency chain are the current open blockers before we can proceed with the database changes. We've talked about this _at length_... Especially why these blockers exist.
Other PM: No, we need to start now. I've _examined_ at the blockers or "dependency chain" as you call it.
(Examined.... He opened on his currently streaming laptop, which was connected to the active beamer, the mentioned ticket with a detailed blocker ... And quickly scrolled. Yeeah. Warmonger...).
Me: I'm very tired of discussing this. But since you are already presenting us the ticket, read out the referenced meeting notes... We explained it in great detail.
Other PM: Why? This is just a waste of time!!!!!!!
--
Yes. This happened. Other PM was my nemesis.
In this meeting were 2 PMs (Him, Me)… I think 5 - 7 devs... And we were sitting in this meeting since 2 hours at least. Everyone was angry...
After this "manifesto of intelligence"… I simply left the room, followed by a few devs.
And yes. Other PM did this on a regular basis....5 -
I suddenly realized all the technical debt shit I told my boss would happen years ago given the way things were done/heading then... Just occurred pretty much all at once last week in the form of critical production issues...
The teams like:
-we need real time server process monitoring
-structured logging for apps
-containerization so one app didn't affect others
Me thinking: yes.... I told you so like 3/4 years ago when I first joined the team and kept repeating so much I got tired of saying at every annual review...
This is exactly what happens when you let technical debt grow and have no free time for developers to look into and fix then while they were small and not critical production processes... Or properly document and peer review them... (Got a shit pile of projects that no one knows how to use or even exists because the devs left the team) and they'll have a lot more when I finally leave... Hopefully this year.... If I can find another role and not need another medical procedure... (Doubtful)3 -
I think I want to quit my first applicantion developer job 6 months in because of just how bad the code and deployment and.. Just everything, is.
I'm a C#/.net developer. Currently I'm working on some asp.net and sql stuff for this company.
We have no code standards. Our project manager is somewhere between useless and determinental. Our clients are unreasonable (its the government, so im a bit stifled on what I can say.) and expect absurd things from us. We have 0 automated tests and before I arrived all our infrastructure wasn't correct to our documentation... And we barely had any documentation to begin with.
The code is another horror story. It's out sourced C# asp.net, js and SQL code.. And to very bad programmers in India, no offense to the good ones, I know you exist. Its all spagheti. And half of it isn't spelled correctly.
We have a single, massive constant class that probably has over 2000 constants, I don't care to count. Our SQL projects are a mess with tons of quick fix scripts to run pre and post publishing. Our folder structure makes no sense (We have root/js and root/js1 to make you cringe.) our javascript is majoritly on the asp.net pages themselves inline, so we don't even have minification most of the time.
It's... God awful. The result of a billion and one quick fixes that nobody documented. The configuration alone has to have the same value put multiple times. And now our senior developer is getting the outsourced department to work on moving every SINGLE NORMAL STRING INTO THE DATABASE. That's right. Rather then putting them into some local resource file or anything sane, our website will now be drawing every single standard string from the database. Our SENIOR DEVELOPER thinks this is a good idea. I don't need to go into detail about how slow this is. Want to do it on boot? Fine. But they do it every time the page loads. It's absurd.
Our sql database design is an absolute atrocity. You have to join several tables together just to get anything done. Half of our SP's are failing all the time because nobody really understands the design. Its gloriously awful its like.. The epitome of failed database designs.
But rather then taking a step back and dealing with all the issues, we keep adding new features and other ones get left in the dust. Hell, we don't even have complete browser support yet. There were things on the website that were still running SILVERLIGHT. In 2019. I don't even know how to feel about it.
I brought up our insane technical debt to our PM who told me that we don't have time to worry about things like technical debt. They also wouldn't spend the time to teach me anything, saying they would rather outsource everything then take the time to teach me. So i did. I learned a huge chunk of it myself.
But calling this a developer job was a sick, twisted joke. All our lives revolve around bugnet. Our work is our BN's. So every issue the client emails about becomes BN's. I haven't developed anything. All I've done is clean up others mess.
Except for the one time they did have me develop something. And I did it right and took my time. And then they told me it took too long, forced me to release before it was ready, even though I had never worked on what I was doing before. And it worked. I did it.
They then told me it likely wouldn't even be used anyway. I wasn't very happy at all.
I then discovered quickly the horrors of wanting to make changes on production. In order to make changes to it, we have to... Get this
Write a huge document explaining why. Not to our management. To the customer. The customer wants us to 'request' to fix our application.
I feel like I am literally against a wall. A huge massive wall. I can't get constent from my PM to fix the shitty code they have as a result of outsourcing. I can't make changes without the customer asking why I would work on something that doesn't add something new for them. And I can't ask for any sort of help, and half of the people I have to ask help from don't even speak english very well so it makes it double hard to understand anything.
But what can I do? If I leave my job it leaves a lasting stain on my record that I am unsure if I can shake off.
... Well, thats my tl;dr rant. Im a junior, so maybe idk what the hell im talking about.rant code application bad project management annoying as hell bad code c++ bad client bad design application development16 -
I tried writing this rant before, but I was (and still am) in too good of a mood so it was lengthy, meandering, and over-specific. so I'll summarize(ish).
summary:
* miscommunication
* working weekends
* incompetence and/or screwy integrations
summary of the summary:
* I can't fix someone else's mess if you don't talk to me!
Summary^3: #TODO: learn telepathy
Shortened rant:
Bossman at work signed up a very lucrative client by promising them something he couldn't deliver because he misunderstood and miscommunicated scope -- anti-fraud, if you've been following my rants.
Their signup (all four...) are screwy and cause issues and nobody knows why. I didn't write the code, have barely even glanced through it, and it uses a third-party (Clover) that's rather screwy.
Bossman has been asking me to do various things concerning the merchant, but has never been around to provide specifics, so I'm left to guess. I've done my best, but due to the aforementioned screwiness, I really have no idea what's going on. I just sort of muddled my way through.
Bossman also asked me, super late on Friday night (after 8:30pm), to rename one of the merchants because there are two with the same name (with different Clover creds, etc.) and that's just confusing. I didn't see the message because late and tired, and he didn't follow up or text/call me until two days later (today, Sunday). I also thought these were strictly for diagnosing and were de-listed. I had no idea the merchant was live and people were actually purchasing things for it. Had I known this I would have freaked out and demanded specifics on Thursday/Friday because wtf? debugging in production? with broken merchants? selling things for real money? scary bad? hello?
Anyway, I didn't see his message until he texted me about it at like 5pm today while I was about 2 hours from my computer. He's understandably frustrated, and I totally don't blame him, but fuck, miscommunication is a serious problem in this company, and that's amazing because it's so freaking small.
But the short version is that I'm likely going to get blamed for all this, Clover screwiness included. Bossman and I set up a call for 10am tomorrow and I'm positive he's going to try pinning it on me. Totally not going to let him, but his social is lv16+ while mine's like. 2 or 3. 😕 I'll see how it goes.
Really though, I should read @rutee07's book and just roast these fuckers.rant weekend work debugging in production miscommunication no call no text still my fault hope you see this it's urgent clover strip club3 -
Step 1: Run to the store to buy a USB card reader because all of a sudden you have a need to use a 16Mb CF card that was tossed in a junk drawer for 20 years (hoping it still works, of course), but that was the easy part...
Step 2: Realize that the apps - your own - you want to run on your new (old) Casio E-125 PocketPC (to re-live "glory" days) are compiled in ARM format, not MIPS, which is the CPU this device uses, and the installer packages you have FOR YOUR OWN APPS don't include MIPS, only ARM (WHY DID I DO THAT?!), so, the saga REALLY begins...
Step 3: Get a 20-year old OS to install in a Hyper-V VM... find out that basic things like networking don't work by default because the OS is so damn old, so spend hours solving that and other issues to get it to basically run well enough to...
Step 4: Get that OS updated so that it's at least kind/sorta/maybe (but between you and me, not really!) safe online, all without a browser that will work on ANY modern site (oh, and good luck finding a version of Firefox that runs on it - that all took a few hours)...
Step 5: Okay, OS is ready to go, now get 20-year old dev tools that you haven't even seen in that many years working. Oh, do this with a missing CD key and ISO's that weren't archived in a format that's usable today, plus a bunch of missing dependencies because the OS is, again, SO old (a few MORE hours)...
Step 6: Get 20-year old code written in a language you haven't used in probably almost that long to compile, dealing with pathing issues, missing libs, and several other issues, all the while trying to dust off long-dormant knowledge somewhere in the deep, dark recesses of your brain... surprisingly, it all came back to me, more or less, in under an hour, which lead to...
Step 7: FINALLY get it all to work, FINALLY get the code to compile, FINALLY get it transferred to the device (which has no network capabilities, by the way, which is where the card reader and CF card came into play) and re-live the glory of your old, crappy PocketPC apps and games running on the real thing! WOO-HOO!
Step 8: Realize it's 3:30am by the time that's all done and be VERY thankful that you're on vacation this week or work tomorrow would SSUUCCKK!!!!
Step 9. Get called into work the next day for a production issue despite being tired from the night before and an afternoon of errands, lose basically a whole day of vacation (7 hours spent on it) and not actually resolve it by after midnight when you finally say that's enough :(
Talk about your highs and your lows.6 -
DEAR CTOs, PLEASE ASK THE DEVELOPER OF THE SOFTWARE WHICH YOU ARE PLANNING TO BUY IN WHAT LANGUAGE AND WHAT VERSION THEY ARE WRITTEN IN.
Background: I worked a LONG time for a software company which developed a BIG crm software suite for a very niche sector. The softwary company was quite successfull and got many customers, even big companies bought our software. The thing is: The software is written in Ruby 1.8.7 and Rails 2. Even some customer servers are running debian squeeze... Yes, this setup is still in production use in 2022. (Rails 7 is the current version). I really don't get it why no one asked for the specific setup, they just bought it. We always told our boss, that we need time to upgrade. But he told every time, no one pays for an tech upgrade... So there it is, many TBs of customer data are in systems which are totally old, not updated and with possibly security issues.9 -
Refactored an authentication library a while back and teams are now getting around to updating their nuget packages.
It is a breaking change, but a simple one. The constructor takes a connection string, application name, and user name.
A dev messages me yesterday saying ...
Tom: "I made the required changes, but I'm getting a null reference exception when I try to use the authorization manager"
Odd because the changes have been in production for months in other apps, so I asked him to send me a screen shot of how he was using the class (see attached image below).
Me: "Send me a screenshot of how you are using the class"
<I look at what he sent>
Me: "Do you really not see the problem why it is not working?"
<about 10 minutes later>
Tom: "Do I need to pass a real connection string? The parameter hint didn't say exactly what I should pass."
<not true, but I wasn't going to embarrass him any more>
<5 minutes later>
Tom: "The authorization still isn't working"
Me: "Do you still have 'UserName' instead of the actual user name?"
<few minutes later>
Tom: "Authorization is working perfect, thanks!"
A little while later my manager messages me..
B:"I'm getting reports from managers that developers are having a lot of problems with the changes to the authorization nuget package. Were these changes tested? Can you work with the teams to get these issues resolved as soon as possible? I want this to be your top priority today."
Me: "It was Tom"
B: "Never mind."11 -
So I work as a "Web Development Lead"
Which means I lead (frontend,backend and infrastructure teams)
Also I am in charge of infrastructure or devops or whatever you call it, which means I handle production issues, dev and staging environments,...etc
and I am a team of one, and today I asked for a day off because it's my wife's birthday
and suddenly everyone is blocked, everything is on fire, and the phone is not stopping ringing, I had to go out of the cinema theater to answer the non-stopping calls
I AM ASKING FOR A SINGLE DAY, A FUCKING DAY, EVEN IF SOMEONE IS BLOCKED SO WHAT IT'S NOT EVEN A DAY I ONLY NEED 6 HOURS
IS TO TOO MUCH FUCKING TO ASK4 -
Worst collaboration experience story?
I was not directly involved, it was a Delphi -> C# conversion of our customer returns application.
The dev manager was out to prove waterfall was the only development methodology that could make convert the monolith app to a lean, multi-tier, enterprise-worthy application.
Starting out with a team of 7 (3 devs, 2 dbas, team mgr, and the dev department mgr), they spent around 3 months designing, meetings, and more meetings. Armed with 50+ page specification Word document (not counting the countless Visio workflow diagrams and Microsoft Project timeline/ghantt charts), the team was ready to start coding.
The database design, workflow, and UI design (using Visio), was well done/thought out, but problems started on day one.
- Team mgr and Dev mgr split up the 3 devs, 1 dev wrote the database access library tier, 1 wrote the service tier, the other dev wrote the UI (I'll add this was the dev's first experience with WPF).
- Per the specification, all the layers wouldn't be integrated until all of them met the standards (unit tested, free from errors from VS's code analyzer, etc)
- By the time the devs where ready to code, the DBAs were already tasked with other projects, so the Returns app was prioritized to "when we get around to it"
Fast forward 6 months later, all the devs were 'done' coding, having very little/no communication with one another, then the integration. The service and database layers assumed different design patterns and different database relationships and the UI layer required functionality neither layers anticipated (ex. multi-users and the service maintaining some sort of state between them).
Those issues took about a month to work out, then the app began beta testing with real end users. App didn't make it 10 minutes before users gave up. Numerous UI logic errors, runtime errors, and overall app stability. Because the UI was so bad, the dev mgr brought in one of the web developers (she was pretty good at UI design). You might guess how useful someone is being dropped in on complex project , months after-the-fact and being told "Fix it!".
Couple of months of UI re-design and many other changes, the app was ready for beta testing.
In the mean time, the company hired a new customer service manager. When he saw the application, he rejected the app because he re-designed the entire returns process to be more efficient. The application UI was written to the exact step-by-step old returns process with little/no deviation.
With a tremendous amount of push-back (TL;DR), the dev mgr promised to change the app, but only after it was deployed into production (using "we can fix it later" excuse).
Still plagued with numerous bugs, the app was finally deployed. In attempts to save face, there was a company-wide party to celebrate the 'death' of the "old Delphi returns app" and the birth of the new. Cake, drinks, certificates of achievements for the devs, etc.
By the end of the project, the devs hated each other. Finger pointing, petty squabbles, out-right "FU!"s across the cube walls, etc. All the team members were re-assigned to other teams to separate them, leaving a single new hire to fix all the issues.5 -
90%+ of production issues are caused by developers believing production machines are magic and have unlimited resources so can handle any shit code they write11
-
TL;DR : 38837+ stargazers repo thought it is cool to design snow on top of their UI buttons, and also changed their titles to “Ho ho ho”. Received "This is not good for production!!!" issues tickets.
People from /r/programming or devs from China or react devs might know this story by now.
Story : http://blog.shunliang.io/frontend/...
Lesson of the story : Do not go crazy extent for cool things for your product.9 -
We started a project in January for which I was the sole developer, to automate tedious interaction with a vendor's ticketing system. We have a storage environment with about 400,000 commodity disks attached(for this vendor-- there are other vendors too), in sites around the US and Canada. With a weekly failure rate of about 0.0005%, that means about 200 disks a week need to be replaced.
This work-- hardware investigation through storage appliance frontends, internal ticket creation, external ticket creation, watching the external ticket for updates to include in our internal ticket --was all manual, and for around 200 issues a week, it was done by one guy for two years. He was hopelessly behind. This is all automated now, and this morning, I pushed this automation from dev/test to production.
It feels great to see your work helping people around you.8 -
Spent 4 hours debugging a “button” styling, worked fine locally but not on production.!
After striking out “cache” issues, “browser” versions, “fonts”, “sass errors” the error was with a stupid chrome extension that appended a css class attribute to the “HTML” tag 😡
And the other developer thought that was a part of what was written in the code !!
Hate these kinda plugins that manipulate the DOM 😪
P S the plugin is "Grammarly".2 -
"It works on our end", the sentence that made me lose my shit.
I've been working on a project were we're supposed to integrate an API into our system.
When trying to get some user id's (UUID) from said API, we got a type-error in the response (???), so I called their integration support and asked what the fuck they were doing (not really, i was kinda calm at this point).
The answer I got was following:
Integration guy: "Uh, bro, like, I don't even know, it's probably on your end"
Me: "We literally used this endpoint with the same parameters yesterday, and got a result we expected. I noticed you updated your API this morning, did you make any major changes?"
Integration guy: "Yeah we changed the type of user id from string to number"
Me: "So, you changed the type of a UUID (uuid4) from string to number? How did you not think that would be an issue? I can see in your forums that everyone else is having the same issue."
Integration guy: "Nah, it's probably a bug in your code, it works on our end"
Me in my mind: *IT WORKS ON YOUR END?!? IT DOESN'T FUCKING MATTER IF IT WORKS ON YOUR END, FUCKTARD.*
What I actually said: "Uhm, I'm not sure if works on your end either, I'm not even sure how this change made it to production. But hey, thanks I guess, bye."
WHY AM I NOT ABLE TO YELL AT PEOPLE WHEN THEY ARE BEING RETARDED???
But really though, when you're maintaining an API, you shouldn't fucking care if things work on your end in your dev environment. What matters is how it works in production, for the end user/users.
And I know that 99% of cases it's the users fault by entering the wrong parameters or trying to request with wrongly setup auth and what not, but still.
Don't ASSUME nothing's wrong on your end. It's your fucking job to fix the issues.
And guess what? The problem was on their side.
I'm going fucking bald.2 -
It's cool.
I have less than four weeks before I have to have my project production ready
But why not make me waste my time in useless web based fucking trainings for bullshit HR issues I'll never deal with.
"Don't give gifts of more than $100 to potential clients!" Shut fuck up I don't work with clients.
Piss off and let me do my job.1 -
I was testing database migration and there was some issues which I couldn't fix them. So I drop the table just to test if the issues are still there.
"Why nothing have been changed?" After some minutes I realize it was production database which I had dropped 😓1 -
Don't hire monkeys that write shitty code that cause production issues.
Just spent the entire morning with our global team (10+ ppl) looking into the cause of a production issue.
Root cause: shitty code that anyone that has read an algorithm book (array resizing costs) and understanding how DB functions should be used and why (bulk inserts vs one at a time) would never write.
Even the code itself is a mess...8 -
Okay. So my dumbass boss took this project that had a steep timeline. I told him straight up, it won't work because we won't make the timeline. If we do this, I will be the one bending over backwards to deliver. I don't like to promise and fail. I got the oh don't worry let's just try. If we don't make it that's fine. Unfortunately that's not how I work. I refuse to deliberately fail. So I say okay and we begin. I suggested open source is the fastest way to deliver bit the fucked up part is, I am the only senior dev in the team. I will be expected to reverse engineer the open source app to connect our own deployment parameters. Use tech I have never used before. Connect frontend and backend. Handle dns bullshit. I have literally been working on Vibes and coffee for the past two weeks because ofcourse I ran into so many issues. Now I have an extension for Monday and I hate to fail. So I am not sleeping or resting just working on a fucking java app I didnt build and I am expected to make it work seemlessly on our production environment. I made some progress. Deployed frontend, deployed backend. Forgot to connect production dB so I decided to go with azure database for mysql driver since we have credits on azure. Now my java app is pissing itself over ssl handshake. I generate my keystore and add it and now java socket just times out. I want to pummel somebody or a punching bag that looks like my boss.15
-
A bit of backstory...
I have been the sole dev at my organization for awhile now (other two left for other jobs), so I have been maintaing and writing new code to support the business.
Our company was recently acquired by a larger entity and it has been very strange so far.
1. It has taken 5 weeks to acquire local admin rights on my own machine (I work remote) as well as a visual studio license.
2. We have known for a few weeks now we are getting a jr dev who will need the SAME procedures done on his machine/account and it has been two weeks now and nothing has been done. (Tickets have been put it - the issues have been escalated etc etc)
3. All of our code from our old company is in Azure Devops (which is connected to Azure AD) for some reason I haven't been able to add an external account (for my new account and org) to move the code elsewhere. I don't have the authority (I don't think) to place all of our code in a new location (GitHub,GitLab, self hosted solutions, etc)
4. All of our production VMs are billed through our old org located in Azure, so eventually that bill will stop being paid since we transitioned - I've brought this up to my manager (more non technical) who wasn't terribly worried about it.
5. I'm feeling slightly unfulfilled in this position. Earlier in my time here it was new and exciting, but there isn't much direction, not many goals, or interesting problems to solve.
Just wanted to express some issues that had been going on. Feel free to add ant feedback of suggestions 😄3 -
Around 11:30pm I honestly thought I was about done, but a series of production issues has kept me up until 4:30am. Solving one revealed another, and so on.
Finally, all is quiet.
Goodnight World.4 -
Never buy crappy, consumer-grade SSDs for use in production servers/RAIDs. This might sound obvious but at the company I used to work for, through a series of bad decisions by management and cheapness, we ended up with the cheapest consumer SSDs you can imagine powering all of our storage.
This turned into a nightmare spanning years of failed hard drives and a continues cycle of ridiculousness. Drive failed after a few days, gets taken out, sent back to manufacturer and then replaced with another equally crappy drive destined to fail within days/weeks.
Our ops people were going to the data center multiple times per week to replace failed drives. Lesson I learned: cheaping out on system-critical hardware and software can have long standing consequences and in the end usually doesn't end up actually saving money when you account for time employees have to spend dealing with issues that result from it. -
When a PR merged to improve functionality breaks the functionality 🥳 and now I have to fix the FIX to fix it.2
-
Discovered pro tip of my life :
Never trust your code
Achievements unlocked :
Successfully running C++ GPU accelerated offscreen rendering engine with texture loading code having faulty validation bug over a year on production for more than 1.5M daily Android active users without any issues.
History : Recently I was writing a new rendering engineering that uses our GPU pipeline engine.. and our prototype android app benchmark test always fails with black rendering frame detection assertion.
Practice:
Spend more than a month to debug a GPU pipeline system based on directed acyclic graph based rendering algorithm.
New abilities added :
Able to debug OpenGL ES code on Android using print statement placed in source code using binary search.
But why?
I was aware of the issue over a month and just ignored it thinking it's a driver bug in my android device.. but when the api was used by one of Android dev, he reported the same issue. In the same day at night 2:59AM ....
Satan came to me and told me that " ok listen man, here is what I am gonna do with you today, your new code will be going production in a week, and the renderer will give you just one black frame after random time, and after today 3AM, your code will not show GL Errors if you debug or trace. Buhahahaha ahhaha haahha..... Puffff"
And he was gone..
Thanks satan for not killing me.. I will not trust stable production code anymore enevn though every line is documented and peer reviewed. -
Every single stakeholder in my company tells me that I should be working on something different, every time I talk to them. For example - we've got some issues, that I've ranted on previously. I go to my manager, and tell him that it's going to take longer than I'd hoped, because the author of this part of the codebase wasn't familiar with functional programming or OOP, didn't document anything, and just generally produced an unmaintainable, borderline indescribable mess. The next guy after him made it all so much worse, because they're both a couple of tryhard douchebags, and I hope they fucking die. For real. I hope fire ants are involved.
Anyway, getting carried away there, whew. So I tell my manager that we'd be further ahead just replacing the code, because it's only doing a couple of things, and should not be so complex. He says "cool, but what you really need to be doing is rebuilding this other thing." So I switch gears and work on that other thing until I hit a point that requires the input of another stakeholder. I go to talk to this guy, and all hell breaks loose "why are you working on that, this is higher priority", and I explain the sequence of events. Manager denies having said what he said, I look like an asshole, yet again. Then the old "this should be simple, just change this" from the dudes who don't know code, and don't want to know. I try to explain, offer to show them precisely why their "simple ask" is anything but, but they just start screaming about how they hate technology. Yeah, well me fucking too. I keep hearing about how much "job security" I have, but man I'm going to lose my mind at this rate. I have seventeen motherfucking things that are "emergencies", and as many fucking dumb ass unintuitive workflows to go through to get them changed. All on production, because this place is fucking stupid. Just let me discard this shitty legacy code and be done with it already. FUCK.
Thank fucking fuck it's friday. In about six, seven hours, my goal is to be so fucking wasted that I can't feel my face. Get drunk, play with the dog, install a new distro on the desktop, maybe play a little guitar (the guitar is normal sized. It's not a ukulele or anything). Perfect friday night.9 -
I really think there should be a subject in every CS course to teach us how to handle/work-under Grade-A assholes and dumbfucks. Not that it would help, but atleast warn us on what we are getting into.
In my opinion, development is not *that* hard or frustrating but is made so by these shitty people. But again, what do I know.
I was scolded by my boss for using for-loop to iterate through an array recently. Apparently for-loop is not used in real world projects and this iteration should be done "in-memory". My colleagues and I are still trying to understand and process that.
I was asked to add fitbit integration to a project within 2 hours just because I had "already done it a week ago" in *another* project. Luckily, it was then given to a "senior" developer who took 4 days for it and essentially copy-pasted my work without much changes, ofcourse it stopped working every now and then.
I am given unreal deadlines on my tasks, on technologies I haven't worked on before, and then expected to churn out production ready code with no bugs in them.
My boss literally just sends me the links of 1st three google results on the problems I encounter and report, after humiliating me ofcourse. Yes, I did google it and yes I went through all I could find from Google forums to GitHub issues. When the library/plugin author himself says that this feature is not yet available, don't expect me to develop it in 2 hours you dumbfuck.
And for the love of God, please stop changing the data model every single day and justify it with agile development. Think before making any changes to it. Ever heard of Join queries? Foreign keys? Or any other basic database concepts.
We reached a point where each branch in the repo had different data model. Not kidding. And we were a team of just 4 developers. Atleast inform us when you change models after discussing it with your shit for knowledge "senior" developer, so we don't have to redo it all over again. The channels on slack are not for sharing random articles only.
I am just waiting to complete my year here.
I should have known what I got myself into the day he asked me to remove the comments I had added to explain what my code does. Why you ask? Because "we don't write comments". -
developer makes a "missed-a-semicolon"-kind of mistake that brings your non-production infrastructure down.
manager goes crazy. rallies the whole team into a meeting to find "whom to hold accountable for this stupid mistake" ( read : whom should I blame? ).
spend 1-hour to investigate the problem. send out another developer to fix the problem.
... continue digging ...
( with every step in the software development lifecycle handbook; the only step missing was to pull the handbook itself out )
finds that the developer followed the development process well ( no hoops jumped ).
the error was missed during the code review because the reviewer didn't actually "review" the code, but reported that they had "reviewed and merged" the code
get asked why we're all spending time trying to fix a problem that occurred in a non-production environment. apparently, now it is about figuring out the root cause so that it doesn't happen in production.
we're ALL now staring at the SAME pull request. now the manager is suddenly more mad because the developer used brackets to indicate the pseudo-path where the change occurred.
"WHY WOULD YOU WASTE 30-SECONDS PUTTING ALL THOSE BRACES? YOU'RE ALREADY ON A BRANCH!"
PS : the reason I didn't quote any of the manager's words until the end was because they were screaming all along, so, I'd have to type in ALL CAPS-case. I'm a CAPS-case-hater by-default ( except for the singular use of "I" ( eye; indicating myself ) )
WTF? I mean, walk your temper off first ( I don't mean literally, right now; for now, consider it a figure of speech. I wish I could ask you to do it literally; but no, I'm not that much of a sadist just yet ). Then come back and decide what you actually want to be pissed about. Then think more; about whether you want to kill everyone else's productivity by rallying the entire team ( OK, I'm exaggerating, it's a small team of 4 people; excluding the manager ) to look at an issue that happened in a non-production environment.
At the end of the week, you're still going to come back and say we're behind schedule because we didn't get any work done.
Well, here's 4 hours of our time consumed away by you.
This manager also has a habit of saying, "getting on X's case". Even if it is a discussion ( and not a debate ). What is that supposed to mean? Did X commit such a grave crime that they need to be condemned to hell?
I miss my old organization where there was a strict no-blame policy. Their strategy was, "OK, we have an issue, let's fix it and move on."
I've gotten involved ( not caused it ) in even bigger issues ( like an almost-data-breach ) and nobody ever pointed a finger at another person.
Even though we all knew who caused the issue. Some even went beyond and defended the person. Like, "Them. No, that's not possible. They won't do such dumb mistakes. They're very thorough with their work."
No one even talked about the person behind their back either ( at least I wasn't involved in any such conversation ). Even later, after the whole issue had settled down. I don't think people brought it up later either ( though it was kind of a hush-hush need-to-know event )
Now I realize the other unsaid-advantage of the no-blame policy. You don't lose 4 hours of your so-called "quarantine productivity". We're already short on productivity. Please don't add anymore. 🙏11 -
I was reluctant to try out flutter earlier on because of claims online stating that hybrid frameworks aren't there yet. That's one hell of a crap!
I fell in love with flutter after completing my first flutter app. Shit was just too easy. So many helpful libraries which has eased my overall workload lately.
We built a Native Android app which took 2months+ to complete and I just finished porting it to flutter for iOS and Android in 3 weeks. Boss was happy, Client was happy, I am freaking joyous, everybody is happy!
From the mouth of a Native Android Dev with over 5yr of exp. This shit called flutter is worthy of all the hype. I fucking kid you not!
I don't know about the past... I assume it was shitty then cus I also blasted it based on git issues but now it seems even more faster to build production worthy apps than anything I've encountered.4 -
Typescript seems like such overkill, but then you need to refactor your code and hit a bunch of issues in production. I don't think I'll ever go without typescript again. Fuck dynamic typing, it doesn't scale5
-
Me: Why are we spending time building reports for Support? AFAIK they never read or use them.
Boss: Seems they expect you to do it.
Me: Then what exactly are they supposed to be doing? All the issues seem to just escalate back to us.... We should just make sure issues never get into PROD
Boss: I agree, they're always firefighting... we gave them more funding so eventually they should catch up **Me: I highly doubt it, you should just stop hiring monkeys** esp. if we prevent PROD issues.
Me: Yea... we should prevent production issues because someone always has to pay off the (technical) debt and interest rates in PROD are very high6 -
Ok, here goes...
I was once asked to evaluate upgrade options for an online shop platform.
The thing was built on Zend 1, but that's not the problem.
The geniuses that worked on it before didn't have any clue about best practices, framework convention, modular thinking, testing, security issues...nothing!
There were some instances when querying was done using a rudimentary excuse for a model layer. Other times, they would just use raw queries and just ignore the previous method. Sometimes the database calls were made in strange function calls inside randomly loaded PHP files from different folders from all over the place. Sometimes they used JOINs to get the data from multiple tables, sometimes they would do a bunch of single table queries and just loop every data set to format it using multiple for loops.
And, best of all, there were some parts of the app that would just ignore any ideea of frameworks, conventions and all that and would be just a huge PHP file full of spagetti code just spalshed around, sometimes with no apparent logic to it. Queries, processing, HTML...everything crammed in one file...
The most amazing thing was that this code base somehow managed to function in production for more than 5 years and people actualy used it...
Imagine the reaction I got from the client the moment I said we should burn it to the ground and rebuild the whole thing from scratch...
Good thing my boss trusted me and backed me up (he is a great guy by the way) and we never had to go along with that Frankenstein monster... -
I don't understand how my managers suddenly forgot that my "down weeks" we're due to technical debt I inherited. The whole on boarding hasn't been in my favor. I've stayed at work everyday til long after work hours, digging through code, trying to get JIRA tickets done, encountering issues specific to our code base that no one would ever discover on their own without docs/help from the original dev. The whole time, I was told that they know what's going on and apologize. I constantly expressed that plenty of what we were doing was building on antipatterns. They acknowledged. When a ticket wasn't done, they always knew the very specific reason and I wasn't faulted. 6 months in, I receive a great annual review. 7 months in? I receive an email titled "Performance Discussion," detailing 4 of those incidents where a ticket was pushed back -- with inaccurate depictions of what actually went down. They actually wrote that I didn't communicate. One part of the report expressed that there were "bugs found in production due to inadequate test coverage." WTF!! Everything made it past code review and QA. What are you talking about?? In fact, the person who wrote that merged my code in each time!!!! Insane!! Anyway, Q2 is partly about cleaning up technical debt, which is a responsibility I have been vested (fantastic). I've deleted about 800 lines of code in the last 2 weeks and added plenty of doc strings. Two of the most important modules our application works from are about 1000 lines of JavaScript each without any comments/docs. I'm changing that, but I don't know if my managers truly know the significance. Someone was recently promoted to my position but manually wrote out a sorting algorithm (specified numeric indexes and all); didn't do shit to earn it but breathe. And while they get more and more praise and responsibility, I'm over here stuck trying to prove myself and live up to why I assume they hired me. It's ridiculous. I love the company, but I'm not getting any sleep and I'm stressed out. It's only been about 7 months and I've been doing everything I can. Why is this happening? What am I doing wrong? I've been developing a recurring (physical) headache and ticks. My heart/chest area sometimes feels like it's lifting weights. I sound like an idiot, pushing so hard for a company that isn't mine, but I take so much pride in being in this position, and I'm so set on proving myself this early in my career (I'm 25).8
-
I've been working for the last 5 years on some large legacy code used in production, more than 100K LOC, poor comments (when existing) often outdated, huge parts of code that can no longer be reached, over-engineered class hierarchy, functions of thousands lines, huge parts of deprecated code that cannot be removed because "someone might still be using it". Statistically, every small change caused 3 new issues somewhere else and every bug fix or new feature required 10 times the time that would be necessary with a decent codebase. But after five years in hell I can finally say that... Oh wait, nothing changed, the code is still legacy and nobody is going to do anything about that.1
-
If they followed my suggestion and went straight to debugging the server issues they would have been solved it from week 1 and everyone would have thought the migration had a minor performance hiccup. In fact, we have already done such at least twice before and nobody batted an eye.
Instead they self-labelled the migration a failure on first error, setting the stage for apologizing to the client, and put themselves on the spot for a whole staging / production signoff, replication / backup worfklow, almost a blue-green "seamless" deployment reminiscent of DigitalOcean.
Well they're not DigitalOcean, and anyone who has spent any time understanding users knows they will not participate in "new system" tests long enough to find or report issues.
So of course the migration stretched out to almost three months up until the whole reason for the migration - the rapidly escalating risk of the old provider disappearing - hit like a freight train and now they have to go through the problem of debugging the server like I told them to on week 1. Only this time they've set the client mindset against it, lost any chance of reverting, have had grave risk for data loss, and are under pressure to debug other people's code in real-time.
This is why I don't trust devs to do ops. A dev's first solution to any problem is to throw tech at it. -
Aren't you, software engineer, ashamed of being employed by Apple? How can you work for a company that lives and shit on the heads of millions of fellow developers like a giant tech leech?
Assuming you can find a sounding excuse for yourself, pretending its market's fault and not your shitty greed that lets you work for a company with incredibly malicious product, sales, marketing and support policies, how can you not feel your coders-pride being melted under BILLIONS of complains for whatever shitty product you have delivered for them?
Be it a web service that runs on 1980 servers with still the same stack (cough cough itunesconnect, membercenter, bug tracker, etc etc etc etc) incompatible with vast majority of modern browsers around (google at least sticks a "beta" close to it for a few years, it could work for a few decades for you);
be it your historical incapacity to build web UI;
be it the complete lack of any resemblance of valid documentation and lets not even mention manuals (oh you say that the "status" variable is "the status of the object"? no shit sherlock, thank you and no, a wwdc video is not a manual, i don't wanna hear 3 hours of bullshit to know that stupid workaround to a stupid uikit api you designed) for any API you have developed;
be it the predatory tactics on smaller companies (yeah its capitalism baby, whatever) and bending 90 degrees with giants like Amazon;
be it the closeness (christ, even your bugtracker is closed and we had to come up with openradar to share problems that you would anyway ignore for decades);
be it a desktop ui api that is so old and unmaintained and so shitty, but so shitty, that you made that cancer of electron a de facto standard for mainstream software on macos;
be it a IDE that i am disgusted to even name, xcrap, that has literally millions of complains for the same millions of issues you dont even care to answer to or even less try to justify;
be it that you dont disclose your long term plans and then pretend us to production-test and workaround-fix your shitty non-production ready useless new OS features;
be it that a nervous breakdown on a stupid little guy on the other side of the planet that happens to have paid to you dozens of thousands of euros (in mandatory licences and hardware) to actually let you take an indecent cut out of his revenues cos there is no other choice in a monopoly regime, matter zero to you;
Assuming all of these and much more:
How can you sleep at night with all the screams of the devs you are exploiting whispering in you mind? Are all the money your earn worth?
** As someone already told you elsewhere, HAVE SOME FUCKING PRIDE, shitty people AND WRITE THE FUCKING DOCS AND FIX THE FUCKING BUGS you lazy motherfuckers, your are paid more than 99.99% of people on earth, move your fucking greasy little fingers on that fucking keyboard. **
PT2: why the fuck did you remove the ESC key from your shitty keyboards you fuckshits? is it cos autocomplete is slower than me searching the correct name of a function on stackoverflow and hence ESC key is useless? at least your hardware colleagues had the decency of admitting their error and rolling back some of the uncountable "questionable "hardware design choices (cough cough ...magic mouse... cough golden charging cables not compatible with your own devices.. cough )?12 -
A coworker changed the application deployment process. He told all three of the other developers who need deployments, but not me. We sit six feet away from each other and I've run/managed deployments for a year longer than him.
His new process doesn't work and he's blaming the dev ops team for not following it. The new process clearly doesn't fit their workflow and never could have.
The lack of deployments have caused production issues and he still won't ping dev ops to remind them about the deployment because "it's not in the new workflow".
He's been painting dev ops as incompetent at the last three retrospectives without having ever personally reminded the deployment guy.
Ugggh. -
What do you tell interviewers as a "Senior developer" when they ask you what you do at your current job.
I've been with my current for almost 8 years, since graduating... Few different time but not very well managed (semi/barely agile). Hasn't really provided any skill growth opportunities. Mostly fixing production issues, chasing other teams.
The projects I've worked on are in many different languages either enhancements or some standalone stuff. But nothing that's huge and I don't think I've learned anything from them. I usually apply what I learn and practice outside of work to work.
So to me I can probably list a whole lot of projects but to me their not that amazing, I didn't learn anything from them.
Also about those algorithm questions. I've never used any of this stuff actually at work. Concepts yes but not how do you implement ... And honestly I've never once had a situation that required algorithmic thinking other than maybe writing recursive functions in rare occasions...
But to me I've never once done anything harder or new which I haven't already done on my own....
Sorry for the disorderly rambling this turned into... which is sorta a problem too.
Everytime I think about interviews, I want to give rants about we technical questions are BS, how I probably have enough real experience to tackle any problem and come up with a good plan/solution (in a realistic timeframe, not 20 minutes from design to implementation)2 -
Part? The whole damn thing.
We are supposed to be practicing agile, but I haven't seen a single sprint plan till date. All we do is solve issues that are reported from production/QA.
Nobody follows proper documentation, no reviews, no proper version control... Can't wait to finish my contract here and get the hell out.1 -
Well... I can think of several bugs that I found on a previous project, but one of the worst (if not the worst, because the damage scope) it's one bug that only appears for a couple of days at the end of every month.
What happens is the following: this bug occurs in a submodule designed (heh) to control the monthly production according the client requirements (client says "I want 1000 thoot picks", that submodule calculates the daily production requirements in order to full fill the order).
Ideally, that programming need to be done once a week (for the current month), because the quantities are updated by client on the same schedule, and one of the edge cases is that when the current date is >= 16th of the month, the user can start programming the production of the following month.
So, according to this specific case, there's an unidentified, elusive, and nasty bug that only shows up on the two last days of every month, when it doesn't allow to modify/create anything for the following month. I mean, normally, whenever you try to edit/create new data, the application shows either an estimated of the quantities to produce, or the previous saved data. But on those specific days it doesn't show any information at all, disregarding of there's something saved or not.
The worst thing is that such process involves both a very overcomplicated stored procedure, and an overcomplicated functionality on the client side (did I mentioned that it dynamically generates a pseudo-spreadsheet with the procedure dataset? Cell by cell), that absolutely no one really fully understands, and the dude that made those artifacts is no longer available (and by now, I'm not so sure that he even remember what he done there).
One of the worst thing is that at this point, it's easier to handle with that error rather to redesign all of that (not because technical limitations, but for bureaucratic and management issues).
The another worst thing (the most important none) is that this specific bug can create a HUGE mess as it prevents the programming of the production to be done the next day (you know, people tends to procrastinate and start doing things at the very end of the day/week/month)... And considering that the company could lose a huge amount of money by every minute without production, you can guess the damage scope of this single bug.
Anyway, this bug has existed since, I don't know, 2015 (Q4?) and we have tried so many things trying to solve it, but that spaghettis refuse to be understood (specially the stored procedure, as it has dynamically generated queries). During my tenure (that ended last year) I spent a good amount of time (considering what I mentioned on the last rant, about the toxic environment) trying to solve that, just giving up after the first couple of weeks.
Anyway... I'm guessing that this particular bug will survive another 4-ish years, or even outlive the current full development team... But, who knows ¯\_(ツ)_/¯ ? -
This week we had a live production issue that our staff were catching/fixing on the fly. We're a relatively small software team without any direct external customers, so this is not too unusual.
Unfortunately, the person in charge of dealing with these issues didn't resolve it during the work week, so we were stuck with it over the weekend. Said responsible employee left at 2:30 on Friday without figuring out how we'd deal with the problem without any staff in the office to intercept problem cases. Better yet, he drove all the way back, and was there from 3:30 to 4 and promptly left again without telling the rest of the team what was going on with the production issue. We asked how it happened, what it was, etc, but didn't focus on his fix (in hindsight, a mistake).
Since it's his job, I assumed that he would let us know what was up before he left on Friday. It turns out that he never addressed the production issue at all and just decided to leave.
A junior developer and I spent two hours contacting management (who, at this time are already at home with their families) to get clearance to either shut the system off or fix it. No one wants to give it and no one that's high enough up to approve the decision is available.
In the end, we asked the weekend mechanical support team (some friends of mine) to monitor the issue and they kindly accepted.
All of this could have been avoided if my coworker had either told us his plan earlier (so we could ask about the lack of coverage), gotten approval to shut it down for the weekend, or covered his own ass before he left for the day.
Ugggh. I get that we all make mistakes, but I really hope this guy shapes up soon. -
I'm genuinely shocked at the number of people I see on here bashing automated testing as a waste of time, simply because my entire career has taught me the opposite (and it's usually only non-technical managers I see who don't want to see "time wasted" writing tests.)
I'm also just as genuinely curious - what do you guys do instead? Just don't test and deal with production issues as they occur? Pass it off to a separate UAT / human-based testing department and let them sign it off? Assume that because you're using Haskell / some other discipline it'll work if it compiles?14 -
I hate dev politics...
PM: Hey there is a weird error happening when I upload this file on production, but it works on our test environments.
Me: After looking at this error, I don't find any issues with the code, but this variable is set when the application is first loaded, I bet it wasn't loaded correctly our last deployment and we just need to reload the application.
Senior Dev: We need to output all of the errors and figure out where this error is coming from. Dump out all the errors on everything in production!!
Me: That's dumb... the code works on test... it's not the code.. it's the application.
Senior dev: %$*^$>&÷^> $
Me: Hey I have an idea! If test works... I can go ahead and deploy last week's changes to prod and dump those errors you were talking about!!
Senior Dev: OK
Me: *runs Jenkins job the deploys the new code and restarts the application*
PM: YAY you fixed it!!
Senior Dev: Did you sump put those errors like I said.
Me: Nope didn't touch a thing... I just deployed my irrelevant changes to that error and reloaded the application.2 -
So, I've had a personal project going for a couple of years now. It's one of those "I think this could be the billion-dollar idea" things. But I suffer from the typical "it's not PERFECT, so let's start again!" mentality, and the "hmm, I'm not sure I like that technology choice, so let's start again!" mentality.
Or, at least, I DID until 3-4 months ago.
I made the decision that I was going to charge ahead with it even if I started having second thoughts along the way. But, at the same time, I made the decision that I was going to rely on as little external technology as possible. Simplicity was going to be the key guiding light and if I couldn't truly justify bringing a given technology into the mix, it'd stay out.
That means that when I built the front end, I would go with plain HTML/CSS/JS... you know, just like I did 20+ years ago... and when I built the back end, I'd minimize the libraries I used as much as possible (though I allowed myself a bit more flexibility on the back end because that seems to be where there's less issues generally). Similarly, any choice I made I wanted to have little to no additional tooling required.
So, given this is a webapp with a Node back-end, I had some decisions to make.
On the back end, I decided to go with Express. Previously, I had written all the server code myself from "first principles", so I effectively built my own version of Express in other words. And you know what? It worked fine! It wasn't particularly hard, the code wasn't especially bad, and it worked. So, I considered re-using that code from the previous iteration, but I ultimately decided that Express brings enough value - more specifically all the middleware available for it - to justify going with it. I also stuck with NeDB for my data storage needs since that was aces all along (though I did switch to nedb-promises instead of writing my own async/await wrapper around it as I had previously done).
What I DIDN'T do though is go with TypeScript. In previous versions, I had. And, hey, it worked fine. TS of course brings some value, but having to have a compile step in it goes against my "as little additional tooling as possible" mantra, and the value it brings I find to be dubious when there's just one developer. As it stands, my "tooling" amounts to a few very simple JS scripts run with NPM. It's very simple, and that was my big goal: simplicity.
On the front end, I of course had to choose a framework first. React is fine, Angular is horrid, Vue, Svelte, others are okay. But I didn't want to bother with any of that because I dislike the level of abstraction they bring. But I also didn't want to be building my own widget library. I've done that before and it takes a lot of time and effort to do it well. So, after looking at many different options, I settled on Webix. I'm a fan of that library because it has a JS-centric approach. There's no JSX-like intermediate format, no build step involved, it's just straight, simple JS, and it's powerful and looks pretty good. Perfect for my needs. For one specific capability I did allow myself to bring in AnimeJS and ThreeJS. That's it though, no other dependencies (well, at first, I was using Axios because it was comfortable, but I've since migrated to plain old fetch). And no Webpack, no bundling at all, in fact. I dynamically load resources, which effectively is code-splitting, and I have some NPM scripts to do minification for a production build, but otherwise the code that runs in the browser is what I actually wrote, unlike using a framework.
So, what's the point of this whole rant?
The point is that I've made more progress in these last few months than I did the previous several years, and the experience has been SO much better!
All the tools and dependencies we tend to use these days, by and large, I think get in the way. Oh, to be sure, they have their own benefits, I'm not denying that... but I'm not at all convinced those benefits outweighs the time lost configuring this tool or that, fixing breakages caused by dependency updates, dealing with obtuse errors spit out by code I didn't write, going from the code in the browser to the actual source code to get anywhere when debugging, parsing crappy documentation, and just generally having the project be so much more complex and difficult to reason about. It's cognitive overload.
I've been doing this professionaly for a LONG time, I've seen so many fads come and go. The one thing I think we've lost along the way is the idea that simplicity leads to the best outcomes, and simplicity doesn't automatically mean you write less code, doesn't mean you cede responsibility for various things to third parties. Those things aren't automatically bad, but they CAN be, and I think more than we realize. We get wrapped up in "what everyone else is doing", we don't stop to question the "best practices", we just blindly follow.
I'm done with that, and my project is better for it! -
At what point do you stop optimizing queries and realize it's a database architecture, scaling problem?
We've been having production issues this week because a lot more users with more demands, and I'm going we need more servers... We can't just have one db, we need to parallelize like Hadoop...
Everyone else is going, how do we optimize queries, indexes, reduce the load...11 -
I'm considering quitting a job I started a few weeks ago. I'll probably try to find other work first I suppose.
I'm UK based and this is the 6th programming/DevOps role I've had and I've never seen a team that is so utterly opposed to change. This is the largest company I've worked for in a full time capacity so someone please tell me if I'm going to see the same things at other companies of similar sizes (1000 employees). Or even tell me if I'm just being too opinionated and that I simply have different priorities than others I'm working with. The only upside so far is that at least 90% of the people I've been speaking to are very friendly and aren't outwardly toxic.
My first week, I explained during the daily stand up how I had been updating the readmes of a couple of code bases as I set them up locally, updated docker files to fix a few issues, made missing env files, and I didn't mention that I had also started a soon to be very long list of major problems in the code bases. 30 minutes later I get a call from the team lead saying he'd had complaints from another dev about the changes I'd spoke about making to their work. I was told to stash my changes for a few weeks at least and not to bother committing them.
Since then I've found out that even if I had wanted to, I wouldn't have been allowed to merge in my changes. Sprints are 2 weeks long, and are planned several sprints ahead. Trying to get any tickets planned in so far has been a brick wall, and it's clear management only cares about features.
Weirdly enough but not unsurprisingly I've heard loads of complaints about the slow turn around of the dev team to get out anything, be it bug fixes or features. It's weird because when I pointed out that there's currently no centralised logging or an error management platform like bugsnag, there was zero interest. I wrote a 4 page report on the benefits and how it would help the dev team to get away from fire fighting and these hidden issues they keep running into. But I was told that it would have to be planned for next year's work, as this year everything is already planned and there's no space in the budget for the roughly $20 a month a standard bugsnag plan would take.
The reason I even had time to write up such a report is because I get given work that takes 30 minutes and I'm seemingly expected to take several days to do it. I tried asking for more work at the start but I could tell the lead was busy and was frankly just annoyed that he was having to find me work within the narrow confines of what's planned for the sprint.
So I tried to keep busy with a load of code reviews and writing reports on road mapping out how we could improve various things. It's still not much to do though. And hey when I brought up actually implementing psr12 coding standards, there currently aren't any standards and the code bases even use a mix of spaces and tab indentation in the same file, I seemingly got a positive impression at the only senior developer meeting I've been to so far. However when I wrote up a confluence doc on setting up psr12 code sniffing in the various IDEs everyone uses, and mentioned it in a daily stand up, I once again got kickback and a talking to.
It's pretty clear that they'd like me to sit down, do my assigned work, and otherwise try to look busy. While continuing with their terrible practices.
After today I think I'll have to stop trying to do code reviews too as it's clear they don't actually want code to be reviewed. A junior dev who only started writing code last year had written probably the single worst pull request I've ever seen. However it's still a perfectly reasonable thing, they're junior and that's what code reviews are for. So I went through file by file and gently suggested a cleaner or safer way to achieve things, or in a couple of the worst cases I suggested that they bring up a refactor ticket to be made as the code base was trapping them in shocking practices. I'm talking html in strings being concatenated in a class. Database migrations that use hard coded IDs from production data. Database queries that again quote arbitrary production IDs. A mix of tabs and spaces in the same file. Indentation being way off. Etc, the list goes on.
Well of course I get massive kickback from that too, not just from the team lead who they complained to but the junior was incredibly rude and basically told me to shut up because this was how it was done in this code base. For the last 2 days it's been a bit of a back and forth of me at least trying to get the guy to fix the formatting issues, and my lead has messaged me multiple times asking if it can go through code review to QA yet. I don't know why they even bother with code reviews at this point.18 -
I have had it with the new pm for my project. We've had to literally write down rules for him to follow on how we want to run our project.
- we always go over time on standup because he's asking everyone specific questions about each ticket we're working on instead of just letting us give our updates. Wrote a rule for not doing that.
- he gets overly excited to get things done to the point that he's approved a PR himself and merged a production release during off hours.
- we're a team of 3 devs but he has 7 big items he wants us to work on concurrently.
- we use jira to keep track of things but he insists on us updating a spread sheet that he made as well. We just straight up told him no but he'll bring it 3 times every week.
- he wants us to write a report of our daily progress in our jira ticket before we give the same updates on standup
- every time I give him an estimate, he tells the clients it's a promise we'll get it done by x date.
- he never pushes back and says yes to everything
- oh here's a funny one, he's "reviewed" my code several times
Idk what to do here anymore. We've literally talked to him about all of these issues. He'll change for a few days and go back to doing whatever he does.6 -
We are 3-4 days away from deployment to production. We are still bug fixing. But one coworkers decided this is the time to make a fuss about the way everything is set up. He doesn't like the dev database. So he knocks it over.. and while so doing it, he doesn't inform the team. And when I ask if something else is gonna knock over? No answer! (And something broke down too..)
Now we have issues to test our bugfixes. The whole thing took me half a day finding out and made me distracted with frustration, and not just for me. Most bugs could've been done in that half a day!
I so wanna punch the guy xD but no, I gotta save face, pfff!2 -
Why things are fucking hard when you're not too good and not too bad at work. I'm like normal dev just throw things at me give me any task any framework I will learn it, I will solve production issues, I will help my co-workers to get their shit done even my JIRA is clean but it feels like I'm going nowhere. I'm like an average guy who knows many things other than normal guys or devs (by considering I'm junior and the people who are working with me).
I'm feeling like I'm in a fucking loop, where every day is same.
Is there anything I can do? which will make me feel little better?
I think every guy on earth have some innovative ideas even I have some(of course some of them are implemented already even they are kinda same, even some ideas are totally new, some are not possible, some requires much knowledge of certain field). But by just having an awesome idea doesn't change anything.
Maybe I'm not trying hard, there are several other reasons which are coming in my way but of course, I shouldn't tell any reasons. -
We use at our company one of the largest Python ORM and dont code ourselfs on it, event tough I can code. Its some special contract which our General Manager made, before we as Devs where in the Project and everything is provided from the external Company as Service. The Servers are in our own Datacenter, but we dont have access.
We have our Consultants (Project Manager) as payd hires and they got their own Devs.
Im in lead of Code Reviews and Interfaces. Also Im in the "Run" Team, which observes, debuggs and keeps the System alive as 3rd-Level (Application Managers).
What Im trying to achieve is going away from legacy .csv/sftp connections to RestAPI and on large Datasets GraphQL. Before I was on the Project, they build really crappy Interfaces.
Before I joined the Project in my Company, I was a Dev for a couple of Finance Applications and Webservices, where I also did coding on Business critical Applications with high demand Scaling.
So forth, I was moved by my Boss over to the Project because it wasn't doing so well and they needed our own Devs on it.
Alot of Issues/Mistakes I identified in the Software:
- Lots of Code Bugs
- Missing Process Logic
- No Lifecycle
- Very fast growing Database
- A lot of Bad Practices
Since my switch I fixed alot of bugs, was the man of the hour for fixing major Incidents and so on so forth. A lot of improvements have been made. Also the Team Spirit of 15+ People inside the Project became better, because they could consult me for solutions/problems.
But damn I hate our Consultants. We pay them and I need to sketch the concepts, they are to dumb for it. They dont understand Rest or APIs in general, I need to teach them alot about Best Practices and how to Code an API. Then they question everything and bring out a crooked flawed prototype back to me.
WE F* PAY THEM FOR BULLCRAP! THEY DONT EVEN WRITE DOCUMENTATION, THEY ARE SO LAZY!
I even had a Meeting with the main Consultant about Performance Problems and how we should approach it from a technical side and Process side. The Software is Core Business relevant and its running over 3 Years. He just argumented around the Problem and didnt provide solutions.
I confronted our General Manager a couple of times with this, but since 3 Years its going on and on.
Im happy with my Team and Boss, they have my back and I love my Job, but dealing with these Nutjobs of Consultants is draining my nerves/energy.
Im really am at my wits end how to deal with this anymore? Been pulling trough since 1 year. I wanna stay at my company because everything else besides the Nutjob Consultants is great.
I told my Boss about it a couple of times and she agrees with me, but the General Manager doesnt let go of these Consultants.
Even when they fuck up hard and crash production, they fucking Bill us... It's their fault :(3 -
I have a web app that is currently running on a production server with no issues, but at the same time, it isn't working on my machine which I used to write, test, and deploy the app. The thing is, I haven't touched the code for a full month.
Now, I know this has to be logical and that there must be an reasonable explanation for all of this that I do not know yet.
However, and out of frustration, my mind wants to believe that there's some sorcery involved here or that a cosmic ray has actually penetrated the machine and messed its registers.
Damn the cosmic ray!3 -
CoWorker: so when are you going to be out?
Me: taking 1 week off Oct 1. I need a break from production issues and all these critical tasks...
CW: ah OK yea, you deserve it. So where you going?
Me: well I'm planning to just stay home unless the weather is really nice. I'm going to try React Native to build a mobile app and maybe look for some open source projects... O yes gotta look into my investments too...
CW: Oohhhh... Ok.... (We go on talking about Trump and why somehow the markets haven't crashed yet...)2 -
I HATE the idea of only releasing on pre-determined schedules despite work being completed and just waiting for that day to arrive.
I'm a co-founder of a small software company. We have partnered with another particular company that also writes software. Some of our clients have access to paid content of that company's services through our application.
Every once in a while, our clients will report issues with that company's service to us, because they access it through our application. They think it's our issue.
We then pass the report on to the partner company, telling them that their stuff is broken. Their reply goes like this:
"Ok. We'll get the bug fix scheduled, and we'll release it next Thursday."
"Next Thursday? The issue is now, they can't use the service."
"That's our scheduled release date."
O.M.G.
We voluntarily walked away from our safe, cushy jobs working for other people, taking enormous pay cuts to start this company. Now, we're 6+ years in, disrupting established fat-and-happy competitors in this space. I GUARANTEE you that if we had that same attitude, we would have been absolutely obliterated early on.
We are quick. Guided by kanban boards, our suite of unit tests and integration tests is vast and kick-ass. With continuous integration and the click of a button we know if we broke something or if the piece we're working on is ready to be pushed to production, IMMEDIATELY. Our "release schedule" is when the damn thing is complete.
It isn't all bad. Our integration with them has been beneficial for both of us. I just loathe their snail's pace which negatively affects our mutual customers. It can make us look bad, and we can do nothing about it.
Blah.3 -
Integrating Google recaptcha into my web service. For some reason it always errors, both on a production and development environment, correct domains configured, and with he simplest setup. I'm fucking lost, documentation assumes it actually works. Similar errors on stack overflow and Google groups either got no answers or have obvious issues.
Fuck this man4 -
Even seniors make mistakes. In case you were ever doubting yourself - just remember that.
I just had a very senior level programmer on my staff add a function to a production system that issues an SQL UPDATE query without a WHERE clause. Fortunately, only the 1st entry succeeded and the rest failed due to "duplicate record" errors. Clearly he had intended to do a SELECT to check if an entry was present. If it was present, do an UPDATE, otherwise do an INSERT (think UPSERT - but done manually). However instead in the insert part they were both UPDATE's. The first update was normal looking but the second UPDATE was just this weird malformed-looking thing where he tried to do an UPDATE but to every field including the key fields. Clearly he was thinking about an insert but actually writing it as an update. Every now and then I need to remind myself that these things happen. The guy's not dumb - just made a mistake.
I'm just happy it "failed unsuccessfully".4 -
Get an issue reported today stating that a report can't be submitted in a product we maintain. Taking their word for it I start investigating on our local copy of the product. Everything works as intended. Ok... strange. Take a look at the production copy aaaaand it was submitted. No issues at all.
Note to self. Don't believe the client. -
TL;DR Shit programer trying pass off stealing code as "Recycling"
Backstory:
Client hires senior dev. He lied and knows nothing. Has been causing havoc in production since day 1. My crusades to defend production have been without much success.
Since he wants to LITERALLY put his name on every big project, he finds any reason to make a new version of it (or make a slight astetic modification) to say he did something.
The client doesn't know or care about the programming side of things. Which means it is incredibly difficult to get him to understand the issues this brings. Not to mention that the "senior dev" is acting as a consultant to the client, altering the facts.
Story:
The piece of shit, is trying to make a new version of a big project. It was originally made by my mentor. Again, if you are using someone else's work to complete your own, I don't care. But if you take 99% of another person's work and then say...
"I took and existing project, which was similar to what I'm trying to make. Then I modified it to fit our needs."
Fuck you man!
You took someone else's work. Now you're trying to present it as your own. No references to our team. Again, there is literally nothing new about this project. It's exactly like the original. The client didn't even ask for this.3 -
(Part 1/2?)
Ohhh my god am I furious and this one's a gem.
Also I'm gonna namespoil all of the entities in my post. If this is against rant rules I'll reframe it.
So the story starts over an year ago. Me, being in a bad place, where I couldn't do a job due to external issues, wanted to try out an internship. Thought I could pull off a 5 hour shift and then attend to my problems.
THE INTERNSHALA ARC:
I apply to a bunch of applications on Angel, Internshala and Indeed.
I was contacted by a few handful of these places. One of them was called "ARCHITECTA SOFTWARE SOLUTIONS". These guys had arranged an online aptitude test for me which I promptly took.
I looked up this company and they seemed like a pretty okay big firm from the outset but didn't have many reviews on Glassdoor and likes of such. (first red flag). Post aptitude test, I was quite sure I fucked up and wouldn't get further contact. Surprisingly, a person from the company sends me his Whatsapp number over chat and asks me to save it. The message is worded like a bulk email (Starting with Hello everyone!!) which I thought was quite odd since the interaction from these platforms has always been a person-to-person contact for me. Since Internshala showed that only around 40 people applied for the position I was quite intrigued but attributed this to my lack of exp in internship operations.
THE WHATSAPP ARC:
I was contacted by the number on WhatsApp saying that they'd be interested in moving forward and I gave them my work experience details.
The person sends me over a development assignment to complete within a few days. The assignment consists of massive scope of details. I'm talking production level concept and implementation. Asks to me implement a custom emotion detection CV model (worded as "emotion camera" lmao), generate a 3d model (specified nowhere and expects to implement a mono-ocular system for the curious) and deploy it over AWS with a website to go along with it and also host that. The website should contain a VR ("360 rotatable") view that can explore the depth-map ("not worded as depth-map") of the face. My first assumption was that they had picked this work up for outsourcing and didn't bother to chip off parts so as to create an assignment out of it (I know very optimistic).
So I shoot it at him on WhatsApp asking which parts of the assignment should I do?
Him: So, which parts CAN you do?
I thought of it as an HR thing.
Me: I could do most of it but given the time-frame of the assignment and my applied position as a web developer it is perhaps out of scope for my application.
Him: Don't worry about the assignment. You can submit when you complete the whole assignment.
I was visibly angry over the stupidity of this man.
Me: This task is a Full-Stack + CV + VR task. It will take over two months to get working. Am I supposed to work on it for that long for an assignment?
Him: Okay just do the basic functionalities like add to cart. But also try to do the camera thing before next week.
At this point I'm sure that they are having trouble handling an eager client and they're offloading work to interns. So I do only the backend and minimal frontend and submit the assignment (a 2 day job done over a weekend).
Nothing. Empty. No messages since then. I tried sending in a Whatsapp message on the application and how to proceed. Then, if I could get to know if I have been rejected. Nothing.
And all this time I can clearly see the account is active as it pushes pretentious motivational quotes over it's Whatsapp status.3 -
When the CTO/CEO of your "startup" is always AFK and it takes weeks to get anything approved by them (or even secure a meeting with them) and they have almost-exclusive access to production and the admin account for all third party services.
Want to create a new messaging channel? Too bad! What about a new repository for that cool idea you had, or that new microservice you're expected to build. Expect to be blocked for at least a week.
When they also hold themselves solely responsible for security and operations, they've built their own proprietary framework that handles all the authentication, database models and microservice communications.
Speaking of which, there's more than six microservices per developer!
Oh there's a bug or limitation in the framework? Too bad. It's a black box that nobody else in the company can touch. Good luck with the two week lead time on getting anything changed there. Oh and there's no dedicated issue tracker. Have you heard of email?
When the systems and processes in place were designed for "consistency" and "scalability" in mind you can be certain that everything is consistently broken at scale. Each microservice offers:
1. Anemic & non-idempotent CRUD APIs (Can't believe it's not a Database Table™) because the consumer should do all the work.
2. Race Conditions, because transactions are "not portable" (but not to worry, all the code is written as if it were running single threaded on a single machine).
3. Fault Intolerance, just a single failure in a chain of layered microservice calls will leave the requested operation in a partially applied and corrupted state. Ger ready for manual intervention.
4. Completely Redundant Documentation, our web documentation is automatically generated and is always of the form //[FieldName] of the [ObjectName].
5. Happy Path Support, only the intended use cases and fields work, we added a bunch of others because YouAreGoingToNeedIt™ but it won't work when you do need it. The only record of this happy path is the code itself.
Consider this, you're been building a new microservice, you've carefully followed all the unwritten highly specific technical implementation standards enforced by the CTO/CEO (that your aware of). You've decided to write some unit tests, well um.. didn't you know? There's nothing scalable and consistent about running the system locally! That's not built-in to the framework. So just use curl to test your service whilst it is deployed or connected to the development environment. Then you can open a PR and once it has been approved it will be included in the next full deployment (at least a week later).
Most new 'services' feel like the are about one to five days of writing straightforward code followed by weeks to months of integration hell, testing and blocked dependencies.
When confronted/advised about these issues the response from the CTO/CEO
varies:
(A) "yes but it's an edge case, the cloud is highly available and reliable, our software doesn't crash frequently".
(B) "yes, that's why I'm thinking about adding [idempotency] to the framework to address that when I'm not so busy" two weeks go by...
(C) "yes, but we are still doing better than all of our competitors".
(D) "oh, but you can just [highly specific sequence of undocumented steps, that probably won't work when you try it].
(E) "yes, let's setup a meeting to go through this in more detail" *doesn't show up to the meeting*.
(F) "oh, but our customers are really happy with our level of [Documentation]".
Sometimes it can feel like a bit of a cult, as all of the project managers (and some of the developers) see the CTO/CEO as a sort of 'programming god' because they are never blocked on anything they work on, they're able to bypass all the limitations and obstacles they've placed in front of the 'ordinary' developers.
There's been several instances where the CTO/CEO will suddenly make widespread changes to the codebase (to enforce some 'standard') without having to go through the same review process as everybody else, these changes will usually break something like the automatic build process or something in the dev environment and its up to the developers to pick up the pieces. I think developers find it intimidating to identify issues in the CTO/CEO's code because it's implicitly defined due to their status as the "gold standard".
It's certainly frustrating but I hope this story serves as a bit of a foil to those who wish they had a more technical CTO/CEO in their organisation. Does anybody else have a similar experience or is this situation an absolute one of a kind?2 -
Windows 10 insider preview had a critical bug like half a year ago where most browsers would freeze the PC. I've reported it multiple times but my feedback didnt get any attention. The bug made it to production months later...
Now, the story is repeating. I've discovered a bug. Everytime there's a software update for apps that with a built-in updater in the main .exe (JetBrains IDEs, VS etc.) the updater fails, program is corrupted and needs to be reinstalled. For instance, there's a minor bump in Android Studio version, if you try to use autoupdater it will corrupt your Android Studio and you'll have to reinstall.
Been trying to reach out to them but the only "real issues" that are highlighted are "no CPU temp in task manager" or "pls improve automatic problem resolving"
Why the fuck do you even have a feedback program if you're ignoring the people who are reaching out to you, you pieces of shit4 -
So, we are having a SaaS service for people where they can build X stuff. It is all fine as long as you are using basic things there, no complex cases and so on. Even on some complex - it does work just fine.
Here's the rant itself:
The production server throws us errors every 5-10 minutes that something broke and fails to do job X. At first we were all hands on deck fixing it ASAP to make it stable to later realise that most of these cases were users doing stupid shit. Then we began to fix the core issues rather than chasing every single issue there is (costs are important you know) - funny enough, we get few support requests a week and our 1h response time + 24h fix time usually buys us that customer and allows t o leave a great impression.
So all in all, bugles production is good but great support - is way better. Users can deal with issues especially if they are experimenting there but when they need answers - you'd better give it to them.1 -
My boss is being a stupid cunt. To give you a background we were facing issues with our Collections system. First week December 2019, I and a colleague of mine came up with a new efficient collections architecture. My colleague and I started to Code and create automation scripts mid December and completed it in First week of Jan 2020. This PoC version was supposed to be just between the Dev team(App Dev and Back end, also one from the Ops side to verify the data). I did not receive any feedback on the actual collections system and the data integrity but during this time all they’ve done is take meetings with no real outcome. I raised this and the only email I got is data is looking fine when I know it is not.Now in First week of Feb, he is stressing us to go ahead and deploy the architecture in Production and we have not done any Code Review, Static Code analysis, any real tests on Code and deployment scripts. Have not discussed any metrics for our dashboard and alerting. I have no idea how to handle this cunt. I have even asked for resources to atleast productionalize the code and move ahead the deployment and still no out come. I’ll go in a meeting with him in an hour, I will be very blunt and tell him that whatever he is doing is a foolish way and maybe resign in couple of weeks6
-
Why can't my team including my boss learn to stop making assumptions... And mixing seperate issues into one...
If there's a fucking production issue, first step is to reproduce it... AKA ask what the user did and what he expects....
Not...
User: hey we call this url and get an error
Dev: ok rollback -
The default font was different between development and production. The default font was never selected for the application. It has been this way for years. I found and fixed this. Now a huge amount of UI issues are just gone. People are like "Why was this never set?" Internally I am saying "Because they got lucky and never thought about it in the past."
Other people's code...3 -
I was working for a project with one of the project managers. Despite several discussions, he was not ready to have provisioned for procurement of couple of extra drives for database backups. Also because it's always how they worked, developers were allowed to make changes to the production databases directly.
Since I knew it was going to be burning some day, despite his negligence, I ran a script to take full database backups every night, compress, and remove old backups all to do in the drives we had on server. Sat it automated using scheduler.
One day it happened that one of the junior developers deleted one major table taking whole production down. Next thing you know everyone went crazy. Since I felt bad for the managers and users, I was able to restore database using backup from last night.
You know who jumped in first before senior management to take credit of all this and got some nice kudos..that project manager. Also, you know who got burned..it would not be a rant if I did not got schooled for not following on the wisdom of project manager.
Anyways, we are still not taking database backups (as per project manager) -
Ranting to a recruiter about what I am looking for:
A mature team where im not always fixing production issues and telling other how they should do things...
Guess recruiters are like free psychs... -
So I joined this digital agency where they are working on this ad-tech product and right from day one, I was given a task to implement a new feature on the product. No knowledge transfer. No onboarding process. So, I had given estimation about the task and apparently it took longer than expected. But what were they expecting. Anyways, my manager asked me to have a KT with the only senior guy that has been working there for last couple of years. And man, since the KT started, it's been hell for me. The guy is such an asshole and won't even give me a basic walkthrough of the system. He only took one call and that ended within 30 minutes. On top of that he went ahead and told the product manager that I am not keeping up and am not ready. And my product manager apparently wants me to take his place within a month. It's been only two months since I joined. I have already pushed two major features, tried to understand the system architecture, codebase and everything on my own. On top of that, I got yelled at by that senior dev in a meeting about a PR. I was quite confident guy when I joined and now I am anxious everyday at work and i am scared that they'll let me go because I won't be able to meet their unrealistic expectations. I also can't stand this senior dev and he can't stand me which makes me really demotivated to work. I have anxiety issues and now I am thinking if I stay, I am gonna mess up big time and they'll fire me or worse. I might break something in production because I didn't have proper onboarding.2
-
A developer couldn't get a application performance monitoring (APM) tool to trace his application. They claimed that their libraries and their configurations were alright and that the APM tool was non-performant.
The developer then argues with sysadmin that the APM tool can't trace the application and that there's nothing wrong with the application or the configurations. When sysadmin questions whether the developer got the tool to work anywhere, they say, "No" and head off to make it work at least in one place. They come back saying that it works on their development environment (which is their local machine). Sysadmin claims that the system configurations on the server instances cannot be matched by the development environment and there could be a lot more factors to be considered for the problem. The sysadmin asks to prove it on a server instance on one of the test environments and then they'd agree that it is a problem with the tool. They also argue that this is not the only application that uses the APM tool and the tool happily traces other applications with no issues.
The developer tries the same configuration on a staging instance and fails. In order to make it work, they silently uninstall the existing version of the APM tool and then compiles an unstable branch of the tool. It finally works with this version.
They go back to the sysadmin and show that it works on the staging environment, but does not on production. After banging their head on the wall for a while, the sysadmin figure that the tool had been swapped out for the unstable branch that was manually compiled. When questioned, the developer responds, "It works with this version on staging, so deploy the same version on production"
WTF? You don't deploy an unstable branch to production. Just because you can't make it work on the stable branch doesn't mean that it is the problem with the tool itself. There's a big difference between a stable branch and a non-stable branch. How would you feel if the sysadmin retorted by asking you to deploy the staging branch of your application to production? -
Long post, TLDR: Given a large team building large enterprise apps with many parts (mini-projects/processes), how do you reduce the bus-factor and the # of Brent's (Phoenix Project)?
# The detailed version #
We have a lot of people making changes, building in new processes to support new flows or changes in the requirements and data.
But we also have to support these except when it gets into Production there is little information to quickly understand:
- how it works
- what it does/supposed to do
- what the inputs and dependencies are
So often times, if there's an issue, I have to reverse engineer whatever logic I can find out of a huge mess.
I guess the saying goes: the only people that know how it works is whoever wrote it and God.
I'm a senior dev but i spend a lot of time digging thru source code and PROD issues to figure out why ... is broken and how to maybe fix it.
I think in Agile there's supposed to be artifacts during development but never seen em.
Personally whenever i work on a new project, I write down notes and create design diagrams so i can confirm things and have easy to use references while working.
I don't think anyone else does that. And afterwards, I don't have anywhere to put it/share it. There is no central repo for this stuff other than our Wiki but for the most part, is like a dumping ground. You have to dig for information and hoping there's something useful.
And when people leave, information is lost forever and well... we hire a lot of monkeys... so again I feel a lot of times i m trying to recover information from a corrupted hard drive...
The only way real information is transferred is thru word of mouth, special knowledge transfer sessions.
Ideally I would like anything that goes into PROD to have design docs as well as usage instructions in order for anyone to be able to quickly pick it up as needed but I'm not sure if that's realistic.
Even unit tests don't seem to help much as they just test specific functions but don't give much detail about how a whole process is supposed to work.9 -
I am just student looking for job, and got this pre interview test:
Develop an Android or iOS app with login and password input field, download button, place for image we prvided.
... reading further:
What we are looking for in the code ?
internal quality:
-consistent formatting of the source code
-clean, robust code without smells
-consistent abstractions and logical overall structure
-no cyclic dependencies
-code organized in meaningful layers
-low coupling and high cohesion
-descriptive and intention-revealing names of packages, classes, methods etc.
-single small functions that do one thing
-truly object-oriented design with proper encapsulation, sticking to DRY and SOLID principles, without procedural anti-patterns
-lots of bonus points for advanced techniques like design patterns, dependency injection, design by contract and especially unit (or even functional or integration) tests
external quality:
-the app should be fully functional, with every state, user input, boundary condition etc. taken care of (although this app is indeed very small, treat it as a part of big production-ready project)
-the app should correctly handle screen orientation changes, device resources and permissions, incoming calls, network connection issues, being pushed to the background, signing deal with the devil :D and other platform intricacies and should recover from these events gracefully
-lowest API level is not defined - use what you think is reasonable in these days
-bonus points if the app interacts with the user in an informative and helpful way
-bonus points for nice looks - use a clean, simple yet effective layout and design
... I mean really ? and they give me like 2 days ?4 -
This is more of an advice seeking rant. I've recently been promoted to Team Leader of my team but mostly because of circumstances. The previous team leader left for a start-up and I've been somehow the acting Scrum Master of the team for the past months (although our company sucks at Scrum generally speaking) and also having the most time in the company. However I'm still the youngest I'm my team so managing the actual team feels a bit weird and also I do not consider myself experienced enough to be a Technical lead but we don't have a different position for that.
Below actions happen in the course of 2-3 months.
With all the things above considered I find myself in a dire situation, a couple of months ago there were several Blocker bugs opened from the Clients side / production env related to one feature, however after spending about a month or so on trying to investigate the issues we've come to the conclusion that it needs to be refactorised as it's way too bad and it can't be solved (as a side note this issue has also been raised by a former dev who left the company). Although it was not part of the initial upcoming version release it was "forcefully" introduced in the plan and we took out of the scope other things but was still flagged as a potential risk. But wait..there's more, this feature was part of a Java microservice (the whole microservice basically) and our team is mostly made of JS, just one guy who actually works as a Java dev (I've only done one Java course during uni but never felt attracted to it). I've not been involved in the initial planning of this EPIC, my former TL was an the Java guy. Now during this the company decides that me and my TL were needed for a side project, so both of us got "pulled out" of the team and move there but we've also had to "manage" the team at the same time. In the end it's decided that since my TL will leave and I will take leadership of the team, I get "released" from the side project to manage the team. I'm left with about 3 weeks to slam dunk the feature.. but, I'm not a great leader for my team nor do I have the knowledge to help me teammate into fixing this Java MS, I do go about the normal schedule about asking him in the daily what is he working on and if he needs any help, but I don't really get into much details as I'm neither too much in sync with the feature nor with the technical part of Java. And here we are now in the last week, I've had several calls with PSO from the clients trying to push me into giving them a deadline on when will it be fixed that it's very important for the client to get this working in the next release and so on, however I do not hold an answer to that. I've been trying to explain to them that this was flagged as a risk and I can't guarantee them anything but that didn't seem to make them any happier. On the other side I feel like this team member has been slacking it a lot, his work this week would barely sum up a couple of hours from my point of view as I've asked him to push the branch he's been working on and checked his code changes. I'm a bit anxious to confront him however as I feel I haven't been on top of his situation either, not saying I was uninvolved but I definetly could have been a better manager for him and go into more details about his daily work and so on.
All in all there has been mistakes on all levels(maybe not on PSO as they can't really be held accountable for R&D inability to deliver stuff, but they should be a little more understandable at the very least) and it got us into a shitty situation which stresses me out and makes me feel like I've started my new position with a wrong step.
I'm just wondering if anyone has been in similar situations and has any tips or words of wisdom to share. Or how do you guys feel about the whole situation, am I just over stressing it? Did I get a good analysis, was there anything I could have done better? I'm open for any kind of feedback.2 -
More and more, I am getting frustrated/depressed from the attitude of our customers who complain, moan and get angry about issues in their infrastructure, while at the same time, refusing to pay more so the issues could be mitigated.
Like, a client's angry with us today for having one of their non-production-critical databases inaccessible for... Hmm... About 8 hours now (So a whole workday).
Like... I get it, some of your employees couldn't work with it offline, but like... What the hell do we do? You keep data from as far back as several years ago in there, without partitioning, without exports, in a mix of innodb and myisam, so when the DB crashes, and its replication has to be reset from zero, reimporting all the data takes hours upon hours, and importing .sql files just takes time.
Or another client who got angry when their app fell out of the internet, cuz one of their myisam-based log tables crashed, and had to be repaired, with data spanning several years back, meaning it took hours to fix...
The more I work with these "basic" and "simple" infrastructure designs that is *not* redundant, or HA, the more I wonder -- How do the big names out there do it? How do you design systems with fault tolerance so a single DB table crash doesn't lead to the whole app getting inaccessible?
We have... One, exactly one, client, who uses MariaDB with Gallera, and that cluster is *amazing*, it just keeps chugging along, without a care in the world. But it cost them quite a lot, as they had to buy 3 DB servers, instead of 1...1 -
Best:
Seeing ALL the members of my team finally coming into their own. One person tackled our entire not-at-all-simple CI/CD setup from scratch knowing nothing about any of it and, while not without bumps in the road, did an excellent job overall (and then did the same for some other projects since he found himself being the SME). Two of my more junior people took on some difficult tasks that required them to design and build some tricky features from the ground-up, rather than me giving them a ton of guidance, design and even a start on the basic code early on (I just gave them some general descriptions of what I was looking for and then let them run with it). Again, not without some hiccups, but they ultimately delivered and learned a lot in the process and, I think, gained a new sense of self-confidence, which to me is the real win. And my other person handled some tricky high-level stuff that got him deep in the weeds of all the corporate procedures I'd normally shield them all from and did very well with it (and like the other person, wound up being an SME and doing it for some other projects after that). It took a while to get here, but I finally feel like I don't need to do all the really difficult stuff myself, I can count on them now, and they, I think, no longer feel like they're in over their heads if I throw something difficult at them.
Worst:
A few critical bugs slipped into production this year, with a few requiring some after-hours heroics to deal with (and, unfortunately, due to the timing, it all fell on me). Of course, that just tells us that next year we really need to focus on more robust automated testing (though, in reality, at least one of the issues almost certainly would not - COULD NOT - have been caught before-hand anyway, and that's probably true for more than just one of them). We had avoided major issues the previous three years we've been live, so this was unusual. Then again, it's in a way a symptom of success because with more users and more usage, both of which exploded this year, typically does come more issues discovered, so I guess it tempers the bad just a little bit.2 -
Just look at the open issues counters for "state-of-the-art" "production-ready" JS packages:
https://github.com/storybookjs/....
Almost 1900.
https://github.com/vercel/next.js
More than 1200.
It's just depressing5 -
I have 3 jobs but one salary... I look after some financial enterprise system as one of my jobs ... In short am an integration support specialist, earlier today system was down ussd gateway... Some client pinged me on fucking skype asking for integration support on uat i ignored cause i was busy fixing production issues , the nigga escalated me to the CTio, sneakey bastard.. Some clients are shitty. Hope they make money
-
I get a late start (two weeks) on a jumping in on a project because I was assisting with production issues. The service is not running and basically nothing has been checked in. Mind you, we're not doing anything new.
"Senior" (while I'm trying to work on my part ) : Hey can you hurry up and finish your part? I'm thinking about coming up with a completely different way than what the group wants. (heard this several times)
Me : *finishs my part with coverage and gets the service up running and rating in a week because I'm avoiding code conflicts*
"Senior" : OK well nevermind what I said about coming up with a different strategy. I'll develop the last bit of the service since again everything has been laid out already on what to do.
Me : OK, I'll work on code coverage for the rest of the project and updating the code based on feedback from the other team members.
Me (a week later after hearing that he has moved on to another task) : Did you finish up that last bit?
"Senior" : Well I shifted focus working on feedback from the review. Feel free to finish that last bit I was supposed to work on because I don't know wtf I'm doing and I would rather ride your ass instead of attempting anything significant on my own.
Me: Heard. -
We're remediating tls issues on production servers. gotta be pic compliant. It's been an hour and a half for one server and were not even close to done... we have at least 5 more to go for this particular app... my organization controls over 300... the company has thousands... for the love of god save me.5
-
Sometimes I don't get "don't test on production".
And I'm definitely not a front-end guy, I only have debug and release in mobile development.
And I definitely often test on release, because it may be broken while debug build works fine.
You know what that means?
1. Test locally
2. Try to fix issues
3. Realize that this issues would ever appear ONLY locally
4. Move to staging and test
5. Fix issues
6. Realize that most of them are caused by workarounds for localhost
7. Move to production
8. Realize that everything is fucked up and you don't have any idea why, because `h5aqq2 was called on null"4 -
A while back I was looking for a new job and was given an interview by one company who shall remain nameless. Before the interview, they asked me look through their current site, nothing unusual there, so I started browsing. Then I received an email with all the details I needed to access their production server. Apparently they wanted me to look through the code, unusual but I did so.
First thing all the passwords, including those belonging to members of the public were stored in plain text and many were still the default passwords which were based on the Id so were sequential.
I highlighted these issues at the interview and they then asked me to do a test, not the usual test though, they asked me to add some charts to their prod site. Needless to say that didn’t happen and I got another job elsewhere.1 -
I'm tired. I don't want to do these tests anymore. These vague test scenarios I have to decrypt on my own lest asking business shows signs of weakness. I'm slow to test and going way beyond the hours the client estimated and you folks just accepted. How can I finish this when I get pulled to meetings which I am not the decision maker but I'm supposed to be the technical one to help them decide. In between this testing I get emails to help check on issues I'm not even a part of. Production issues I can understand because those have a feel of critical and priority but if you pull me to that I lose time testing. I'm trying. But I'm truly very slow at this. I'm a slow tester for this set of test cases. I'm hating myself every minute as the hours inch to the deadline which is today. I want to sleep but I want to finish as well. Shitty days of drone work that could have been given to somebody else but I can't say no to because you guys accepted. Someone from management just see please, don't give this to me. But you can't see. You probably don't even understand. They asked, you caved because you can't see the list of tasks and level of detail that comes with each thing they ask. This testing is a ridiculous use of my time but I can't say that to the client. You could have. I want to. I truly want to say "Fuck these tests". I tried to push back. But the client of course reasoned back and it was understandable to ask. To do what's good and what's best. How can I say no to that?! I'm almost depleted. I'll just finish this somehow.
-
After an idea has rolled into the production stage I become bored with it. I no longer want to maintain or even review git issues.
However when even the hint of a new project rolls in I start "pre-developing" in anticipation and excitement of a new project. Development and base design of a project is what keeps me going.
#firstworlddeveloperproblems3 -
Went to a hackathon and tried to use ARCore. Most painful experience of my life. There are so many issues and critical bugs that I can't even fit them all into a 5000 character rant, Google has shittier code than a highschool startup.
So instead of typing 5000 characters I'll just save you all some time. If you're forced to use ARCore, don't even try to use the AcquireCameraImageBytes or related apis for actually accessing the camera feed. Just use unity's screen capture API (draw an invisible rectangle on the whole screen, make a texture, readPixel entire rectangle). Turning off all models for 1 frame and taking a screen capture is easier, faster, and somehow more optimal than using Google's code.
Also, they released Augmented Faces on Friday. Their demo plainly doesn't work the way they intended on many devices because the list never gets populated since their engineers are dumb fucks. Just force the face mesh to always remain active and you'll instantly support all devices! You can deactivate it using your own methods but Google's doesn't work on many devices. There's an issue in their repo about this that they are plainly ignoring.
Also if you're interested I have a (working?) engine to use Object Detection for interactions within AR + a create your own adventure game demo made w/ object detection + ar on my git:
https://github.com/pshah123/...
My code is 100% crap so definitely don't use it in production but I was able to get the individual pieces working so hopefully this helps someone! Unless you're from Google, then fuck you please uninstallrant please uninstall google fuck google mv google /dev/null sudo rm google sudo kill -9 google git rm google16 -
So I'm assigned once again to fix a new someone else created and that seems to be the case whenever there's an issue...
Boss just assigns it to whoever is most likely to be able to investigate it... which is basically me. Other than the little time I can use to develop stuff, I'm usually cleaning up other people's messes.
And these other people are to busy working on new crap to properly explain how their existing code/processes/changes works.
And well the fact that anything breaks in production (that's not due to upstream one off issues) whoever does not think he needs to take responsibility for it.
So everyone else and especially me has to spend time understanding the shit they wrote and fixing it for them.
How do I tell my boss this nicely that we need clearly definitely ownership and whenever a component blows up in prod, the guy that wrote the code fixes it no matter what? Thereby incentivizing him to not write shit code in the first place and be more proactive in making sure it doesn't in the first place since he knows otherwise he's doing overtime to fix it?
Is it just me or is there really no such thing as a dev job where something doesn't blow up due to poorly tested and designed code every other day?3 -
Currently debugging a project that was written over 4 years ago...
At first all was well in the world, besides the ever present issue off our goddamn legacy framework. This framework was written 7 years ago on top of an existing open source one, because the existing one was 'lacking some features' & 'did not feel right'.
Now those might be perfectly fine reasons to write a layer on top of a framework, but please, for all future devs sanities, write fucking documentation and maintain it if you're going to use said framework in all major projects!!
Anyhow back to the situation at hand, I'm getting familiar with the project, sighing at the use of our stupid legacy framework, attempting to recreate the reported bugs...
Turns out I can't, well I get other bugs & errors, but not the reported ones. I go to the production server, where I suddenly do can reproduce them...
Already thinking, fuck my life, and scared for the results... I try a 'git status' on the production server....
And yep, there it is, lo and behold, fucking changes on production, that are not in git, fuck you previous dev who worked on this and your stupid lazy ass modifcations on production!
Bleh, already feeling royally pissed, there's only 1 thing I can do, push changes back to git in a seperate branch, and pray I can merge them back in master on my dev environment without to much issues...
Only I first have to get our sysadmi. to allow pushing from a production server back to our git server...
Sigh, going to put on my headphones, retreat to my me space and try to sort out this shitpile now... -
Gah, I just received this Ubuntu 18.04 VM with 8 cores and 8 gigs of ram, and since it'll be a production server both serving public and "private" networks (yes, shout at me, but projects won't be about hosting sensitive information, I wouldn't put all that on one server), and I'm struggling between my options.
Docker, or not docker?
The server's main use is to host our growing blog and install Varnish, which will hog some ram after a while. I use Laradock for my dev projets, it's really easy to develop with it, but I am unsure if it fits a production environment with performance, security and traffic load in mind :(
I read Docker has stability issues (in 2016-2017), and can bring the machine down with it, I don't know if I should just install the software (nginx, apache, percona/mysql/maria) without "containerizing" it and go for it
I'm lost xD7 -
WHY DOES EVERYTHING BREAK WHEN I WANT TO DEPLOY?
Finally fixed the last few bugs on my project and the thing is pretty much set to go. Then both the production and my local copy (identical in every way except for connecting to different MySQL servers) have the EXACT same query issues. The cursor fetch methods ALL get nothing EVEN WHEN I CAN SEE THE CURSOR HAS THE DATA I REQUESTED in the debugger.
I'm already in hot water for taking longer than expected with this project and this is going to push it back even further. -
First project at new company ended up shit as clients kept using the backlog to define and refine their business requirements. Did not go to production.
Second project at same company ended up the same way, except it had more infrastructure issues than technical debt (and an asshole for a project manager).
Basically I'm scoring 2 for 2, and totally expecting my next project to be doomed too for a 3 score. Maybe I'll build up enough rep as that guy who dooms projects to just sit on my ass and collect my paycheck while I work on my personal stuff. -
Compare and harmonize the web configs
Oh no someone set execution timeouts to 14 days
Fuck fuck fuckity duck
Hey compare all the web configs of all environments and harmonize them all wtf cmon bruh do your job as a developer
Take them and back them up into svn. What do you mean svn isn't a back up system of course it is well its the only thing we have fuck
What do you mean we have shit logging where people will catch an exception and only print the word exception in the log you can figure it out can't you we have live produxtion issues that hace to be solved now what the fuck
How dare you make a. Mistake copying our shitload of a bloated codebase and configuring our 100s of different options all by fukcing hand what the fuck dude do yoh write anyrhing down?
Please catalogue all the exception mails we are getting but we have no db or error reporting system so they all just plop into tue inbox and thats all ypur fuckjng data figure it out kid
This is a rewarding, fulfilling job whwrw you can be both dev ops and a developer and manage all of our fucking environments of which there are about 15 of all your own with no sort of tool or software to aid you because haha what the fuck we wouldn't make your life easy
Whata that you want to spend time to write stuff or change stuff that will nake it easier fot you fuxk that bruh get back to your biklable tasks like holy shit you thjnk this is a charity ofr aomw shit
Live production issues
Live production issues
Produxtion issues. A ghost in the machine. Find it fix if find it fix it find it fix it cmon why can't you fix it I expect you to spend your day hopelessly pretending to try to solve something you fucker
One of the only peopel able to help you sometimes though hes a bit of an old laxky, yeah hea fucking leaving see ya seeya kid and now we're not hirinf anyone to fuckjng help you no no no managing and monitoring the environments its your jov alll fof them every sngle on do you knkw all the xonfiguraiton values for them yet??
Instead we are hiring a new sales person to fucking make us some more money and we don't need naother seceloper to help you infqct lets have you use this mid end retail computer from 2014 to develop on yeah yeah oh but all our shitty code and visual studip will destry your memory but too bad!! Hahahahahdhsj
Go lice is all you, why sare you so slow
How long will it take
How long will it take
How long will it take
How long witll it tqk2
How long will it take holy shit
Give time estimate for sonethign that I don't fucking know how about it will tqke till fuxk you oxloxk4 -
#define SOAPBOX
Data will contain errors and inconsistencies, and the only player that can detect them is: the computer itself.
if the computer detects that it is in a situation where it cannot meaningfully continue, it should not allow itself to continue.
Only the computer can make that determination.
If the software does not aggressively test its data, it is usually impossible to determine whether the problem is with the software or with the data or both.
Per contra, if it does do so, it becomes impossible to assert that the input data does not contain the issues that are being tested for ... a very important thing to be able to say in a real-world production setting, where hundred-megabyte input files are common.
#undef SOAPBOX -
Somehow mocking xhr requests (?) for Axios is really hard to make it work. I use React Cosmos as I'm re-doing the frontend of this already running in production and works great, but when my component communicates with the backend it breaks and I'm unable to test the full behavior.
Then, it occurred to me that trying to mock Axios may not be the best. So I came with this scheme where I would have a configuration variable with a default value and change that when I need to work with React Cosmos, which in turn changes the behavior of `/auth` to return a valid JWT in response to a GET, put an Axios interceptor in my outermost Cosmos decorator and BAM! suddenly was able to develop and test my React components closer to how they would work in production.
It surprises me how simple this endeavor was, and because everything runs orchestrated by docker compose things run smoother.
(this is not an excuse to not to learn how to deal with the mocking issues of Axios, after all I wont have a working backend every time I work in some frontend application)5 -
How does one get better at responding to issues that occur in production?? I feel the only way I learn is by having the issue occur and then fixing it...any more proactive ways available??3
-
I want to build a "report-error" function into the app to make errors in production easier to track and debug. But i am unsure, what it should do actually.
Create a GitHub Issue? Then every Hoster needs a specific GitHub account and repo for that.
Send an Email? Then connectivity-related issues are not sendable. Send an Email with the frontend? Brr. Any Tips?16 -
Anyone use docker in production handling monies and hundreds accounts? In Django in my case but doesnt matter the framework. More concerned with security and stability moving from paas to docker based paas. Worried I'll move everything to docker and end up moving back to vms bc of some issues or some vulnerability.
-
Recently joined new Android app (product) based project & got source code of existing prod app version.
Product source code must be easy to understand so that it could be supported for long term. In contrast to that, existing source structure is much difficult to understand.
Package structure is flat only 3 packages ui, service, utils. No module based grouped classes.
No memory release is done. So on each screen launch new memory leaks keep going on & on.
Too much duplication of code. Some lazy developer in the past had not even made wrappers to avoid direct usage of core classes like Shared Preference etc. So at each place same 4-5 lines were written.
Too much if-else ladders (4-5 blocks) & unnecessary repetitions of outer if condition in inner if condition. It looks like the owner of this nested if block implementation has trust issues, like that person thought computer 'forgets' about outer if when inside inner if.
Too much misuse of broadcast receiver to track activities' state in the era of activity, apપ life cycle related Android library.
Sometimes I think why people waste soooo... much efforts in the wrong direction & why can't just use library?!!
These things are found without even deep diving into the code, I don't know how much horrific things may come out of the closet.
This same app is being used by many companies in many different fields like banking, finance, insurance, govt. agencies etc.
Sometimes I surprise how this source passed review & reached the production. -
Uri Josef Drucker - Information
Uri Josef Drucker, nicknamed Uri Drucker, or just Drucker is an entrepreneur with many years of experience across different markets.
Drucker formed a company in 1984, producing a range of women’s hygiene products, employing over 100 staff. The products were distributed across Israel and Europe. The company was sold with a successful exit in the 1990’s.
Uri Josef Drucker produced, printed, and distributed a newspaper called ‘The Main Issue’ for 10 years. The paper focused on regional municipal and environmental issues and was successfully sold in 2015 and is still printing to this day. The production was based in Kiryat Tivon, near Haifa, Israel.
Uri Drucker has been living in Kiryat Tivon for many years and was born as Uri Josef Drucker in the city of Haifa, Israel.
Drucker was also a political candidate for the local elections in Kiryat Tivon in 2018. During the race, Drucker connected to many people in his town and managed to increase his great ability of listening to others and giving satisfying solutions to common issues. Although he did not win the local elections, Uri Drucker continues giving to his community until this day.
If you want to learn more about Uri Josef Drucker, you should also visit Uri Josef Drucker's social media profile pages. The links to Drucker’s social media profiles are listed at the bottom of this page.
Also, you can feel free to message Drucker in his various profile pages and please be sure to follow him or add him as your friend on Social media. Connect with Drucker and send him a message for any questions, inquiries, or just to chat.
It’s very important to state that Uri Josef Drucker can be found online in many different social media websites and he will do his best to answer you in each and every single one, so connect to him on your favorite network
Take into account that this website profile is solely dedicated to Uri Josef Drucker, but he does not manage it personally and it might take him time to respond.
Please note that Uri Drucker is not responsible for creating this profile and we can not guarantee that Uri Josef Drucker will indeed reply here. If you want Uri Drucker to contact you back, please visit some of his other profile pages that represent Uri Josef Drucker and try to contact him there, as if he doesn’t answer in one profile, he will surely answer in another one.
Drucker has over 50 social media profiles in order to satisfy different people that use different websites. -
Today, I began learning about the wonders and horrors of HA in production environments.
My head feels like when I first joined the company as a total noob who never worked an IT job in his life. Soooooooo much new information and concepts and potential issues to learn to avoid.
But all super interesting!