Dev: "Ah, I finally fixed that code I was working on the other day and got it pushed to staging!"

Almond: "Ah, great! What was the issue in the end?"

Dev: "It was an odd one - it wasn't actually my code that was the issue, there was a bunch of other code getting in the way."

Almond: "How do you mean?"

Dev: "It kept complaining about something called a "unit test" failing - so after a while I found the right unit tests, deleted them, and now it works great!"

Almond: "..."

!rant
!!git

Who here uses `master` for development?

My boss (api guy) tried to convince me that was normal practice. I gently told him that it sounded crazy and very very bad.

Here's the dev path I'm enforcing on my repos:

(feature branches) -> dev -> qa* -> master -> production*

*: the build server auto-pulls from these branches, and pushes any passing builds to staging/production.

Everyone works on their own feature branches, and when they're happy with their work, they merge it into `dev`. `dev`, therefore, is for feature integration testing. After everything is working well on `dev`, it gets merged into `qa` for the testers to fawn over and beat with sticks. Anything that passes QA gets merged into `master`, where it sits until we're ready to release it. When that time comes (it's usually right away, but not always), `master` gets merged into `production`.

This way, `master` is always stable and contains the newest code, so it's perfect for forking/etc. Is this standard practice, or should I be doing something different?

Also, api guy encourages something he calls "running a racetrack" -- each dev has their own branch (their initials) and they push to that throughout the day. everyone else pulls from it regularly and pushes to their own branch. When anyone's happy with their code, they push from their (updated) branch to `qa` (I insisted on `dev` instead.)

Supposedly this drastically reduces the number of merge conflicts when pushing to an upstream branch due to having a more recent ancestor node?

I don't quite follow that, but it seems to me that merging/pushing throughout the day would just make them happen sooner? idk.

What are your thoughts?

Series of events between me (Mi) and dude in office (DIO).

Instance 1
DIO: There is not psql installed on staging.
Mi: Install it.
DIO: YUM is not working.
Mi: *tries yum it works* It is
DIO: Oh. Didn't work earlier.
Mi: *blank* Make sure you install 9.6
DIO: Cannot find psql
Mi: *types psql, it is already installed*
DIO: Oh, didn't work earlier.

Instance 2
DIO: Made this change to the API, the endpoint is not returning the right value
Mi: *restarts server, shit starts working*
DIO: I am pretty sure I did that, don't know what happened.

Instance 3
DIO: Cannot alter role to give login to this db user.
MI: *runs alter role db_user with login* works
DIO: Don't know why it wasn't working before.

Instance 4
DIO: I have been stuck on this test for the past 1 day, cannot get the API to return the right data while the Rest Endpoint works fine.
Mi: You are hitting the wrong endpoint in the test.
DIO: Oh, I put an extra 's'
Mi: BTW you are testing Spring-Boot with that test and nothing else.
DIO: Yes but what if Spring Boot has a bug?
Mi: ok.

Unaware that this had been occurring for while, DBA manager walks into our cube area:

DBAMgr-Scott: "DBA-Kelly told me you still having problems connecting to the new staging servers?"
Dev-Carl: "Yea, still getting access denied. Same problem we've been having for a couple of weeks"
DBAMgr-Scott: "Damn it, I hate you. I got to have Kelly working with data warehouse project. I guess I've got to start working on fixing this problem."
Dev-Carl: "Ha ha..sorry. I've checked everything. Its definitely something on the sql server side."
DBAMgr-Scott: "I guess my day is shot. I've got to talk to the network admin, when I get back, lets put our heads together and figure this out."
<Scott leaves>
Me: "A permissions issue on staging? All my stuff is working fine and been working fine for a long while."
Dev-Carl: "Yea, there is nothing different about any of the other environments."
Me: "That doesn't sound right. What's the error?"
Dev-Carl: "Permissions"
Me: "No, the actual exception, never mind, I'll look it up in Splunk."
<in about 30 seconds, I find the actual exception, Win32Exception: Access is denied in OpenSqlFileStream, a little google-fu and .. >
Me: "Is the service using Windows authentication or SQL authentication?"
Dev-Carl: "SQL authentication."
Me: "Switch it to windows authentication"
<Dev-Carl changes authentication...service works like a charm>
Dev-Carl: "OMG, it worked! We've been working on this problem for almost two weeks and it only took you 30 seconds."
Me: "Now that it works, and the service had been working, what changed?"
Dev-Carl: "Oh..look at that, Dev-Jake changed the connection string two weeks ago. Weird. Thanks for your help."
<My brain is screaming "YOU NEVER THOUGHT TO LOOK FOR WHAT CHANGED!!!"
Me: "I'm happy I could help."

My department is focused solely on web development. Of course we are part of the major portion of I.T

The entire I.T department got acknowledged for a very important piece of software. That I wrote.
The ceremony in which we were being recognized did not listed MY department, no, they listed the ENTIRETY of I.T.

Thing is, if this product was not delivered, then I was told that the blame would be MINE (I am speaking as the head of my department) but apparently if it succeeded (which it did) it is to be attributed to people that were not even involved in the project.

My employees tried calming me down when I got upset, one of them stated that it was not even our department's effort, but mine alone. And yes, I was the one that developed the solution. By myself, with complete testing, staging, the whole works. Everything, developed by me. BUT my employees held the entire department down while I was behind close doors developing this solution.
I was fucking upset, more so because my director sent an email thanking the entire I.T department for this "win"
I asked him through or messaging service if he could point out to me who else was involved, since I did not know of anyone else that did absolutely anything in this process other than myself and my guys.

Maybe the output of my program was parsed by another I.T department and something happened from it, maybe the money generated by the application (obscene amounts of it btw) were used to add more to the infrastructure etc, who knows, but as far as I know, you cannot say "if this fails it is on you" just for them to later on thank people that were not involved in the project.

This is why I would gladly move on to a different field. I don't want to be patted on the back constantly, I know how fucking good I am at what I do. But if I do something amazing I do not want to see those efforts being given to someone else.

The dev world is usually a thankless industry, but if thanks are given, then I want the sole credit.

If I am winning or loosing I want the whole fucking credit and you can be any more gangstah than that.

Joined a new company / team to work on an iOS app that has 2 different backend environments "Dev" and "Prod". Also being referred to in iOS speak as "Debug" and "Release".

Been trying to get accounts on these backends (no sign up in app, its controlled via another process). Eventually get access to "Dev" for one of the regions, so I load up "Debug" and its not working.

This is odd, so I open the Android app and load "Dev" and it works? I then Notice Android has "Dev", "QA", "Staging" and "Prod" for every region where as iOS only has 2 of these.

So I go back to iOS and find the file for the settings and it has iOS Debug assigned a variable for the backend Dev ... which is actually pointing to QA. Because they use QA to Debug and not Dev.

... confused? join the club

It was my internship and I've end up working on a law company specializing on Australian construction laws they're working on a website that will take care of all the paperworks for the contractors. They have a dev team who's working on it but they don't have a web designer. I was accepted for the job as an intern/web designer/tester. I was so happy that I've got a really cool internship as a designer but that's only for a second.

The hell starts on day one. They've told me that they're using agile workflow and that they need to make the website responsive. It was based on bootstrap and gosh their code was so broken. HTML tags overlay on each other, some are unclosed. I've tried to fix the problems and did a great job at that. Made the front page responsive and all laid out. When I went to the next php file it has a different header.php and footer.php and same problems apply and we're not even touching the worst.

They didn't use any version management and they're cowboying everything. Now that the website is on the staging server they use Cpanel text editor to edit the code! My headache started to pileup.

The Australian client asked me to provide icons and fix the colors of the website. Also the typography looks great already. I've fixed almost all the problems and I'm satisfied with the design when suddenly a new co-worker from a famous and expensive college was absorbed by the company. He worked as the marketing specialist who has no experience at web design at all. He told me to do this and that and the whole website changed. He bullied me for my skills in design (I'm an intern) and just took over the whole design. Everyone even the boss listen to him as if everything he say is right. He's skilled at design but not web design. He made the website look like a freakin movie poster.

All my works are for nothing, I got headache for nothing and I've got hated for nothing.

It was the day when I finished my internship. It was a long 3 months. After a month I've heard from my co-interns that the whole dev team was fired including the marketing specialist. Also the whole website is scrapped and has been rebuilt by a single guy who used WordPress which he did in only a month.

Have a couple I want to air today.

First was at my first gig as a dev, 4-5 months out of school. I was the only dev at a startup where the owner was a computer illiterate psycopath with serious temper tantrums. We're talking slamming doors, shouting at you while you are on the phone with customers, the works...

Anyways, what happened was that we needed to do an update in our database to correct some data on a few order lines regarding a specific product. Guess who forgot the fucking where-clause... Did I mention this boss was a cheap ass, dollar stupid, penny wise asshole that refused to have anything but the cheapest hosting? No backups, no test/dev/staging environment, no local copies... Yeah, live devving in prod, fucking all customers with a missing semi-colon (or where clause).

Amazingly, his sheer incompetance saved my ass, because even if I explained it, he didn't get it, and just wanted it fixed as best we could.

The second time was at a different company where we were delivering managed network services for a few municipalities. I was working netops at that time, mostly Cisco branded stuff, from Voice-over-IP and wifi to switches and some routing.

One day I was rolling out a new wireless network, and had to add the VLAN to the core switch on the correct port. VLAN's, for those who don't know, are virtual networks you can use to run several separated networks on the same cable.

To add a VLAN on a Cisco switch one uses the command:

switchport access vlan add XYZ

My mistake was omitting the 'add', which Cisco switches happily accept without warning. That command however can be quite disruptive as it replaces all of the excisting VLAN's with the new one.

Not a big deal on a distribution switch supplying an office floor or something, but on a fucking core switch in the datacenter this meant 20K user had no internet, no access to the applications in the DS, no access to Active Directory etc. Oh and my remote access to that switch also went down the drain...

Luckily a colleague of mine was on site with a console cable and access to config backups. Shit was over within 15 minutes. My boss at that time was thankfully a pragmatic guy who just responded "Well, at least you won't make that mistake again" when we debriefed him after the dust settled.

I used to work on a production management team, whose job was, among other things, safeguarding access to production. Dev teams would send us requests all the time to, "run a quick SQL script."

Invariably, the SQL would include, "SELECT * FROM db_config."

We would push the tickets back, and the devs would call us, enraged. I learned pretty quickly that they didn't have any real interest in dev, test, or staging environments, and just wanted to do everything in prod, and see if it works.

But they would give up their protests pretty fast when I offered to let them speak to a manager when they were upset I wouldn't run their SQL.

Don't you just love it when something works in your development environment, your staging environment and then randomly breaks on production? :^)

Giant, month-and-a-half-long-ticket.
After learning six or so complicated areas of the system and updating them all to work with the new changes, make them all play nicely, etc. I finally got everything working. 95% spec coverage, though no ui tests because I haven't gotten selenium working. whatever, everything's done and works.

Second dev bases her ticket off of mine and continues working. Work elsewhere continues and there's an official release, so we both merge in master. I run tests, everything passes, and go back to working on other tickets.

She finishes her ticket.
We do end-to-end testing, and everything works perfectly. Time for a demo!

She merges in master again, and pushes her branch to two staging servers. (idk why two.)

Demo starts.
We connect to the staging servers, and... none of the UI changes exist; they aren't running the correct code!

So she runs it locally for a demo instead. Two features in my ticket no longer work. She throws me under the bus. She throws me under the bus again by criticising a rake task I scrapped because she wanted to do it. Then again because I didn't update my branch to master and push it before the demo, despite having no reason to. and despite the demo being of her branch.

Then she continues to show off and brag about how she's like the "legend" (senior dev) she envies. QAbuys it.

I'm having an emotion, and it's called anger.

//rant

So I'm a BI consultant, been doing this for about 6 years now, and I'm pretty good at the data stuffs. Now I had to complete a project for a client where we call a web service and it had to be done in .NET. I wrote a console app in C# that called the WS, dumped the data then a stored proc processed the staging tables into final tables that our visualization tool can consume.

It works, it's done.

Mind you I'm not a pure .NET developer.

And now that it's completed and working this fucking .NET dude that works for my client is basically giving me an attitude talking about "why wasn't it done as a Windows service? Blah, blah" Like WTF!!??? I get that he's the C# BSD but like chill bruh!!

It's annoying as fuck having to work on projects that are not your area of EXPERTISE and then be ridiculed by other elitist assholes about it.

Doesn't happen much, but fuck it's something I hate about dev. FYI, if it was the opposite I would just be asking questions for understanding, not being a sarcastic prick.

//rant done

>Adds new feature
>New feature works fine on dev
>New feature works fine on staging
>New feature doesn't work on live
>You can't easily figure out what is wrong because you need to wait an hour for it each time :|

Me: "Oh right I still had to fix that"
Me: *fixes thing*
Me: *checks on staging to make sure it all works*
Thing: *works absolutely fine*
Me: "Cool, let's promote it"
Me: *updates production stack*
Me: *checks on production to make sure it works*
Thing: *still broken*

dsfhjh dsaghdsaupot nvhudfot vnhgue

Long story ahead

Background:
I recently started a job in a smallish startup doing web development in a mostly js stack as an entry-junior engineer/dev. I’m the only person actively working on our internal tools as my Lead Engineer (the only other in house dev) is working on other stuff.

Now I was given a two week sprint to rebuild a portion of our legacy internal app from angular 1.2 with material-ui looking components with no psd’s or cut-outs of any kind to a React and bootstrap ui for the front end and convert our .net API routes into Node.js ones. I had to build the API routes, SQL queries (as there were plenty of changes and reiterations that I had to go through to get the exact data I needed to display), and front end. I worked from 9am until 11pm every day for those two weeks including weekends as our company has a huge show this upcoming week.

I finish up this past sunday and push to our staging environment. The UI is 5.5/10 as we’re changing all of our styling to bootstrap and I’m no ui expert. The api has tests and works flawlessly (tm).

So we go into code review and everything is working as expected until one tab that I made erred out and was written down as a “Needs to be fixed.”

This fix was just a null value handler that took three minutes and a push back to staging, but that wasnt before a stupendous amount of shit being flung my way for the ui not looking great and that one bug was a huge deal and that he couldnt believe it slipped through my fingers.

Honestly, I’m feeling really unmotivated to do anything else. I overworked myself for that only to be shit on for one mistake and my ui being lack-luster with no guides.

Am I being a baby about this or is this something to learn from?

Finally made my node production server stable enough that I could focus on writing tests*. I start by setting up docker, mocking cognito, preparing the database and everything. Reading up on Node test suites and following a short tut to set up my first unit test. Didn't go smoothly, but it's local and there are no deadlines so who cares. 4 days later, first assert.equal(1+1, 2) passes and I'm happy.

I start writing all sorts of tests, installing everything required into "devDependancies," and getting the joy of having some tests pass on first try with all asserts set up, feels good!

I decide to make a small update to production, so I add a test, run and see it fail, implement the feature, re-run and, it passes!

I push the feature to develop, test it, and it works as intended. Merge that to master and subsequently to one of my ec2 production servers**, and lo and behold, production server is on a bootloop claiming it "Cannot find module `graphql`". But how? I didn't change any production dependencies, and my package lock json is committed so wth?

I google the issue, but can't find anything relevant. The only thing that I could guess was that some dependencies (including graphql) were referenced*** in both, prod and dev, and were omitted when installed on a prod NODE_ENV, but googling that specific issue yielded no results, and I would have thought npm would be clever enough to see that and would always install those dependencies (spoiler: it didn't for me).

With reduced production capacity (having one server down) I decided to npm uninstall all dev dependencies anyway and see what happens. Aaaaand it works.....

So now I have a working production server, but broken local tests, and I'm not sure why npm is behaving like this...

* Yes I see the irony.
** No staging because $$$, also this is a personal project.
*** I am not directly referencing the same thing twice, it's probably a subdependency somewhere.

> push code on staging and promote on production works fine
>.come home
> pull from master and code doesn't work locally
> pull from the feature branch, same issue
> fml

I'd be so fucking disappointed in myself if I hadn't pushed it and merged it.

A developer couldn't get a application performance monitoring (APM) tool to trace his application. They claimed that their libraries and their configurations were alright and that the APM tool was non-performant.

The developer then argues with sysadmin that the APM tool can't trace the application and that there's nothing wrong with the application or the configurations. When sysadmin questions whether the developer got the tool to work anywhere, they say, "No" and head off to make it work at least in one place. They come back saying that it works on their development environment (which is their local machine). Sysadmin claims that the system configurations on the server instances cannot be matched by the development environment and there could be a lot more factors to be considered for the problem. The sysadmin asks to prove it on a server instance on one of the test environments and then they'd agree that it is a problem with the tool. They also argue that this is not the only application that uses the APM tool and the tool happily traces other applications with no issues.

The developer tries the same configuration on a staging instance and fails. In order to make it work, they silently uninstall the existing version of the APM tool and then compiles an unstable branch of the tool. It finally works with this version.

They go back to the sysadmin and show that it works on the staging environment, but does not on production. After banging their head on the wall for a while, the sysadmin figure that the tool had been swapped out for the unstable branch that was manually compiled. When questioned, the developer responds, "It works with this version on staging, so deploy the same version on production"

WTF? You don't deploy an unstable branch to production. Just because you can't make it work on the stable branch doesn't mean that it is the problem with the tool itself. There's a big difference between a stable branch and a non-stable branch. How would you feel if the sysadmin retorted by asking you to deploy the staging branch of your application to production?

After three months of development, my first contribution to the client is going live on their servers in less than 12 hours. And let me say, I shall never again be doing that much programming in one go, because the last week and a half has been a nightmare... Where to begin...

So last Monday, my code passed to our testing servers, for QA to review and give its seal of approval. But the server was acting up and wouldn't let us do much, giving us tons of timeouts and other errors, so we reported it to the sysadmin and had to put off the testing.

Now that's all fine and dandy, but last Wednesday we had to prepare the release for 4 days of regression testing on our staging servers, which meant that by Wednesday night the code had to be greenlight by QA. Tuesday the sysadmin was unable to check the problem on our testing servers, so we had to wait to Wednesday.

Wednesday comes along, I'm patching a couple things I saw, and around lunch time we deploy to the testing servers. I launch our fancy new Postman tests which pass in local, and I get a bunch of errors. Partially my codes fault, partially the testing env manipulating server responses and systems failing.

Fifteen minutes before I leave work on the day we have to leave everything ready to pass to staging, I find another bug, which is not really something I can ignore. My typing skills go to work as I'm hammering line after line of code out, trying to get it finished so we can deploy and test when I get home. Done just in time to catch the bus home...

So I get home. Run the tests. Still a couple failures due to the bug I tried to resolve. We ask for an extension till the following morning, thus delaying our deployment to staging. Eight hours later, at 1AM, after working a full 8 hours before, I push my code and leave it ready for deployment the following morning. Finally, everything works and we can get our code up to staging. Tests had to be modified to accommodate the shitty testing environment, but I'm happy that we're finally done there.

Staging server shits itself for half a day, so we end up doing regression tests a full day late, without a change in date for our upload to production (yay...).

We get to staging, I run my tests, all green, all working, so happy. I keep on working on other stuff, and the day that we were slated to upload to production, my coworkers find that throughout the development (which included a huge migration), code was removed which should not have. Team panics. Everyone is reviewing my commits (over a hundred commits) trying to see what we're missing that is required (especially legal requirements). Upload to production is delayed one day because of this. Ended up being one class missing, and a couple lines of code, which is my bad (but seriously, not bad considering I'm a Junior who was handed this project as his first task at his first job).

I swear to God, from here on out, one feature per branch and merge request. Never again shall I let this happen. I don't even know why it was allowed to happen, it breaks our branch policies. But ohel... I will now personally oppose crap like this too...

Now if you'll excuse me... I'm going to be highly unproductive and rest, because I might start balding otherwise after these weeks...

Sometimes I don't get "don't test on production".
And I'm definitely not a front-end guy, I only have debug and release in mobile development.
And I definitely often test on release, because it may be broken while debug build works fine.
You know what that means?
1. Test locally
2. Try to fix issues
3. Realize that this issues would ever appear ONLY locally
4. Move to staging and test
5. Fix issues
6. Realize that most of them are caused by workarounds for localhost
7. Move to production
8. Realize that everything is fucked up and you don't have any idea why, because `h5aqq2 was called on null"

When you spend hours in a messy codebase to fix a bug properly and add an integration spec to cover that specific case.
And even you do a round of testing on staging + providing screenshots, there is always someone on the team that will write in your PR, "It works, I tested the change on my machine".
I understand that some people are skeptical but to the point of not trusting integration tests + screeshots/recordings then please test it on staging or production next time because if it works on your machine doesn't mean it will work there ;)