Welcome back to practiseSafeHex's new life as a manager.

Episode 2: Why automate when you can spend all day doing it by hand

This is a particularly special episode for me, as these problems are taking up so much of my time with non-sensical bullshit, that i'm delayed with everything else. Some badly require tooling or new products. Some are just unnecessary processes or annoyances that should not need to be handled by another human. So lets jump right in, in no particular order:

- Jira ... nuff said? not quite because somehow some blue moon, planets aligning, act of god style set of circumstances lined up to allow this team to somehow make Jira worse. On one hand we have a gigantic Jira project containing 7 separate sub teams, a million different labels / epics and 4.2 million possible assignees, all making sure the loading page takes as long as possible to open. But the new country we've added support for in the app gets a separate project. So we have product, backend, mobile, design, management etc on one, and mobile-country2 on another. This delightfully means a lot of duplication and copy pasting from one to the other, for literally no reason what so ever.

- Everything on Jira is found through a label. Every time something happens, a new one is created. So I need to check for "iOS", "Android", "iOS-country2", "Android-country2", "mobile-<feature>", "mobile-<feature>-issues", "mobile-<feature>-prod-issues", "mobile-<feature>-existing-issues" and "<project>-July31" ... why July31? Because some fucking moron decided to do a round of testing, and tag all the issues with the current date (despite the fact Jira does that anyway), which somehow still gets used from time to time because nobody pays attention to what they are doing. This means creating and modifying filters on a daily basis ... after spending time trying to figure out what its not in the first one.

- One of my favourite morning rituals I like to call "Jira dumpster diving". This involves me removing all the filters and reading all the tickets. Why would I do such a thing? oh remember the 9000 labels I mentioned earlier? right well its very likely that they actually won't use any of them ... or the wrong ones ... or assign to the wrong person, so I have to go find them and fix them. If I don't, i'll get yelled at, because clearly it's my fault.

- Moving on from Jira. As some of you might have seen in your companies, if you use things like TestFlight, HockeyApp, AppCenter, BuddyBuild etc. that when you release a new app version for testing, each version comes with an automated change-log, listing ticket numbers addressed ...... yeah we don't do that. No we use this shitty service, which is effectively an FTP server and a webpage, that only allows you to host the new versions. Sending out those emails is all manual ... distribution groups?? ... whats that?

- Moving back to Jira. Can't even automate the changelog with a script, because I can't even make sense of the tickets, in order to translate that to a script.

- Moving on from Jira. Me and one of the remote testers play this great game I like to call "tag team ticketing". It's so much fun. Right heres how to play, you'll need a QA and a PM.

*QA creates a ticket, and puts nothing of any use inside it, and assigns to the PM.
*PM fires it back asking for clarification.
*QA adds in what he feels is clarification (hes wrong) and assigns it back to the PM.
*PM sends detailed instructions, with examples as to what is needed and assigns it back.
*QA adds 1 of the 3 things required and assigns it back.
*PM assigns it back saying the one thing added is from the wrong day, and reminds him about the other 2 items.
*QA adds some random piece of unrelated info to the ticket instead, forgetting about the 3 things and assigns it back.

and you just continue doing this for the whole dev / release cycle hahaha. Oh you guys have no idea how much fun it is, seriously give it a go, you'll thank me later ... or kill yourselves, each to their own.

- Moving back to Jira. I decided to take an action of creating a new project for my team (the mobile team) and set it up the way we want and just ignore everything going on around us. Use proper automation, and a kanban board. Maybe only give product a slack bot interface that won't allow them to create a ticket without what we need etc. Spent 25 minutes looking for the "create new project" button before finding the link which says I need to open a ticket with support and wait ... 5 ... fucking ... long ... painful ... unnecessary ... business days.

... Heres hoping my head continues to not have a bullet hole in it by then.

Id love to talk more, but those filters ain't gonna fix themselves. So we'll have to leave it here for today. Tune in again for another episode soon.

And remember to always practiseSafeHex

!Story

The day I became the 400 pound Chinese hacker 4chan.

I built this front-end solution for a client (but behind a back end login), and we get on the line with some fancy European team who will handle penetration testing for the client as we are nearing dev completion.

They seem... pretty confident in themselves, and pretty disrespectful to the LAMP environment, and make the client worry even though it's behind a login the project is still vulnerable. No idea why the client hired an uppity .NET house to test a LAMP app. I don't even bother asking these questions anymore...

And worse, they insist we allow them to scrape for vulnerabilities BEHIND the server side login. As though a user was already compromised.

So, I know I want to fuck with them. and I sit around and smoke some weed and just let this issue marinate around in my crazy ass brain for a bit. Trying to think of a way I can obfuscate all this localStorage and what it's doing... And then, inspiration strikes.

I know this library for compressing JSON. I only use it when localStorage space gets tight, and this project was only storing a few k to localStorage... so compression was unnecessary, but what the hell. Problem: it would be obvious from exposed source that it was being called.

After a little more thought, I decide to override the addslashes and stripslashes functions and to do the compression/decompression from within those overrides.

I then minify the whole thing and stash it in the minified jquery file.

So, what LOOKS from exposed client side code to be a simple addslashes ends up compressing the JSON before putting it in localStorage. And what LOOKS like a stripslashes decompresses.

Now, the compression does some bit math that frankly is over my head, but the practical result is if you output the data compressed, it looks like mandarin and random characters. As a result, everything that can be seen in dev tools looks like the image.

So we GIVE the penetration team login credentials... they log in and start trying to crack it.

I sit and wait. Grinning as fuck.

Not even an hour goes by and they call an emergency meeting. I can barely contain laughter.

We get my PM and me and then several guys from their team on the line. They share screen and show the dev tools.

"We think you may have been compromised by a Chinese hacker!"

I mute and then die my ass off. Holy shit this is maybe the best thing I've ever done.

My PM, who has seen me use the JSON compression technique before and knows exactly whats up starts telling them about it so they don't freak out. And finally I unmute and manage a, "Guys... I'm standing right here." between gasped laughter.

If only it was more common to use video in these calls because I WISH I could have seen their faces.

Anyway, they calmed their attitude down, we told them how to decompress the localStorage, and then they still didn't find jack shit because i'm a fucking badass and even after we gave them keys to the login and gave them keys to my secret localStorage it only led to AWS Cognito protected async calls.

Anyway, that's the story of how I became a "Chinese hacker" and made a room full of penetration testers look like morons with a (reasonably) simple JS trick.

Hi,
I'm not a ranty person so I never actually thought I'd post anything here but here it goes.

From the beginning.
We use ancient technologies. PHP 5.2, Symfony 1.2 and a non RFC complient SOAP with NO documentation.
A year ago We've been thrown a new temporary project. An VOIP app for every OS.
That being iOS, Android, MAC, PC, Linux, Windows mobile. With a 3 month deadline. All that thrown at 4 PHP developers. The idea being that They'll take it, sign the delivery protocol, everyone happy. No more updates for the app needed. They get their funds they needed the app for and we get paid.

Fast forward to today...

Our dev team started the year with great news that We'll most likely have to create a new project. Since the amount of new features would be far greater than current feature set, we managed to finally force our boss to use newer technologies (ie. seperate backend symfony4 PHP7+/frontend react, rest api and so on). So we were ecstatic to say the least. With preestimates aimed at a minimum 3 month development period. Since we're comfortable with everything that needs to be done.
Two days later our boss came to me that one of our most annoying clients needs a new feature. Said client uses ancient version written on a napkin because They changed half of the specification 2 weaks before deadline in a software made not by a developer but some sysadmin who didn't know anything. His MVC model was practically VVV model since he even had sql queries in some views. Feature will take 3 days - fixing everything that will break in the meantime - 1-2 months.
F*** it, fine. A little overtime won't kill me.
Yesterday boss comes again... Apparently someone lost a delivery protocol for a project we ended that half a year ago. Whats even better at the time when we asked for hardware to test we never got any. When we asked about any testing enviornment - nothing. The app being SEMI-stable on everything is an overstatement but it was working on the os'es available at the time. Since the client started testing now again, it turns out that both Android app does not work on 8.1/9 and the iOS app does not work on ios12. The client obviously does not want to pay and we can do little with it without the protocol, other than rewriting the apps.
It will take months at least since all of those apps were written by people that didn't know neither the OS'es nor the languages. For example I started writing the iOS one in swift. Only to learn after half of the development time, that swift doesn't like working by C Library rules and I had to use ObjC also. With some C thrown in due to the library. 3 unknown languages, on an unknown platform in 3 months. I never had any apple device in my hand at that time nor do I intend to now. I'm astonished it worked out then. It was a clusterf**k of bad design and sticking everything together with deprecated apis and a gum. So I'll have to basically fully rewrite it.

If boss decides we'll take all those at the same time I'll f***ing jump of a bridge.

Me: Right, its Monday, time for a fresh start. Things have been unbearable, but i've nowhere else to go just yet. I gotta just dig deep, ignore everything bad and just get it done, It's all about positivity right? Lets just ignore the little things and keep moving.

*My morning so far, 2 hours in*

Remote dev: (timezone 5 hours earlier than me) Hey so whats the plan for this quarter?

Me: ... I posted a big detailed plan in the group chat on Friday night so you wouldn't be delayed ... but anyway, lets just move on. I need you to work on A, B and C. A is just copying what Android has already done, for B one of the backend guys working next to you is doing this, he'll be able to help you. C is all documented in the ticket.

Remote dev: cool thanks.

Local dev: So I was just chatting with remote dev ... yeah he told me he has no idea what he's suppose to do.

Me: ..... Ok i'll book a video call with him in the morning. Can't do it right now.

==========

Remote dev: Hey i'm helping the BE team do some testing. I found a bug in Android. Homepage says theres no trips. But Offers screen says there is.

Me: Ok so just to confirm, The "available" offers screen has offers to accept, but the white notification on the homepage saying "You have X offers to accept" is not showing up?

Remote dev: Correct!

*debugging for 5 mins*

Remote dev: actually no, the "accepted" offers tab has offers, but the homepage says there are no upcoming offers to work on.

Me: ..... ok, thats very different ... but sure, let me have a look.

Me: Right so the BE are ... again ... sending down expired offers. Looks like the accepted tab isn't catching it and the homepage is.

Remote dev: Right i'll open a ticket for Android.

Me: ... and BE team.

Remote dev: why?

Me: ... because they once again have timezone issues. This keeps causing issues in random places. BE need to fix this everywhere.

Remote dev: right, i'll chat to them and see if they can fix it.

==========

Product: So this ticket xxxxx is clear right?

Me: eh, kind of, so you want us to add feature X to user type A?

Product: correct.

Me: right but I don't see anywhere talking about the time it will take to build the screen for feature X

Product: What do you mean the screen?

Me: ... well, feature X is only accessible on screen Y ... we would have to change screen Y to support user type A ... you know ... so they can ... use the feature

Product: .... hhhhmmm .... i suppose you are right. Well we can't just add screen Y, we'll have to add W and Z, it won't make sense without them.

Me: ... ok sure, but our estimates put us over for this quarter. I don't think we can just add in 3 screens.

Product: No this is a must have.

Me: Ok so we'll have to drop something else.

Product: hhhmmm, don't think we can ... let me get back to you.

==========

Backend team invited me to a meeting at 6am my time on Friday.

==========

... 2 hours into Monday ... there must be vodka around here somewhere

Whats worse than TestDrivenDevelopement?

Starting TDD once a 15K LoC project has well started...

sooo.... here I am testing the entirity of clinl ;_;

Deadline : Friday evening
Testing : There is only prod
Deploy : Friday evening
Why : Boss says customer wants it
Me : . . .

* Teams meeting, doing some early testing with the client *

Dev Team: This is an early build, so please be aware that bugs can occur at this stage. (That's why we didn't want to show it but you insisted)

Client: Sure, don't worry.

* 15 minutes later, first bug happens because some state management at the front end was not tunned yet *

Client: Can you solve this bug?

Dev Team: Sure, just let us replicate it and we come back to you as soon as...

Client: Can you solve it now?

Dev Team: ... Okay, let us ser whats happening and we will come ba...

Client: You don't need to hang up, just solve it.

*Dev Team internal chat*

Me: Fuck
Other dev: Fuck
My leader at the time: Fuuuuuck

PS: This happened in two separate occasions. I hated that project.

If hiring managers really want to hire based on skill, what they should be doing is testing for one thing:

The ability to take a specification, written in general language, notice deficiency, communicate with the 'client' (manager) to hash out what needs done, and the (explicit) ability to read documentation on libraries or tooling outside the dev's core skillset.

If a dev can read a spec, talk to a client to work out whats lacking, and then identify what they need to know and where to find it, thats 90% of the skills they need from what I can see.

tl;dr version of it, is they should be explicit about the requirements for reading/implementing specs and finding the correct documentation.
Something along the lines of
"can you form your letters? Are you able to follow instructions on the back of a cake box? Then there may be a position waiting for you!"

Our employee management system, for some reason, stored Testlists (I work in QA) linked to the user accounts that created them. Now after an colleague who worked there for five years left pretty much all our data was suddenly down the drain and nobody backed the fricking server up because, hey, whats the fun in that. Now all the tests need to be rewritten and other than the whole gui test automation of our product, maintenance of the same for another product, manually testing dev issues and training my new code monkeys to frickin not commit non working code to the trunk I have now also "Make a better Employee management system" (roughly translated those are the specs I've got) on my plate... I can remember back to the care free days of just before my boss asked me if I wanted to try to automate some of the test cases... How did I ever survive this paralyzing tranquility. Ha, surprise.
!rant, I fucking love the stress and juggling a shit ton of problems at the same time keeps ine on edge.

Why won't you just approve my PR???

Whats wrong with you?!

I don't understand your cryptic one-sentence feedback. I'm not even sure you understand what you're asking yourself.

What the hell does "make it a transaction" mean? Don't give me pseudo-code examples that don't even work fucking asshole!

Its a small change that does NOT need a canary build dammit. Don't go testing the ORM, its a goddamn standard library. Why does working with you make everything so complicated?!?!

The code fucking works! There is no need to make it comply to your specific tastes goddamn it. Working with you is like pulling teeth!

/endrant

ideal sprint fallacy.
total days 10 , total hours(excluding breaks ) 8 hrs per day= 80 hrs per dev

code freeze day = day 8, testing+ fixing days : 8,9,10. release day : day 10
so ideal dev time = 7days/56 hr
meetings= - 1hr per day => 49 hrs per dev

- 1 day for planning i.e d1 . so dev time left . 6 days 42 hrs.
-----------
all good planning. now here comes the messups

1. last release took some time. so planning could not happen on d1. all devs are waiting. . devtime = 5 days 35 hrs.

2. during planning:
mgr: hey devx what's the status on task 1?
d: i integrated mock apis. if server has made the apis, i will test them .
mgr : server says the apis are done. whats your guestimate for the task completion?
d : max 1-2 hrs?
m : cool. i assign you 4 hrs for this. now what about task 2?
d : task told to me is done and working . however sub mgr mentioned that a new screen will be added. so that will take time
m : no we probably won't be taking the screen. what's your giestimate?
d : a few more testing on existing features. maybe 1-2 hrs ?
m: cool
another 4 hrs for u. what about task 3?
d : <same story>
m : cool. another 4 hrs for u. so a total of 12 hrs out of 35 hrs? you must be relaxed this sprint.
d : yeah i guess.
m cool.
-------
timelines.
d1: wasted i previous sprint
d2 : sprint planning
d3 : 3+ hrs of meetings, apis for task 1 weren't available sub manager randomly decided that yes we can add another screen but didn't discussed. updates on all 3 tasks : no change in status
d4 : same story. dev apis starts failing so testing comes to halt.
d5 : apis for task1 available . task 3 got additional improvement points from mgr out of random. some prod issue happens which takes 4+ hrs. update on tasks : some more work done on task 3, task 1 and 2 remains same.
d6 : task1 apis are different from mocks. additionally 2 apis start breaking and its come to know thatgrs did not explain the task properly. finally after another 3+ hrs of discussion , we come to some conclusions and resolutions
d7 : prod issue again comes. 4+ hrs goes into it . task 2 and 3 are discussed for new screen additiona that can easily take 2+ days to be created . we agree tot ake 1 and drop 2nd task's changes i finish task 2 new screens in 6 hrs , hoping that finally everything will be fine.

d8 : prod issue again comes, and changes are requested in task 2 and 3

day 9 build finally goes to tester
day 10 first few bugs come with approval for some tasks
day 11(day 1 of new sprint) final build with fixes is shared. new bugs (unrelated to tasks. basically new features disguised as bugs) are raised . we reject and release the build.

day 2 sprint planning
mgr : hey dev x, u had only 12 hrs of work in your plate. why did the build got delayed?

🥲🫡

So... I take over this one ticket to test... the ticket mentions some visual component popping up when a button is clicked. It says there is a success and a failure message. The title of the story also mentions another functionality.

I start testing and some fellow QA asks me why I'm testing in this environment. Turns out, three people are sharing one environment and three different things are deployed...

I ask the dev whats going on because I heard there are multiple people deploying stuff...
He just tells me "oh, my changes are deployed I just checked".
I tell him that it's not about that but about communication and testing one thing at the time. Then I tell him, that I wouldn't test until his stuff is the only stuff there.

Some time later he hits me up again, now with the env to himself.

I test and quickly I see, that there is only the positive message even when I make sure that the backend is not reachable. I tell the dev what I found and he tells me "oh no, it's just the implementation of the popup thing, it's just frontend for now"...
I tell him, that the ticket should say so.
No answer for like 1-2 hours. Then I get an "ok".
End of the day.

Next day I come in and the fellow QA tells me, that the dev asked him to test the ticket.
I ask him if he changed anything about the scope of the ticket, he says no...
I'm like "ok... know what... begin testing and then tell him what I already told him".
So he's testing and then tells him again to update the scope.

Later in the daily the the dev's update is besically "they won't test my ticket..."

It would have taken him like 1 fucking minute to update the ticket...

The whole QA team was always trying to being helpful and even when the tickets where sometimes not 100% clear we always made it work... but now we are more and more going towards "MR does not meet ticketdescription, fix it" and "I don't care if its just a small thing... fix it and then come back to me"...

Seriously frustrating some times...

User :
i've just done testing the system, based from one of the testing data i inserted, the procedure still isn't correct

Me :
- Desperately looking for whats wrong in the procedure -

User :
Oh, nevermind, the testing data itself is not correct

Also ME :
ASDJAHSGDUqa QY(^E*Q^w^EQV%&ABDYDTA^R6b ^#E%&W QE& !!!!!!!!!!!!!

Whats the best way for localhost mysql testing?

How do you deal when you are overpromising and underdelivering due to really shitty unpredictable codebase? Im having 2-3 bad sprints in a row now.

For context: Im working on this point of sale app for the past 4 months and for the last 3 sprints I am strugglig with surprises and edgecases. I swear to god each time I want to implement something more complex, I have to create another 4-5 tickets just to fix the constraints or old bugs that prevent my feature implementation just so I could squeeze my feature in. That offsets my original given deadlines and its so fucking draining to explain myself to my teamlead about why feature has to be reverted why it was delayed again and so on.

So last time basically it went like this: Got assigned a feature, estimated 2 weeks to do it. I did the feature in time, got reviewed and approved by devs, got approved by QA and feature got merged to develop.

Then, during regression testing 3 blockers came up so I had to revert the feature from develop. Because QA took a very long time to test the feature and discover the blockers, now its like 3 days left until the end of the sprint. My teamlead instantly started shitting bricks, asked me to fix the blockers asap.

Now to deal with 3 blockers I had to reimplement the whole feature and create like 3 extra tickets to fix existing bugs. Feature refactor got moved to yet another sprint and 3 tickets turned into like 8 tickets. Most of them are done, I created them just to for papertrail purposes so that they would be aware of how complex this is.

It taking me already extra 2 weeks or so and I am almost done with it but Im going into really deep rabbithole here. I would ask for help but out of other 7 devs in the team only one is actually competent and helpful so I tried to avoid going to him and instead chose to do 16 hour days for 2 weeks in a row.

Guess what I cant sustain it anymore. I get it that its my fault maybe I should have asked for help sooner.

But its so fucking frustrating trying to do mental gymnastics over here while majority of my team is picking low hanging fruit tasks and sitting for 2 weeks on them but they manage to look good infront of everyone.

Meanwhile Im tryharding here and its no enough, I guess I still look incompetent infront of everyone because my 2 weeks task turned into 6 weeks and I was too stubborn to ask for help. Whats even worse now is that teamlead wants me to lead a new initiative what stresses me even more because I havent finished the current one yet. So basically Im tryharding so much and I will get even extra work on top. Fucking perfect.

My frustration comes from the point that I kinda overpromised and underdelivered. But the thing is, at this point its nearly impossible to predict how much a complex feature implementation might take. I can estimate that for example 2 weeks should be enough to implement a popup, but I cant forsee the weird edgecases that can be discovered only during development.

My frustration comes from devs just reviewing the code and not launching the app on their emulator to test it. Also what frustrates me is that we dont have enough QA resources so sometimes feature stands for extra 1-2 weeks just to be tested. So we run into a situation where long delays for testing causes late bug discovery that causes late refactors which causes late deliveries and for some reason I am the one who takes all the pressure and I have to puloff 16 hour workdays to get something done on time.

I am so fucking tired from last 2 sprints. Basically each day fucking explaining that I am still refactoring/fixing the blocker. I am so tired of feeling behind.

Now I know what you will say: always underpromise and overdeliver. But how? Explain to me how? Ok example. A feature thats add a new popup? Shouldnt take usually more than 2 weeks to do my part. What I cant promise is that devs will do a proper review, that QA wont take 2 extra weeks just to test the feature and I wont need another extra 2 weeks just to fix the blockers.

I see other scrum team devs picking low hanging fruit tasks and sitting for 2 weeks on them. Meanwhile Im doing mental gymnastics here and trying to implement something complex (which initially seemed like an easy task). For the last 2 weeks Im working until 4am.

Im fucking done. I need a break and I will start asking other devs for help. I dont care about saving my face anymore. I will start just spamming people if anything takes longer than a day to implement. Fuck it.

I am setting boundaries. 8 hours a day and In out. New blockers and 2 days left till end of the sprint? Sorry teamlead we will move fixes to another sprint.

It doesnt help that my teamlead is pressuring me and asking the same shit over and over. I dont want them to think that I am incompetent. I dont know how to deal with this shit. Im tired of explaining myself again and again. Should I just fucking pick low hanging fruit tasks but deliver them in a steady pace? Fucking hell.