Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "an estimate is still an estimate"
-
Me: Optimize a sort & match method in backend because users complain it's a bit slow.
Coworker: These algorithms are both O(n), so they're identical *closes PR*
Me: *start zoom call* "Heeeeeeeeeey Iiiiiiiiiii wouuuuuuuld liiiiiiiiike toooooo diiiiiisscuuuuus thaaaaaaaat puuuuuuulllll reeeeeequuuueeest yooooouuuuu cloooooossseeeed"
Coworker: "wtf are you doing, why are you talking so slow"
Me: "No matter whether I talk fast or slow, the information still reaches you in O(n) time, so why are you complaining"
I fucking hate it when people misunderstand the purpose of (or abuse) big O notation. It's an estimate of how an algorithm SCALES once the set increases in size, in which case you leave out both less significant terms and constant factors.
But those terms and factors are important when you're talking about the DIRECT PERFORMANCE of the algorithm on fixed-size sets, instead of SCALING to larger sets.
1n and 10n are both O(n), but 10x performance on a job that used to take 10 minutes is still significant.19 -
Boss: I need to know how much resources a server would need to serve 20000 users at the same time
Me: Sure, can I see the webapp?
B: There's not one yet
M: Ok, can I see its documentation then?
B: There is none
M: But ot depends on the programming language, libraries used, what kin-
B: It's an e-commerce. Try browsing Amazon and see how much they nees to handle the page requests.
Me: *still processing* wh-
Boss: I have to give an estimate to a client within 30 minutes. Hurry.
So.... Uh... I guess i have to hack Amazon now?7 -
Finally decided to quit from my current job. Fuck it. They still don't understand that an estimate is an approximation. They still don't understand that I have to fix all the shit made by all the contractors they hire and pay much more than me to implement solutions that work only until they leave the building, something that many barely understand but pretend to be experts in. I quit because of the managers that have no clue about what's happening although I stress them to make changes.
Should I care less about the product an just ship shit? Should I just do my tasks first and stop really helping others but pretend to do so?
Fuck it. I've tried to get it right they want it wrong but in a nice box. I'm an engineer not a fucking magician.12 -
In my country, almost every college student is expected to finish their degree and apply for an internship, with some universities forcing them to do it and making it a requirement to finish their studies.
Now, this wouldn't be so bad if almost every internship employer in the country didn't expect you to work for free. Seriously, I can estimate 80% of the internships pay you NOTHING. WTF.
Fortunately this is not the case for CS, but every time I tell somebody I recently started an internship, they will ask me: "Oh, but they don't pay you anything, do they?". Of course they pay me! I wouldn't be going to an office every day for 4 hours to do someone else's work if they didn't!!
Why the fuck is it even legal to employ somebody and not pay them a cent, just because "it will look good on your resume"?? And why do people still accept this shit??
Is is like that on other countries as well?2 -
I work for a web agency.
Over the last 18 months a company asked us about 5 different quotes for rather minimal changes to their website.
While being minimal changes, estimating costs for them still requires several hours of work for research, meetings, correspondance and writing the damn things. They never even gave us a response (neither positive nor negative), except once where they told us that they wouldn't pay for project management because their instructions are so clear that PM isn't necessary.
As a response to the last one, after several months, they send us a 10 pages long pdf with requirements for a new website (or a "restyle" like they call it, even if it has absolutely nothing in common with the current one).
We inform them that we can't permit ourselves to continue studying new solutions for free and therefore tell them that a detailed offer would cost them something like 300$, and that amount would then be discounted from the eventually accepted job. We also roughly estimate a price range of about 15k - 20k for the new website.
We get an email back, from the CEO (until now it was a secretary), with essentially 3 arguments written in condescending form:
1. he brags about his revenue being over 9 billion $$$ a year, and that being a part of a global holding for which "communication is essential" (sic.) means that they need to coordinate and "can't simply accept an offer" [even if it's 400$, for specific change exactly requested by them, I guess...]
2. 15k is too much [... for the website representing this 9 billion dollar holding on the internet, for which the requirements are written in the 10 pages long pdf]
3. He asks for a meeting
We accept the meeting, we go to their office.
When we arrive there, the secretary informs us that the CEO will not participate. So we talk the her and the head of the "Communication Dept" in videoconference.
I explain them that if the sum, which we thought would be appropriate (~15 - 20k), is too high, the Pareto principle would allow us to, theoretically, achieve about 80% of the features and quality for about 20% of the cost. Their genuine response is:
"So your estimate was wrong! You can do it for much less!".
I try to explain them that the most money in a project goes into "attention to detail".
The "Communication Dept." person, who is "doing this job since too much time" (sic.), refuses to believe and insists that "details" don't exist on the web.
I tell her: "In any kind of work, the more effort you put into something, the better it tends to get, with diminishing returns".
She insists: "I don't understand this".
So now I'm here, doing the 6th offer, free of charge, for a 5k website, for a company that generates 9kkk revenue each year, trying to define a "Definition of Done" that works out.
FML I guess.
Sorry for the long post.7 -
EoS1: This is the continuation of my previous rant, "The Ballad of The Six Witchers and The Undocumented Java Tool". Catch the first part here: https://devrant.com/rants/5009817/...
The Undocumented Java Tool, created by Those Who Came Before to fight the great battles of the past, is a swift beast. It reaches systems unknown and impacts many processes, unbeknownst even to said processes' masters. All from within it's lair, a foggy Windows Server swamp of moldy data streams and boggy flows.
One of The Six Witchers, the Wild One, scouted ahead to map the input and output data streams of the Unmapped Data Swamp. Accompanied only by his animal familiars, NetCat and WireShark.
Two others, bold and adventurous, raised their decompiling blades against the Undocumented Java Tool beast itself, to uncover it's data processing secrets.
Another of the witchers, of dark complexion and smooth speak, followed the data upstream to find where the fuck the limited excel sheets that feeds The Beast comes from, since it's handlers only know that "every other day a new one appears on this shared active directory location". WTF do people often have NPC-levels of unawareness about their own fucking jobs?!?!
The other witchers left to tend to the Burn-Rate Bonfire, for The Sprint is dark and full of terrors, and some bigwigs always manage to shoehorn their whims/unrelated stories into a otherwise lean sprint.
At the dawn of the new year, the witchers reconvened. "The Beast breathes a currency conversion API" - said The Wild One - "And it's claws and fangs strike mostly at two independent JIRA clusters, sometimes upserting issues. It uses a company-deprecated API to send emails. We're in deep shit."
"I've found The Source of Fucking Excel Sheets" - said the smooth witcher - "It is The Temple of Cash-Flow, where the priests weave the Tapestry of Transactions. Our Fucking Excel Sheets are but a snapshot of the latest updates on the balance of some billing accounts. I spoke with one of the priestesses, and she told me that The Oracle (DB) would be able to provide us with The Data directly, if we were to learn the way of the ODBC and the Query"
"We stroke at the beast" - said the bold and adventurous witchers, now deserving of the bragging rights to be called The Butchers of Jarfile - "It is actually fewer than twenty classes and modules. Most are API-drivers. And less than 40% of the code is ever even fucking used! We found fucking JIRA API tokens and URIs hard-coded. And it is all synchronous and monolithic - no wonder it takes almost 20 hours to run a single fucking excel sheet".
Together, the witchers figured out that each new billing account were morphed by The Beast into a new JIRA issue, if none was open yet for it. Transactions were used to update the outstanding balance on the issues regarding the billing accounts. The currency conversion API was used too often, and it's purpose was only to give a rough estimate of the total balance in each Jira issue in USD, since each issue could have transactions in several currencies. The Beast would consume the Excel sheet, do some cryptic transformations on it, and for each resulting line access the currency API and upsert a JIRA issue. The secrets of those transformations were still hidden from the witchers. When and why would The Beast send emails, was still a mistery.
As the Witchers Council approached an end and all were armed with knowledge and information, they decided on the next steps.
The Wild Witcher, known in every tavern in the land and by the sea, would create a connector to The Red Port of Redis, where every currency conversion is already updated by other processes and can be quickly retrieved inside the VPC. The Greenhorn Witcher is to follow him and build an offline process to update balances in JIRA issues.
The Butchers of Jarfile were to build The Juggler, an automation that should be able to receive a parquet file with an insertion plan and asynchronously update the JIRA API with scores of concurrent requests.
The Smooth Witcher, proud of his new lead, was to build The Oracle Watch, an order that would guard the Oracle (DB) at the Temple of Cash-Flow and report every qualifying transaction to parquet files in AWS S3. The Data would then be pushed to cross The Event Bridge into The Cluster of Sparks and Storms.
This Witcher Who Writes is to ride the Elephant of Hadoop into The Cluster of Sparks an Storms, to weave the signs of Map and Reduce and with speed and precision transform The Data into The Insertion Plan.
However, how exactly is The Data to be transformed is not yet known.
Will the Witchers be able to build The Data's New Path? Will they figure out the mysterious transformation? Will they discover the Undocumented Java Tool's secrets on notifying customers and aggregating data?
This story is still afoot. Only the future will tell, and I will keep you posted.6 -
Sometimes we woulg get a request which involves adding something or changing something to a rather large and poorly made codebase which me and my lead have not had the time to change.
This b how shit goes:
* the lead gets a call after an email was sent with apparently only 5 secs of response time( inpatient fucks)
* lead calls me in next to his station to listen to the call
* i b listening and shit, not even taking notes and shit, looking all secret weapon and shit.
Texas as fuck.
* lead puts shit on hold and looks at me
Lead: "Allright. You know the codebase as well as I do, what you think?"
Me: pffft gimme 30 mins and Ill whip out yo solution
Lead: we positive on the estimate?
Me: as positive as the Texas Rangers sucking ass but we still love em, fuck the Astros
Lead: there is only room for one team
Me: only one
**fist bump
* goes back to the call:
Lead: yeah its gonna take 2 days at most.
Aaaaaaaaaaaaaaand we do finish them in 30 mins. The trick is in doing it extra fast so we have enough time to fuck around or do some other shit and to make it seem like we do some hard shit. After maybe 6 hours we tell them that we managed to fix it before time.
Texas....as....fuck
Btw me and the lead tall about whatever while we code the stuff, most of the time I do it since my boy has heavy eye problems and I want him to relax. He has been training me a lot in regards to knowing the codebase, before I got here it was only him for two fucking campuses and the man did an outstanding job. My boy got my ass and I got his.
Teamwork, the southern gentleman's way.
Texas.
P.d while coding it he said the one of the file sizes was too big to handle, i said "das what she said" and our female manager said "i heard that".......i could have sworn that she gave me a lil wink. Well damn.8 -
In last episode of "How SystemD screwed me over", we talked about Systemd's PrivateTMP and how it stopped me from generating SSL certificates.
In today's episode - SystemD vs CGroups!
Mister Pottering and his team apparently felt that CGroups are underused (As they can be quite difficult to set up), and so decided to integrate them into SystemD by default. As well as to provide a friendlier interface to control their values.
One can read about these interactions in the manual page "systemd.resource-control"
All is cool so far. So what happened to me today?
Imagine you did a major system release upgrade of a production server, previously tested on a standalone server. This upgrade doesn't only upgrade the distribution however, it also includes the switch from SysVInit to SystemD. Still, everything went smooth before, nothing to worry now then, right? Wrong.
The test server was never properly stress-tested. This would prove to be an issue.
When the upgrade finishes, it is 4 AM. I am happy to go to bed at last. At 6 AM, however, I am woken up again as the server's webservices are unavailable, and the machine is under 100% CPU load. Weird, I check htop and see that Apache now eats up all 32 virtual cores. So I restart it, casting it off to some weird bug or something as the load returns to normal.
2 hours later, however, the same situation occurs. This time, I scour all the logs I can, and find something weird - Many mentions that Apache couldn't create a worker thread? That's weird.
Several hours of research and tinkering later, I found out the following:
1 - By default, all processes of a system that runs SystemD are part of several CGroups. One of these CGroups is the PID CGroup, meant to stop a runaway process from exhausting all PIDs/TIDs of a system.
This limit is, by default, set to a certain amount of the total available PIDs. If a process exhausts this limit, it can no longer perform operations like fork().
So now, I know the how and why, but how should I solve this? The sanest option would be to get a rough estimate of just how many threads the Apache webserver might need. This option, though, is harder, than apparent. I cannot just take the MaxRequestsWorkers number... The instance has roughly double the amount of threads already. The cause being, as I found out, the HTTP/2 module, which spawns additional threads that do not count towards this limit. So I have no idea what limit to set.
Or I could... Disable the limit for just the webserver via the TasksAccounting switch. I thought this would work. And it did seem to... Until I ran out of TIDs again - Although systemctl status apache2.service no longer reported the number of tasks or a task limit of the process, the PID CGroup stayed set to the previous limit. Later I found out that I can only really disable the Task Accounting for all the units of a given slice and its parents.
This, though, systemctl somewhat didn't make apparent (And I skimmed the manual, that part was my fault)
So... The only remaining option I had was to... Just set the limit to infinite. And that worked, at last.
It took me several hours to debug this issue. And I once again feel like uninstalling systemd again, in favor of sysvinit.
What did I learn? RTFM, carefully, everything is important, it is not enough to read *half* the paragraph of a given configuration option...
Oh, and apache + http/2 = huge TID sink. -
And that, folks, is why you never do a rush job, no matter how urgent, without an RFP and answering estimate followed by a signed statement of work confirming agreement to the estimate. Even with a prearranged, perpetual contracting agreement. And also why you NEVER deliver the end product without payment. No matter how much you trust your client or believe they will do right by you, process still matters.2
-
Estimates.. First, part of the team makes "high-level" estimates which are based on informal, incomplete, still-evolving specs and an unstable back-end. The project people report the estimates to the client and elevate the status of these inaccurate estimates to that of commitments.
Then, before the "sprint", we review our initial estimates *ahum commitments* in greater (technical) detail. Because there are still a lot of unknowns, we tend to estimate more buffer here (back-end is often not ready, always ping-pong between project people and dev-team about unclear specs, more work than originally expected, and often late modifications to the original spec).
When an estimate becomes more than 50% extra time at the "refinement", we are told: "sorry, we gotta do it in less" and when it doesn't work out, we're kindly asked to spend part of our weekend catching up at 100% pay rate (legally it's 150-200%).
FUCK THIS SHIT
*quotes used abundantly because these terms belong to "agile/scrum" terminology but we're only pretending -
So I'm on PTO for a week, and there is some feature for which I gave the estimations to the other team. That team thinks that eastimates are high and I'm getting pinged to think about it.
First of all they did the overriding of some feature which was required by our team without our knowledge and now when we got know what they did, they are asking us to do the work. I have an estimate and yes they are higher than expected because this feature is on their systems and I don't have complete understanding of it. This is ridiculous though but still I think that I'm wrong somewhere but I know this work is not easy for me, that's why estimations are high.
I'm on leave, I'm not supposed to be working anyway even if it's a small clarification.
Adding more here, that guy tagged another person from my team to do the work as I'm on leave. This guy did some work before too. But this is insane. HOW THE FUCK YOU EXPECT US TO DO IT FOR YOU!! YOU FUCKED IT UP!!!
I feel scared, what if the other guy can actually do the work, people will think I'm just stupid.:(((8 -
How do you deal when you are overpromising and underdelivering due to really shitty unpredictable codebase? Im having 2-3 bad sprints in a row now.
For context: Im working on this point of sale app for the past 4 months and for the last 3 sprints I am strugglig with surprises and edgecases. I swear to god each time I want to implement something more complex, I have to create another 4-5 tickets just to fix the constraints or old bugs that prevent my feature implementation just so I could squeeze my feature in. That offsets my original given deadlines and its so fucking draining to explain myself to my teamlead about why feature has to be reverted why it was delayed again and so on.
So last time basically it went like this: Got assigned a feature, estimated 2 weeks to do it. I did the feature in time, got reviewed and approved by devs, got approved by QA and feature got merged to develop.
Then, during regression testing 3 blockers came up so I had to revert the feature from develop. Because QA took a very long time to test the feature and discover the blockers, now its like 3 days left until the end of the sprint. My teamlead instantly started shitting bricks, asked me to fix the blockers asap.
Now to deal with 3 blockers I had to reimplement the whole feature and create like 3 extra tickets to fix existing bugs. Feature refactor got moved to yet another sprint and 3 tickets turned into like 8 tickets. Most of them are done, I created them just to for papertrail purposes so that they would be aware of how complex this is.
It taking me already extra 2 weeks or so and I am almost done with it but Im going into really deep rabbithole here. I would ask for help but out of other 7 devs in the team only one is actually competent and helpful so I tried to avoid going to him and instead chose to do 16 hour days for 2 weeks in a row.
Guess what I cant sustain it anymore. I get it that its my fault maybe I should have asked for help sooner.
But its so fucking frustrating trying to do mental gymnastics over here while majority of my team is picking low hanging fruit tasks and sitting for 2 weeks on them but they manage to look good infront of everyone.
Meanwhile Im tryharding here and its no enough, I guess I still look incompetent infront of everyone because my 2 weeks task turned into 6 weeks and I was too stubborn to ask for help. Whats even worse now is that teamlead wants me to lead a new initiative what stresses me even more because I havent finished the current one yet. So basically Im tryharding so much and I will get even extra work on top. Fucking perfect.
My frustration comes from the point that I kinda overpromised and underdelivered. But the thing is, at this point its nearly impossible to predict how much a complex feature implementation might take. I can estimate that for example 2 weeks should be enough to implement a popup, but I cant forsee the weird edgecases that can be discovered only during development.
My frustration comes from devs just reviewing the code and not launching the app on their emulator to test it. Also what frustrates me is that we dont have enough QA resources so sometimes feature stands for extra 1-2 weeks just to be tested. So we run into a situation where long delays for testing causes late bug discovery that causes late refactors which causes late deliveries and for some reason I am the one who takes all the pressure and I have to puloff 16 hour workdays to get something done on time.
I am so fucking tired from last 2 sprints. Basically each day fucking explaining that I am still refactoring/fixing the blocker. I am so tired of feeling behind.
Now I know what you will say: always underpromise and overdeliver. But how? Explain to me how? Ok example. A feature thats add a new popup? Shouldnt take usually more than 2 weeks to do my part. What I cant promise is that devs will do a proper review, that QA wont take 2 extra weeks just to test the feature and I wont need another extra 2 weeks just to fix the blockers.
I see other scrum team devs picking low hanging fruit tasks and sitting for 2 weeks on them. Meanwhile Im doing mental gymnastics here and trying to implement something complex (which initially seemed like an easy task). For the last 2 weeks Im working until 4am.
Im fucking done. I need a break and I will start asking other devs for help. I dont care about saving my face anymore. I will start just spamming people if anything takes longer than a day to implement. Fuck it.
I am setting boundaries. 8 hours a day and In out. New blockers and 2 days left till end of the sprint? Sorry teamlead we will move fixes to another sprint.
It doesnt help that my teamlead is pressuring me and asking the same shit over and over. I dont want them to think that I am incompetent. I dont know how to deal with this shit. Im tired of explaining myself again and again. Should I just fucking pick low hanging fruit tasks but deliver them in a steady pace? Fucking hell.4