Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
![](/static/devrant/img/pipeless-devrant-banner-white.png)
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "wrong metrics"
-
worst experience with a manager was the (female) one who got away with harassing a female employee for YEARS and thought it was ok because she thought they were friends.
She also retaliated (in ways that were hard to prove), played favorites, didn't know metrics or general business sense, couldn't do her employee's jobs but loved to tell them how they were wrong all the time, and then when she was fired, after the female coworker finally reported the ass slapping and the harassment based on disability, she went back into employee only areas with client personal information and thought taking selfies was ok (she was very quickly corrected and had to wipe her phone and memory card to ensure she hadn't been doing it when she wasn't caught, which cameras later proved she had been).
(I only sound calm. I will take a needle to her probably fake boobs and deflate them with glee if I ever see her again)4 -
You probably know the "marshmellow experiment": have one marshmellow now, or delay the gratification by some time, then get two. What the experiment is supposed to measure is something like intelligence or impulse control.
Hot take: what it also measures, and much more so when it comes to reality, is trust. If I don't trust the other side to be both able and willing to deliver on the promise later, I will rather secure the smaller reward right now.8 -
When I was in college OOP was emerging. A lot of the professors were against teaching it as the core. Some younger professors were adamant about it, and also Java fanatics. So after the bell rang, they'd sometimes teach people that wanted to learn it. I stayed after and the professor said that object oriented programming treated things like reality.
My first thought to this was hold up, modeling reality is hard and complicated, why would you want to add that to your programming that's utter madness.
Then he started with a ball example and how some balls in reality are blue, and they can have a bounce action we can express with a method.
My first thought was that this seems a very niche example. It has very little to do with any problems I have yet solved and I felt thinking about it this way would complicate my programs rather than make them simpler.
I looked around the at remnants of my classmates and saw several sitting forward, their eyes lit up and I felt like I was in a cult meeting where the head is trying to make everyone enamored of their personality. Except he wasn't selling himself, he was selling an idea.
I patiently waited it out, wanting there to be something of value in the after the bell lesson. Something I could use to better my own programming ability. It never came.
This same professor would tell us all to read and buy gang of four it would change our lives. It was an expensive hard cover book with a ribbon attached for a bookmark. It was made to look important. I didn't have much money in college but I gave it a shot I bought the book. I remember wrinkling my nose often, reading at it. Feeling like I was still being sold something. But where was the proof. It was all an argument from authority and I didn't think the argument was very good.
I left college thinking the whole thing was silly and would surely go away with time. And then it grew, and grew. It started to be impossible to avoid it. So I'd just use it when I had to and that became more and more often.
I began to doubt myself. Perhaps I was wrong, surely all these people using and loving this paradigm could not be wrong. I took on a 3 year project to dive deep into OOP later in my career. I was already intimately aware of OOP having to have done so much of it. But I caught up on all the latest ideas and practiced them for a the first year. I thought if OOP is so good I should be able to be more productive in years 2 and 3.
It was the most miserable I had ever been as a programmer. Everything took forever to do. There was boilerplate code everywhere. You didn't so much solve problems as stuff abstract ideas that had nothing to do with the problem everywhere and THEN code the actual part of the code that does a task. Even though I was working with an interpreted language they had added a need to compile, for dependency injection. What's next taking the benefit of dynamic typing and forcing typing into it? Oh I see they managed to do that too. At this point why not just use C or C++. It's going to do everything you wanted if you add compiling and typing and do it way faster at run time.
I talked to the client extensively about everything. We both agreed the project was untenable. We moved everything over another 3 years. His business is doing better than ever before now by several metrics. And I can be productive again. My self doubt was over. OOP is a complicated mess that drags down the software industry, little better than snake oil and full of empty promises. Unfortunately it is all some people know.
Now there is a functional movement, a data oriented movement, and things are looking a little brighter. However, no one seems to care for procedural. Functional and procedural are not that different. Functional just tries to put more constraints on the developer. Data oriented is also a lot more sensible, and again pretty close to procedural a lot of the time. It's just odd to me this need to separate from procedural at all. Procedural was very honest. If you're a bad programmer you make bad code. If you're a good programmer you make good code. It seems a lot of this was meant to enforce bad programmers to make good code. I'll tell you what I think though. I think that has never worked. It's just hidden it away in some abstraction and made identifying it harder. Much like the code methodologies themselves do to the code.
Now I'm left with a choice, keep my own business going to work on what I love, shift gears and do what I hate for more money, or pivot careers entirely. I decided after all this to go into data science because what you all are doing to the software industry sickens me. And that's my story. It's one that makes a lot of people defensive or even passive aggressive, to those people I say, try more things. At least then you can be less defensive about your opinion.53 -
Dev: We have a data quality problem. Combined with data silos, we can't trust the numbers. We need to agree on metric definitions as a company.
CTO: (Points a PM)Help him on the metrics he has defined. We'll discard any other alternate definitions inside the company.
PM: You can start with these. (Throws a bunch of metrics without their definitions. Doesn't respond to questions regarding definitions.)
After a week of generating aggregates,
PM: The numbers seem wrong.
Fuck I know the numbers will be wrong. Because you didn't give me the definitions and I'll have to deduce them myself.
Seriously guys, how do you deal with PMs who don't cooperate in the requirement analysis?8 -
This is the third part of my ongoing series "The Ballad of the Six Witchers and the Undocumented Java Tool".
In this part, we have the massive Battle of Sparks and Storms.
The first part is here: https://devrant.com/rants/5009817/...
The second part is here: https://devrant.com/rants/5054467/...
Over the last couple sprints and then some, The Witcher Who Writes and the Butchers of Jarfile had studied the decompiled guts of the Undocumented Java Beast and finally derived (most of) the process by which the data was transformed. They even built a model to replicate the results in small scale.
But when such process was presented to the Priests of Accounting at the Temple of Cash-Flow, chaos ensued.
This cannot be! - cried the priests - You must be wrong!
Wrong, the Witchers were not. In every single test case the Priests of Accounting threw at the Witchers, their model predicted perfectly what would be registered by the Undocumented Java Tool at the very end.
It was not the Witchers. The process was corrupted at its essence.
The Witchers reconvened at their fortress of Sprint. In the dark room of Standup, the leader of their order, wise beyond his years (and there were plenty of those), in a deep and solemn voice, there declared:
"Guys, we must not fuck this up." (actual quote)
For the leader of the witchers had just returned from a war council at the capitol of the province. There, heading a table boarding the Archpriest of Accounting, the Augur of Economics, the Marketing Spymaster and Admiral of the Fleet, was the Ciefoh Seat himself.
They had heard rumors about the Order of the Witchers' battles and operations. They wanted to know more.
It was quiet that night in the flat and cloudy plains of Cluster of Sparks and Storms. The Ciefoh Seat had ordered the thunder to stay silent, so that the forces of whole cluster would be available for the Witchers.
The cluster had solid ground for Hive and Parquet turf, and extended from the Connection River to farther than the horizon.
The Witcher Who Writes, seated high atop his war-elephant, looked at the massive battle formations behind.
The frontline were all war-elephants of Hadoop, their mahouts the Witchers themselves.
For the right flank, the Red Port of Redis had sent their best connectors - currency conversions would happen by the hundreds, instantly and always updated.
The left flank had the first and second army of Coroutine Jugglers, trained by the Witchers. Their swift catapults would be able to move data to and from the JIRA cities. No data point will be left behind.
At the center were thousands of Sparks mounting their RDD warhorses. Organized in formations designed by the Witchers and the Priestesses of Accounting, those armoured and strong units were native to this cloudy landscape. This was their home, and they were ready to defend it.
For the enemy could be seen in the horizon.
There were terabytes of data crossing the Stony Event Bridge. Hundreds of millions of datapoints, eager to flood the memory of every system and devour the processing time of every node on sight.
For the Ciefoh Seat, in his fury about the wrong calculations of the processes of the past, had ruled that the Witchers would not simply reshape the data from now on.
The Witchers were to process the entire historical ledger of transactions. And be done before the end of the month.
The metrics rumbled under the weight of terabytes of data crossing the Event Bridge. With fire in their eyes, the war-elephants in the frontline advanced.
Hundreds of data points would be impaled by their tusks and trampled by their feet, pressed into the parquet and hive grounds. But hundreds more would take their place. There were too many data points for the Hadoop war-elephants alone.
But the dawn will come.
When the night seemed darker, the Witchers heard a thunder, and the skies turned red. The Sparks were on the move.
Riding into the parquet and hive turf, impaling scores of data points with their long SIMD lances and chopping data off with their Scala swords, the Sparks burned through the enemy like fire.
The second line of the sparks would pick data off to be sent by the Coroutine Jugglers to JIRA. That would provoke even more data to cross the Event Bridge, but the third line of Sparks were ready for it - those data would be pierced by the rounds provided by the Red Port of Redis, and sent back to JIRA - for good.
They fought for six days and six nights, taking turns so that the battles would not stop. And then, silence. The day was won, all the data crushed into hive and parquet.
Short-lived was the relief. The Witchers knew that the enemy in combat is but a shadow of the troubles that approach. Politics and greed and grudge are all next in line. Are the Witchers heroes or marauders? The aftermath is to come, and I will keep you posted.4 -
I recently tried to apply the same data analytics rationale that I use at work to my personal life. This is not a rant, it is more like an data storytelling of an actual use case I would like some input on.
I set a goal - gotta thin up a bit and calm down my ticker - and got a (almost unreasonably expensive) field expert consultant to yell at me about it for a couple hours.
I unravel the metrics - there is like a million weight-related KPIs and most say nothing at all. I have never seen an non-infrastructure measurable subject that could not be resumed to 2-5 performance metrics. I got overall weight, how well my nine-years-old business suit fits me, heart rate, and day-after relative muscle pain (it will make sense soon).
Then its data-pipeline time. I bought a cheap weight scale and smartwatch, and every morning I input the data in an app. Yes, I try to put on the suit every morning. It still does not fit.
After establishing a baseline, I tried to fit different approaches. Doing equipment-free exercises, going to the gym, dieting. None was actually feasible in the long run, but trying different approaches does highlight the impacts and the handling profile of each method.
Looking at the now-gathered data, one thing was obvious - can't do dieting because it is not doable to have a shopping list and meals for me and another for the family.
Gym is also off the table - too much overhead. I spend more time on the trip there and back than actually there.
And home exercise equipment is either super crappy or very expensive. But it is also the most reasonable approach.
So it is solutions time. I got a nice exercise bycicle (not a peloton), an yoga mat (the wife already had that one) and an exercise program that uses only those two resources. Not as efficient without dieting, not as measurable and broad as the gym, but it fits my workflow. Deploy to production!
A few months pass and the dataset grows. The signal is subtle but has support - it works! The handling, however, needs improvement, since I cannot often enough get with the exercise program. Some mornings are just after some hard days.
I start thinking about what else I can improve in the program, but it is already pretty lean and full of compromises.
So I pull an engineer and start thinking about the support systems and draft profile. What else could be draining my willpower and morning time?
Chores. Getting the kids ready for school, firing up the moka pot, setting the off-brand roomba, folding the overnight-dried clothes, cooking breakfast, doing the dishes, cleaning the toilets. All part of my morning routine. It might benefit from some automation.
Last month I got that machine our elders call "wasteful" and "useless crap lazy entitled Americans invented because they feel oh-so-insulted for simply doing something by hand like everyone always did" - a "dish-washer".
Heh, I remember how hard was to convince my mother-in-law that an remote-controled electric garage door would not make she look like an spoiled brat.
Still to early to call, but I think that the dishwasher just saved me about 25 mins every morning. It might be enough to save willpower for me to do more exercise.
This is all so reflective of all data analytics cases really are out in the wild - the analytics phase seems so small compared to the gathering and practical problem-solving all around. And yet d.a. is what tells you that you are doing the wrong thing all along. Or on what you should work next.7 -
Applying Occam's razor and I might be wrong..
Hiring a candidate and job hunt, both are fucking exhaustive process.
We, as a human race, have aimed for Moon and Mars but are unable to solve the problem at hand which can save millions of hours each year reflecting in immediate cost savings.
Here's my (idealistic) solution:
A product to connect job seekers and recruiters eliminating all the shitty complexities.
LinkedIn solved it, but then hired some PMs who started chasing metrics and bloated the fuck out of the product.
Here are some features of the product I am envisioning:
1. Job seeker signs up and builds their entire profile.
2. Ability to add/remove different sections (limited choices like certifications, projects, etc.), no custom shit allowed because each will have their own shit.
3. By default accept GDPR, Gender Identity, US equality laws, Vetran, yada yada..
4. No resume needed. Profile serves as resume. Eliminate the need to build a resume in word or resume builders.
5. Easy updates and no external resume, saves the job seeker time and gives a standard structure to recruiters to scan through eliminating cognitive load.
6. Recruiters can post their jobs and have similar sections (limited categories again).
7. Add GDPR, Vetran, etc. check boxes need basis.
8. No social shit. Recruiters can see profiles of job seekers and job seekers can see jobs. Period.
9. Employee working in Google? Awesome. Will not show Google recruiters thier profile and employee such job posts.
10. No need to apply or hunt heads. System will automatch and recommend because we are fucking in AI generation and how hard it is to match keywords!!
11. Saves job seekers and recruiters a fuck ton of time hunting the best fit.
12. This system gets you the best job that fits your profile.
Yes, there are flaws in this idea.
Yes, not all use cases are covered.
Yes, shit can be improved and this is hypothetical.
But hey! Surely doable with high impact than going on Moon or Mars right now.
Start-up world has lost its way.12 -
So the project I work on basically has to talk to a 3rd party plugin, through a 3rd party framework. The 3rd party plugin is a black box. This conversation happened:
Software guy: so we aren't sure what is breaking the thing. It's either us or the plugin, but it's probably both.
Systems guy: well then if we aren't sure then why are we writing an issue for it.
SWG: because we aren't sure but we know we are doing at least something that contributes. We read int X from a table and put it into a float. X doesn't perfectly represent in a float. It comes out X.0001. Then they take it and when it comes back it comes back as Y.0001. We cram it into an int so it becomes Y, we compare it to X which is really X.0001 and it comes back invalid.
SG: well as long as we are sending them the right number . . .
SWG: but we aren't sending them the right number. They are expecting X not X.0001. Then they send us back Y.0001 but it should be X so it's wrong.
SG: so they're giving us the wrong return value.
SWG: yes, but because we're giving them the wrong number.
SG: well not exactly . . .
SWG: yes exactly. It is off by .0001 because of floating point math.
SG: well . . .
Me: look it doesn't matter how it's breaking. But it IS broken. Which is why we're filling out the damn problem report. THEY ARE EDITABLE. We talked to the customer and gave them the risk assessment. They don't care. It happens rarely any way.
SG: then can we lower the severity?
Me: no. Severity doesn't relate to risk. That is a whole different process. Severity assumes it has already happened. It's a a high severity.
SG: but the metrics.
Me: WE GIVE THE METRICS TO THE CUSTOMER. WE TALKED TO THE CUSTOMER. THEY DON'T GIVE A SHIT.
And that was how I spent Wednesday wondering how a level 4 lead systems engineer got his job. How many push ups did he do? What kind of juice did he drink?2 -
aagh fuck college subjects. over my last 4 years and 7 sems in college, i must have said this many times : fuck college subjects. But Later i realize that if not anything, they are useful in government/private exams and interviews.
But Human computer Interaction? WHAT THE FUCK IS WRONG WITH THIS SUBJECT???
This has a human in it, a comp in it, and interaction in it: sounds like a cool subject to gain some robotics/ai designing info. But its syllabus, and the info available on the net , is worse than that weird alienoid hentai porn you watched one night( I know you did).
Like, here is a para from the research paper am reading, try to figure out even if its english is correct or not:
============================
Looking back over the history of HCI publications, we can see how our community has broadened intellectually from its original roots in engineering research and, later, cognitive science. The official title of
the central conference in HCI is “Conference on Human Factors in Computing Systems” even though we usually call it “CHI”. Human factors for interaction originated in the desire to evaluate whether pilots
could make error-free use of the increasingly complex control systems of their planes under normal conditions and under conditions of stress. It was, in origin, a-theoretic and entirely pragmatic. The conference and field still reflects these roots not only in its name but also in the occasional use of simple performance metrics.
However, as Grudin (2005) documents, CHI is more dominated by a second wave brought by the cognitive revolution. HCI adopted its own amalgam of cognitive science ideas centrally captured in Card, Moran & Newell (1983), oriented around the idea that human information processing is deeply analogous to computational signal processing, and that the primary computer-human interaction task is enabling communication between the machine and the person. This cognitive-revolution-influenced approach to humans and technology is what we usually think of when we refer to the HCI field, and particularly that represented at the CHI conference. As we will argue below, this central idea has deeply informed the ways our field conceives of design and evaluation.
The value of the space opened up by these two paradigms is undeniable. Yet one consequence of the dominance of these two paradigms is the difficulty of addressing the phenomena that these paradigms mark as marginal.
=============================7 -
Wanted to add alerting for systemd services in Prometheus today, which spontaneously turned out to be a huge pain in the lower human backend.
For some reason, on Ubuntu 16.04 systemd adds services without unit files for software, that isn't even installed on the damn server (in this case for mysql-server / mysql-common and mysql-client are installed) and lists them as "not-found" and "inactive". The prometheus node exporter that we use, has a little bug in the systemd collector that makes sure that the states of *all* services are collected - even those without a unit file.
so those metrics are pulled by prometheus and now I have to take with those faulty metrics in the condition logic of the alert, because I'm trying to trigger that one on a service which is listed with state "active" = 0 or "failed" = 1.
now guess. right! If the unit file doesn't exist, the regarded systemd service is marked as "inactive", which is another possible state of the metrics in the node exporter. the problem is that the value 1 for state "inactive" means, that "active" has the value 0 (not even wrong) and the alert is triggered.
so systemd fucks up somehow, the node exporter collector fucks up because systemd fucked up and I have to unfuck this with some crazy horse shit logic. w.t.f. to that.
the only good news is, that it works like a charm on Ubuntu 18.04, as far, as I can tell.
while writing this little rant, I thought of a solution.
I could try to change the alert condition to state "active" = 0 AND "failed" = 1.. but that will wait till tomorrow.
one does not simply patch monitoring conditions at midnight..3 -
so i'm sitting here staring inwardly at the learning rate optimizer...
i think it works
but i find myself wanting to scream at the nuances of the method being hidden from me
I know its probably fairly simple.
i want to write my own.
i want to plot neat graphs that give me metrics at the learning results for each epoch showing how much closer the values are getting to the training data some neat spiral of values and lines and flashy too.
but i feel.
...welll
strangely lackluster and a tad paralyzed for some reason. partly because it feels like i've done this all before... sigh.
on the topic of things I already did.
https://en.wikipedia.org/wiki/...
can you believe they made this bullshit into A TV SHOW ? IN THIS WEIRD ASS HYPERSENSITIVE ENVIRONMENT ? THIS RAPE WEIRDOS WET DREAM ? 5 SEASONS AT LEAST.
GOD WHAT IS WRONG WITH THIS COUNTRY ? ITS EITHER ADULT MEN WATCHING RUGRATS OR THIS SHIT !2 -
I just want to test my PHP code using phpunit on GitLab, what am I doing wrong?
image: php:latest
before_script:
# Install & enable Xdebug for code coverage reports
- pecl install xdebug
- docker-php-ext-enable xdebug
# Run our tests
# If Xdebug was installed you can generate a coverage report and see code coverage metrics.
test:
script:
- phpunit Test.php4