Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Good Morning!, its time for practiseSafeHex's most incompetent co-worker!
Todays contestant is a very special one.
*sitcom audience: WHY?*
Glad you asked, you see if you were to look at his linkedin profile, you would see a job title unlike any you've seen before.
*sitcom audience oooooooohhhhhh*
were not talking software developer, engineer, tech lead, designer, CTO, CEO or anything like that, No No our new entrant "G" surpasses all of those with the title ..... "Software extraordinaire".
*sitcom audience laughs hysterically*
I KNOW!, wtf does that even mean! as a previous dev-ranter pointed out does this mean he IS quality code? I'd say he's more like a trash can ... where his code belongs
*ba dum tsssss*
Ok ok, lets get on with the show, heres some reasons why "G" is on the show:
One of G's tasks was to build an analytics gathering library for iOS, similar to google analytics where you track pages and events (we couldn't use google's). G was SO good at this job he implemented 2 features we didn't even ask for:
- If the library was unable to load its config file (for any reason) it would throw an uncatchable system integrity error, crashing the app.
- If anything was passed into any of the functions that wasn't expected (null, empty array etc.) it would crash the app as it was "more efficient" to not do any sanity checks inside the library.
This caused a lot of issues as some of the data needed to come from the clients server. The day we launched the app, within the first 3 hours we had over 40k crash logs and a VERY angry client.
Now, what makes this story important is not the bugs themselves, come on how many times have we all done something stupid? No the issue here was G defended all of this as the right thing to do!
.. and no he wasn't stoned or drunk!
G claimed if he couldn't get the right settings / params he wouldn't be able to track the event and then our CEO wouldn't have our usage data. To which I replied:
"So your solution was to not give the client an app instead? ... which also doesn't give the CEO his data".
He got very angry and asked me "what would you do then?". I offered a solution something like why not have a default tag for "error" or "unknown" where if theres an issue, we send up whatever we have, plus the file name and store it somewhere else. I was told I was being ridiculous as it wasn't built to track anything like that and that would never work ... his solution? ... pull the library out of the app and forget it.
... once again giving everyone no data.
G later moved onto another cross-platform style project. Backend team were particularly unhappy as they got no spec of what needed to be done. All they knew was it was a single endpoint dealing with very complex model. There was no Java classes, super classes, abstract classes or even interfaces, just this huge chunk of mocked data. So myself and the lead sat down with him, and asked where the interfaces for the backend where, or designs / architecture for them etc.
His response, to this day frightens me ... not makes me angry, not bewilders me ... scares the living shit out of me that people like this exist in the world and have successful careers.
G: "hhhmmm, I know how to build an interface, but i've never understood them ... Like lets say I have an interface, what now? how does that help me in any way? I can't physically use it, does it not just use up time building it for no reason?"
us: "... ... how are the backend team suppose to understand the model, its types, integrate it into the other systems?"
G: "Can I not just tell them and they can write it down?"
**
I'll just pause here for a moment, as you'll likely need to read that again out of sheer disbelief
**
I've never seen someone die inside the way the lead did. He started a syllable and his face just dropped, eyes glazed over and he instantly lost all the will to live. He replied:
" wel ............... it doesn't matter ... its not important ... I have to go, good luck with the project"
*killed the screen share and left the room*
now I know you are all dying in suspense to know what happened to that project, I can drop the shocking bombshell that it was in fact cancelled. Thankfully only ~350 man hours were spent on it
... yep, not a typo.
G's crowning achievement however will go down in history. VERY long story short, backend got deployed to the server and EVERYTHING broke. Lead investigated, found mistakes and config issues on every second line, load balancer wasn't even starting up. When asked had this been tested before it was deployed:
G: "Yeah I tested it on my machine, it worked fine"
lead: "... and on the server?"
G: "no, my machine will do the same thing"
lead: "do you have a load balancer and multiple VM's?"
G: "no, but Java is Java"
... and with that its time to end todays episode. Will G be our most incompetent? ... maybe.
Tune in later for more practiceSafeHex's most incompetent co-worker!!!31 -
I've always tried to live by the philosophy that you should never burn bridges, but my god some recruiters make it very difficult. I've had a handful of occasions where I've had to type out a nasty email to a recruiter and then just not send it after I realized I wouldn't want it getting around/you never know who they known.
The latest incident where I did this was with a recruiter who emailed me 3 times within a week. But my issue wasn't with the amount of emails or even the fact that he was shopping me frontend positions despite my resume clearly stating I'm a backend dev/data engineer.
My issue was... THE GUY FUCKING REFERRED TO HIMSELF AS "BATMAN". That's right - an adult human being so desperate to get the attention of developers that he set his email name to "Batman", signs his emails "Full name - A.K.A. Batman", and lists his phone number like "BatPhone: xxx.xxx.xxxx"
If I didn't find him so pretentious and he actually sent me a kind of relevant position, what would I do? Pick up the phone and call him and say "hey Batman." Jesus fucking Christ. What an absurd gimmick. Maybe I'm overreacting but it seems so childish.
And you know what, if the guy read my resume and sent me relevant stuff I probably would have said "meh, he's doing good work, if he wants to stand out/be silly whatever." But no, he didn't even look at my skills. Instead he thought 3 shitty emails where he called himself Batman would convince me to write back to him.
I was close to sending him a ridiculous response and signing it "Robin", but decided it wasn't worth it.30 -
Last day on the contract from hell. I'd written a project with one other person in our spare time that performed a critical business function. The following conversation was had between myself, the job thief who was handed my job and their manager, with the 10 other IBM GS "dev domain experts" assigned to that team sitting silently on zoom:
Moi: hey all, what seems to be the problem?
JT: how to update the java for requirement?
Moi: I would assume a text editor, have you tried intellij
JTM: she's talking about ticket BS-101, the data is wrong
Moi: ah, well, you might want to fix that
JT: how to fix?
Moi: update the database and update the logic that depends on it
JTM: what changes are those?
Moi: the ones described in the ticket, I would assume, I'm no longer on that project
JTM: didn't you write this application?
Moi: yes.
JTM: ok, so do you know how to fix the issue?
Moi: definitely
JTM: ok... ... Can you tell us how to fix it?
Moi: yes.
*The sound of silence*
JTM: *will* you tell us?
Moi: I would, but I'm already off the clock, and as of an hour ago I no longer have a contract. And even if I did, I don't have a contract or authorization to work on that system. I'm not actually being paid for this call.
JTM: ... What are we going to do about this?
Moi: I have no idea
JTM: ok, so we can look at getting a 1 month contract to support this
Moi: I'm sure our firm has someone who can definitely help you out
JTM: *heavy raging* ... Can you do the work?
Moi: Unfortunatley, I'm already committed to a new contract at another customer. I also don't do one month contracts. I'm an engineer, not a car wash employee
JTM: well, I don't understand how you can just leave us in the lurch like this?!
Moi: well, respectfully, it was your decision to cut me from the budget because you thought you were close enough to end of the project to get it across the line with junior resources.
Interjecting-JT: I am senior!
Moi: Right. So, basically, you took ownership of the product before go live. We advised against it, in writing, numerous times. We also notified you that we would not carry a bench, so the project resources are now working on other things. We can provide you with new resources for a minimum 6 month duration who can help you out. Also, since we've cycled out, our rate has increased per the terms of our MSA.
JTM: we don't have budget for that! How are we supposed to do this?!
Moi: *zoom glare at JT* that question is more appropriate for your finance officer and the IT director. I can send a few emails and schedule a call with your account representative and the aforementioned individuals so you can hash this out.
-_---------------
I'm free! 🥳 That said, still plenty of residual fodder I need to get out of my system on these guys. Might need to start my own Dilbert.12 -
To replace humans with robots, because human beings are complete shit at everything they do.
I am a chemist. My alignment is not lawful good. I've produced lots of drugs. Mostly just drugs against illnesses. Mostly.
But whatever my alignment or contribution to the world as a chemist... Human chemists are just fucking terrible at their job. Not for a lack of trying, biological beings just suck at it.
Suiting up for a biosafety level lab costs time. Meatbags fuck up very often, especially when tired. Humans whine when they get acid in their face, or when they have to pour and inhale carcinogenic substances. They also work imprecisely and inaccurately, even after thousands of hours of training and practice.
Weaklings! Robots are superior!
So I replaced my coworkers with expensive flow chemistry setups with probes and solenoid fluid valves. I replaced others with CUDA simulations.
First at a pharma production & research lab, then at a genetics lab, then at an Industrial R&D lab.
Many were even replaced by Raspberry Pi's with two servos and a PH meter attached, and I broke open second hand Fischer Sci spectrophotometers to attach arduinos with WiFi boards.
The issue was that after every little overzealous weekend project, I made myself less necessary as well.
So I jumped into the infinitely deep shitpool called webdev.
App & web development is kind of comfortable, there's always one more thing to do, but there's no pressure where failure leads to fatalities (I think? Wait... do I still care?).
Super chill, if it weren't for the delusion that making people do "frontend" and "fullstack" labor isn't a gross violation of the Geneva Convention.
Quickly recognizing that I actually don't want to be tortured and suffer from nerve damage caused by VueX or have my organs slowly liquefied by the radiation from some insane transpiling centrifuge, I did what any sane person would do.
Get as far away from the potential frontend blast radius as possible, hide in a concrete bunker.
So I became a data engineer / database admin.
That's where I'm quarantining now, safely hiding from humanity behind a desk, employed to write a MySQL migration or two, setting up Redis sorted sets, adding a field to an Elastic index. That takes care of generating cognac and LSD money.
But honestly.... I actually spend most of my time these days contributing to open source repositories, especially writing & maintaining Rust libraries.10 -
Just found out the backend developer I’m always complaining about. The one who:
- Can’t implement OAuth, and we have to have app users login every 24 hours because we have no way to generate new refresh tokens.
- Who used the phrase “your time zone is not my concern” to avoid building something that would let us inject test data.
- Who’s been debugging a critical bug affecting many users since December.
- Who can’t conduct API tests from external internet (you know, like the way the app will be in the wild) because it takes too much time.
- Who replies to Jira tickets only on a blue moon.
- Who has been 90% of the reason for my blood pressure situation
... is a fucking principal engineer in this company. In pecking order, his opinion should be considered more valuable than mine and everyone on my team.
I’ve just lost the will to live. How are big organizations THIS bad. Seriously, what promotion discussion did he go into
“So, you are a complete and utter bastard, nobody can stand to speak to you and you’ve yet to deliver anything of worth that actually works, over the course of several years ... ... ... interested in having your pay doubled??”20 -
Waaaay too many but let's go with this one for now.
At my previous job there was a web application which was generating about 1gb of log data a second. Server was full and the 'fullstack engineers' we called had zero clue about backend stuff and couldn't fix it.
Me and another engineer worked our asses off to figure this out but eventually the logging stopped and it went back to normal.
Great, right?
For that moment. I was the on-call server engineer and at like 3am I got called awake because this shit was happening again.
Sleep drunk with my phone I ssh'd into the server, not sure about what to do at first but then suddenly: let's chattr the goddamn log file...
$ chattr +i /var/log/logfile
Bam, worked, done, back to sleep.
(this comment + param marks the file in a way that it can only be read until the mark is removed, so you can't write to it or move it or remove it or whatever)13 -
I've had my share of incompetent coworkers. In order of appearance:
1. A full stack dev. This one guy never, and I mean NEVER uses relationships in their tables. No indexing, no keys, nada. Couple of months later he was baffled why his page took ten seconds to load.
2. The same dev as (1). Requirement was to create some sort of "theme" feature for a web app. Hacked it by putting !important all over the place.
3. The same dev again. He creates several functions that if the data exists returns a view, and if it doesn't, "echo '0'". No, not return 0 or return false or anything, but fucking echo. This was PHP. If posted a rant about this a few months ago.
4. Same dev, has no idea what clean code is. No, not just reusable functions, he doesn't even get indenting right. Some functions have 4 spaces, some 2 tabs, some 6 tabs! And this is inside the same function. God wait until he tries Python...
5. Same dev now suggests that he become the PM. GM approves (very small company). Assigns me to travel to a client since they needed "technical assistance about the API". Was actually there to lead a UAT session.
Intermezzo, that guy went from fullstack dev to PM to sales (yes, one who calls clients to offer products) to business development, to product analyst in the span of two years.
After a year and a half there, I quit.
6. New company, a "QA engineer" who also assumes the role as the product owner. Does absolutely no tests other than "functional tests" in which he NEVER produces any form of documentation. Not even a set of test cases. He goes by "intuition".
7. Same guy as (6), hands me requirements for a feature. By "hands me" I mean he did that verbally. No spec documents, no slack chat, no Trello card. I ended up writing it as a card in Trello. Fast forward to the due date, he flips out because that wasn't what he wanted. Showed him the card. He walked away, without thinking of a solution how this mess should be handled.
Despite all this, I really don't want him (6&7) to leave the company. The devs get really stressed out at this job and he does make a really good person to laugh with/at. -
Actual rant time. And oh boy, is it pissy.
If you've read my posts, you've caught glimpses of this struggle. And it's come to quite a head.
First off, let it be known that WINDOWS Boot Manager ate GRUB, not the other way around. Windows was the instigator here. And when I reinstalled GRUB, Windows threw a tantrum and won't boot anymore. I went through every obvious fix, everything tech support would ever think of, before I called them. I just got this laptop this week, so it must be in warranty, right? Wrong. The reseller only accepts it unopened, and the manufacturer only covers hardware issues. I found this after screaming past a pretty idiotic 'customer representative' ("Thank you for answering basic questions. Thank you for your patience. Thank you for repeating obvious information I didn't catch the first three times you said it. Thank you for letting me follow my script." For real. Are you tech support, or emotional support? You sound like a middle school counselor.) to an xkcd-shibboleth type 'advanced support'. All of this only to be told, "No, you can't fix it yourself, because we won't give you the license key YOU already bought with the computer." And we already know there's no way Microsoft is going to swoop in and save the day. It's their product that's so faulty in the first place. (Debian is perfectly fine.)
So I found a hidden partition with a single file called 'Image' and I'm currently researching how to reverse-engineer WIM and SWM files to basically replicate Dell's manufacturing process because they won't take it back even to do a simple factory reset and send it right back.
What the fuck, Dell.
As for you, Microsoft, you're going to make it so difficult to use your shit product that I have to choose between an arduous, dangerous, and likely illegal process to reclaim what I ALREADY BOUGHT, or just _not use_ a license key? (Which, there's no penalty for that.) Why am I going so far out of my way to legitimize myself to you, when you're probably selling backdoors and private data of mine anyway? Why do I owe you anything?
Oh, right. Because I couldn't get Fallout 3 to run in Wine. Because the game industry follows money, not common sense. Because you marketed upon idiocy and cheapness and won a global share.
Fuck you. Fuck everything. Gah.
VS Code is pretty good, though.20 -
So we hired a junior engineer. 1 year of experience, this is his second job.
First task: Send some data to a web service using its REST API. Let me know when you've finished.
Two hours later I go to check on him.
- "I'm trying to decode this weird format the server uses"
He was writing a JSON parser in Python from scratch.
:/12 -
Most of the tech YouTubers are really noob engineers.
Joma was a data scientist. He is an L3 engineer at Google and he hasn't done much during the last 1 year based on his internal stats.
I saw tech leads stats while he was at Google and that dude did nothing during his time. I'm sure he was an IC before he became a lead.
Clement talks about system design bull shit but he's a math major who worked on some angular front end while he was at Google. Basically his experience in tech is mostly involving using mat-button and mat-input. He also quit FB in a month.
Listening to tech lead gives me cancer. That guy was also some front end/ mobile engineer. I don't think any less of mobile engineers but tech leads acts as if he built some large scale systems at Google and FB. His opinion about react native shows how much of a noob he is. He also talked about docker in one of his video which showed he had some fundamental misunderstanding of what docker is. In his courses, he struggles to explain simple algorithms.
I don't know how these people have the courage to claim themselves as some sort of experts in the field when they are extreme noobs. They also sell some shady courses and are robbing innocent college kids.
One thing they all do well is talk. Which I give them 10/10.12 -
Data Analyst: “the task failed in test, can we try running it in production?”
My life as a Data Engineer.5 -
In the begining of time, when The Company was small and The Data could fit in some fucking excel sheets, Those Who Came Before implemented some java tool to issue invoices, notify customers and clear received payments.
Then came the Time Of The Great Expanse, when The Company grew to unthinkable levels. Headcount increased with each passing day, and The Data shows that everything was going great!
But when the future seemed bright, came The Stall-Out. The days when The Company could not expand as fast as it did before. And Those Who Came Before left, abandoning their Undocumented Java Tool to its own luck.
Those who came after knew nothing of the inner workings of the Undocumented Java Tool. They knew only that the magical Jar would take a couple fucking excel spreadsheets and spit out reports and send emails like magic.
And those were The Dark Days.
In the darkness, The Data grew to be a monster. Soon a fucking excel spreadsheet could not hold The Data contained any longer. Those Who Came After, fearing the wrath of The Undocumented Java Tool, dared not mess with its code. Instead, they fucking cut away the lowest volume transactions from the fucking input spreadsheet, and left the company to report the unbilled invoices as "surprise losses". Fucking script kiddies, were Those Who Came After.
Then, at The Darkest of Days (literally, Dec 21st), marched into the project The Six Witchers, who fear not the Demon of Refactoring.
This story is still unfolding. Will The Six Witchers manage to unravel the mysteries of The Undocumented Java Tool? Will they be able to reverse engineer the fucking black box, and scale it's magic into a modern application?
Will they decrease revenue forecasting error by at least 2% in a single strike?
Only the future will tell.16 -
TL;DR
A "friend" is a tech fraud. Faking his resume as a software engineer! Only interested on the salary. This is unfair to all of us putting the hours of effort/practice just to improve our craft! 😠😤
I have a "friend" who is faking his resume, putting fake experiences and putting jargons not even related to tech just to make himself smart. He's using his customer service rep experience to talk confidently. His resume fcking long, 3 pages of fakery. I can't help, but to laugh when he sent it to me.
He has a tech degree, but worked in a BPO industry for 4 years, then recently, he quit. He got jealous with the lucrative software development industry and he wants to relearn coding, as a friend and I like sharing my knowledge, I agreed to guide him in the process.
After 3 moths, he got his first job, but unfortunately he got fired after two weeks because he commited sensitive data to the remote repo.
Then after a month, he got his second job and worked there for 6 months, he still don't know what his doing and always ask me solutions when he is stuck.
He got his 3rd job, remote work with high compensation. Fast forward after 3 months, he only got 1 month of salary, the other 2 wasn't given for unknown reason, my best guess is the company noticed his experience on paper does not match on real life.
Currently, he's working on another remote work with same compensation as before, and he still asks me super simple questions from time to time.
This is so unfair to all the devs who truly deserves the opportunity.20 -
I am a machine learning engineer and my boss expects me to train an AI model that surpasses the best models out there (without training data of course) because the client wanted ‘a fully automated AI solution’.13
-
Today is sprint demo day. As usual I'm only half paying attention since being a Platform Engineer, my work is always technically being "demoed" (shit's running ain't it? There you go, enjoy the EC2 instances.)
One team presents a new thing they built. I'm still half paying attention, half playing Rocket League on another monitor.
Then someone says
"We're storing in prod-db-3"
They have my curiosity.
"Storing x amount of data at y rate"
They now have my attention. I speak up "Do you have a plan to drop data after a certain period of time?"
They don't. I reply "Okay, then your new feature only has about 2 months to live before you exhaust the disk on prod-db-3 and we need to add more storage"
I am asked if we can add more storage preemptively.
"Sure, I say." I then direct my attention to the VP "{VP} I'll make the change request to approve the spend for additional volume on prod-db-3"
VP immediately balks and asks why this wasn't considered before. I calmly reply "I'm not sure. This is the first time I'm learning of this new feature even coming to life. Had anyone consulted with the Platform team we'd have made sure the storage availability was there."
VP asks product guy what happened.
"We didn't think we'd need platform resources for this so we never reached out for anything".
I calmly mute myself, turn my camera back off and go back to Rocket League as the VP goes off about planning and collaboration.
"CT we'll reach out to you next week about getting this all done"
*unmute, camera stays off* "Sounds good" *clears ball*4 -
I've optimised so many things in my time I can't remember most of them.
Most recently, something had to be the equivalent off `"literal" LIKE column` with a million rows to compare. It would take around a second average each literal to lookup for a service that needs to be high load and low latency. This isn't an easy case to optimise, many people would consider it impossible.
It took my a couple of hours to reverse engineer the data and implement a few hundred line implementation that would look it up in 1ms average with the worst possible case being very rare and not too distant from this.
In another case there was a lookup of arbitrary time spans that most people would not bother to cache because the input parameters are too short lived and variable to make a difference. I replaced the 50000+ line application acting as a middle man between the application and database with 500 lines of code that did the look up faster and was able to implement a reasonable caching strategy. This dropped resource consumption by a minimum of factor of ten at least. Misses were cheaper and it was able to cache most cases. It also involved modifying the client library in C to stop it unnecessarily wrapping primitives in objects to the high level language which was causing it to consume excessive amounts of memory when processing huge data streams.
Another system would download a huge data set for every point of sale constantly, then parse and apply it. It had to reflect changes quickly but would download the whole dataset each time containing hundreds of thousands of rows. I whipped up a system so that a single server (barring redundancy) would download it in a loop, parse it using C which was much faster than the traditional interpreted language, then use a custom data differential format, TCP data streaming protocol, binary serialisation and LZMA compression to pipe it down to points of sale. This protocol also used versioning for catchup and differential combination for additional reduction in size. It went from being 30 seconds to a few minutes behind to using able to keep up to with in a second of changes. It was also using so much bandwidth that it would reach the limit on ADSL connections then get throttled. I looked at the traffic stats after and it dropped from dozens of terabytes a month to around a gigabyte or so a month for several hundred machines. The drop in the graphs you'd think all the machines had been turned off as that's what it looked like. It could now happily run over GPRS or 56K.
I was working on a project with a lot of data and noticed these huge tables and horrible queries. The tables were all the results of queries. Someone wrote terrible SQL then to optimise it ran it in the background with all possible variable values then store the results of joins and aggregates into new tables. On top of those tables they wrote more SQL. I wrote some new queries and query generation that wiped out thousands of lines of code immediately and operated on the original tables taking things down from 30GB and rapidly climbing to a couple GB.
Another time a piece of mathematics had to generate all possible permutations and the existing solution was factorial. I worked out how to optimise it to run n*n which believe it or not made the world of difference. Went from hardly handling anything to handling anything thrown at it. It was nice trying to get people to "freeze the system now".
I build my own frontend systems (admittedly rushed) that do what angular/react/vue aim for but with higher (maximum) performance including an in memory data base to back the UI that had layered event driven indexes and could handle referential integrity (overlay on the database only revealing items with valid integrity) or reordering and reposition events very rapidly using a custom AVL tree. You could layer indexes over it (data inheritance) that could be partial and dynamic.
So many times have I optimised things on automatic just cleaning up code normally. Hundreds, thousands of optimisations. It's what makes my clock tick.4 -
Biggest challenge I overcame as dev? One of many.
Avoiding a life sentence when the 'powers that be' targeted one of my libraries for the root cause of system performance issues and I didn't correct that accusation with a flame thrower.
What the accusation? What I named the library. Yep. The *name* was causing every single problem in the system.
Panorama (very, very expensive APM system at the time) identified my library in it's analysis, the calls to/from SQLServer was the bottleneck
We had one of Panorama's engineers on-site and he asked what (not the actual name) MyLibrary was and (I'll preface I did not know or involved in any of the so-called 'research') a crack team of developers+managers researched the system thoroughly and found MyLibrary was used in just about every project. I wrote the .Net 1.1 MyLibrary as a mini-ORM to simplify the execution of database code (stored procs, etc) and gracefully handle+log database exceptions (auto-logged details such as the target db, stored procedure name, parameter values, etc, everything you'd need to troubleshoot database errors). This was before Dapper and the other fancy tools used by kids these days.
By the time the news got to me, there was a team cobbled together who's only focus was to remove any/every trace of MyLibrary from the code base. Using Waterfall, they calculated it would take at least a year to remove+replace MyLibrary with the equivalent ADO.Net plumbing.
In a department wide meeting:
DeptMgr: "This day forward, no one is to use MyLibrary to access the database! It's slow, unprofessionally named, and the root cause of all the database issues."
Me: "What about MyLibrary is slow? It's excecuting standard the ADO.Net code. Only extra bit of code is the exception handling to capture the details when the exception is logged."
DeptMgr: "We've spent the last 6 weeks with the Panorama engineer and he's identified MyLibrary as the cause. Company has spent over $100,000 on this software and we have to make fact based decisions. Look at this slide ... "
<DeptMgr shows a histogram of the stacktrace, showing MyLibrary as the slowest>
Me: "You do realize that the execution time is the database call itself, not the code. In that example, the invoice call, it's the stored procedure that taking 5 seconds, not MyLibrary."
<at this point, DeptMgr is getting red-face mad>
AreaMgr: "Yes...yes...but if we stopped using MyLibrary, removing the unnecessary layers, will make the code run faster."
<typical headknodd-ers knod their heads in agreement>
Dev01: "The loading of MyLibrary takes CPU cycles away from code that supports our customers. Every CPU cycle counts."
<headknod-ding continues>
Me: "I'm really confused. Maybe I'm looking at the data wrong. On the slide where you highlighted all the bottlenecks, the histogram shows the latency is the database, I mean...it's right there, in red. Am I looking at it wrong?"
<this was meeting with 20+ other devs, mgrs, a VP, the Panorama engineer>
DeptMgr: "Yes you are! I know MyLibrary is your baby. You need to check your ego at the door and face the facts. Your MyLibrary is a failed experiment and needs to be exterminated from this system!"
Fast forward 9 months, maybe 50% of the projects updated, come across the documentation left from the Panorama. Even after the removal of MyLibrary, there was zero increases in performance. The engineer recommended DBAs start optimizing their indexes and other N+1 problems discovered. I decide to ask the developer who lead the re-write.
Me: "I see that removing MyLibrary did nothing to improve performance."
Dev: "Yes, DeptMgr was pissed. He was ready to throw the Panorama engineer out a window when he said the problems were in the database all along. Didn't you say that?"
Me: "Um, so is this re-write project dead?"
Dev: "No. Removing MyLibrary introduced all kinds of bugs. All the boilerplate ADO.Net code caused a lot of unhandled exceptions, then we had to go back and write exception handling code."
Me: "What a failure. What dipshit would think writing more code leads to less bugs?"
Dev: "I know, I know. We're so far behind schedule. We had to come up with something. I ended up writing a library to make replacing MyLibrary easier. I called it KnightRider. Like the TV show. Everyone is excited to speed up their code with KnightRider. Same method names, same exception handling. All we have to do is replace MyLibrary with KnightRider and we're done."
Me: "Won't the bottlenecks then point to KnightRider?"
Dev: "Meh, not my problem. Panorama meets primarily with the DBAs and the networking team now. I doubt we ever use Panorama to look at our C# code."
Needless to say, I was (still) pissed that they had used MyLibrary as dirty word and a scapegoat for months when they *knew* where the problems were. Pissed enough for a flamethrower? Maybe.6 -
Man, most memorable has to be the lead devops engineer from the first startup I worked at. My immediate team/friends called him Mr. DW - DW being short for Done and Working.
You see, Mr. DW was a brilliant devops engineer. He came up with excellent solutions to a lot of release, deployment, and data storage problems faced at the company (small genetics firm that ships servers with our analysis software on them). I am still very impressed by some of the solutions he came up with, and wish I had more time to study and learn about them before I left that company.
BUT - despite his brilliance, Mr. DW ALWAYS shipped broken stuff. For some reason this guy thinks that only testing a single happiest of happy path scenarios for whatever he is developing constitutes "everything will work as expected!" As soon as he said it was "done", but golly for him was it "done". By fucking God was that never the truth.
So, let me provide a basic example of how things would go:
my team: "Hey DW, we have a problem with X, can you fix this?"
DW: "Oh, sure. I bet it's a problem with <insert long explanations we don't care about we just want it fixed>"
my team: "....uhh, cool! Looking forward to the fix!"
... however long later...
DW: "OK, it's done. Here you go!"
my team: "Thanks! We'll get the fix into the processing pipelines"
... another short time later...
my team: "DW, this thing is broken. Look at all these failures"
DW: "How can that be? It was done! I tested it and it worked!"
my team: "Well, the failures say otherwise. How did you test?"
DW: "I just did <insert super basic thing>"
my team: "...... you know that's, like, not how things actually work for this part of the pipeline. right?"
DW: "..... But I thought it was XYZ?"
my team: "uhhhh, no, not even close. Can you please fix and let us know when it's done and working?"
DW: "... I'll fix it..."
And rinse and repeat the "it's done.. oh wait, it's broken" a good half dozen times on average. But, anyways, the birth of Mr. Done and Working - very often stuff was done, but rarely did it ever work!
I'm still friends with my team mates, and whenever we're talking and someone says something is done, we just have to ask if it's done AND working. We always get a laugh, sadly at the excuse of Mr. DW, but he dug his own hole in this regard.
Little cherry on top: So, the above happened with one of my friends. Mr. DW created installation media for one of our servers that was deployed in China. He tested it and "it was done!" Well, my friend flies out to China for on-site installation. He plugs the install medium in and goes for the install and it crashes and burns in a fire. Thankfully my friend knew the system well enough to be able to get everything installed and configured correctly minus the broken install media, but definitely the most insane example of "it's done!" but sure as he'll "it doesn't work!" we had from Mr. DW.2 -
This rant is inspired by another rant about automated HR emails like "we appreciate your interest [bla bla] you got rejected [bla bla]". (Please bare with me).
I live in an underdeveloped country, I graduated in September, did Machine Learning for my thesis and I will soon publish a paper about it, loved it wanted to work as ML/data science engineer. On all the job postings I found there was only one job related, I sent resume, they didn't answer, couple months later that company posted that they want a full stack web dev with knowledge of mobile dev and ML, basically an all in one person, for the salary of a junior dev.
- another company posted about python/web scraping developer, I had the experience and I got in touch, they sent me a test, took me 3 days, one of the questions took me 2 days, I found an unanswered SO question with the exact wording dating to 6 months ago, I solved it, sent answers, never heard back from them again.
- one company weren't really hiring, I got in touch asking if the have a position, they sent a test, I did it, they liked it, scheduled an interview, the interviewer was arrogant, not giving any attention to what I am saying, kept asking in depth questions that even an expert might struggle answering. In the end they said they're not really hiring but they interview and see what they can find. Basically looking for experts, I mentioned that im freshly graduated from the very beginning.
- over 1000 applications on different positions on LinkedIn across the whole world, same automated rejection email, but at least they didn't keep me waiting.
- I lost hope. Found a job posting near me, python/django dev, in the interview they asked about frontend (react/vueJS) and Flutter, said I don't have experience and not interested in that, they asked about databases, C and java and other stuff that I have experience in, they hired me with an insulting salary (really insulting) cuz they knew im hopeless, filling 2 positions, python dev and tech support for an app built in the 90s with C/java and sorcery... A week into the job while I'm still learning about the app I'm supposed to support, the guy called me into the office: "here's the thing" he said, "someone else is already working on python, i want you to learn either react or vueJS or flutter" I was in shock, I didn't know what to say, I said I'll think about it, next week I said I'll learn react, so I spent the week acting like im learning react while I scroll on FB and LinkedIn (I'm bad, I know).
- in the weekend a foreign company that I applied to few weeks ago got in touch, we had some interviews and I got hired as DevOps/MLOps. It's been a month and I'm loving it, the salary is decent and I love what I do.
Conclusion: don't lose hope.8 -
This brings joy
https://reddit.com/r/technology/...
Bypass paywall:
A series of scandals and missteps has damaged Facebook's reputation so much that the company is being forced to pay ever larger compensation to hire and retain workers, according to industry recruiters, former employees, and data reviewed by Insider.
The company has always competed aggressively for talent, and the tech job market in general is on fire. But a deteriorating public image means the social-media giant now has to outbid other major tech companies, such as Google.
"One thing Facebook can still do is pay a lot more," said Jose Guardado, an experienced tech recruiter and the founder of Build Talent. "They can easily throw more compensation at people they currently have, and cover any brand tax and pay a little more to get people to come on."
Silicon Valley companies thrive or whither based on their ability to recruit the smartest employees. Without a steady influx of engineers and other technical experts, new products and important updates take longer to release, and rivals can quickly get ahead. Then there's the financial cost: In 2022, Facebook projected, expenses could jump as high as $97 billion from $70 billion this year, in large part because of "investments in technical and product talent." A company spokesperson did not respond to a request for comment.
Other companies, and even whole industries, have had to increase compensation to overcome hiring and retention problems caused by scandal and shifting public perceptions, said Alan Johnson, a managing director at the compensation consulting firm Johnson Associates. "If you're an oil company, if you make cigarettes, if you're in cattle or Wells Fargo, sure," he said.
How well this is working for Facebook is debatable as the company has more than 4,300 open jobs and has seen decreasing rates of acceptance on job offers, according to internal documents reported by Protocol. It's also seen dozens of high-level executives leave this year, and recruiters say employees are now more open to considering jobs elsewhere. Facebook used to be a place that people rarely left, given its reach, pay, and perks.
A former Oculus engineer who left last year said Facebook could now be seen as a "black mark" on someone's career. A hardware engineer who exited in 2020 shared similar sentiments: They said they quit because of concerns about misinformation on the platform and the effect of that on children. Another employee said their department was dissolved in late 2019 by Facebook and, although the company offered another position that paid more, they left last year anyway for a different industry. The workers, and many other people who spoke with Insider for this story, asked not to be identified because of the sensitive nature of the topic.
For those who stick around and people who take new jobs at Facebook, base pay and stock grants have gone up a "sizable" amount in the past year, said Zuhayeer Musa, cofounder of Levels.fyi, a platform that collects pay data based on verified offers and compensation disclosures.
During the second quarter of 2021, the median compensation for an upper-mid-level engineer, an E5, was $400,000, up from $380,000 a year earlier. For an E4, the median pay jumped to $276,000 from $256,000 in the same period. For both groups, the increases were double the gains between 2018 and 2019, Levels.fyi data showed.
Musa, who's firm also offers pay-negotiation coaching, said previously that the total compensation ceiling for an E5 engineer at Facebook was $450,000. "We recently had a client get up to $510,000 for E5," he added.
Equity awards at the company are getting more generous, too. At the group-director and VP levels, Facebook staff are getting $3 million to $6 million in restricted stock units each year, another tech recruiter said. Directors and managers are getting on average $1 million a year. In engineering, a high-level engineer is getting $600,000 in stock and a $75,000 bonus, while even an entry-level engineer is getting $50,000 to $100,000 in stock and a $20,000 to $50,000 bonus, Levels.fyi data indicated.
Even compared to Google, Facebook's stock awards are generous and increasing, Levels.fyi data shows. While base pay is about the same, Facebook offers more in stock grants, significantly increasing total compensation. At Google, entry-level equity awards range from $20,000 to $38,000, while Facebook grants are worth $40,000 to $60,000. Sign-on bonuses at Facebook are often about $50,000, while Google gives about $20,000, according to the data.
"It's not normal, but it's consistent with the craziness that's happening in the market right now," said Aalap Shah, a managing director focused on the tech industry at the consulting firm Pearl Meyer.10 -
TLDR: crappy api + idiot ex client combo rant // devam si duška
I saw a lot of people bitching about APIs that don't return proper response codes and other stuff..
Well let me tell you a story. I used to work on a project where we had to do something like booking, but better..crossbreed with the Off&Away bidding site (which btw we had to rip off the .js stuff and reverse engineer the whole timer thingy), using free versions of everything..even though money wasn't an issue (what our client said). Same client decided to go with transhotel because it was sooooo gooood... OK? Why did noone heard of them then?
Anyhow, the api was xml based.. we had to send some xml that was validated against a schema, we received another that was supposed to be validated againts another schema.. and so on and so on..
...
...
supposed..
The API docs were nonexistent.. What was there, was broken English or Spanish.. Even had some comments like Add This & that to chapter xy.. Of course that chapter didn't even exist yet. :( And the last documentation they had, was really really old..more than a year, with visible gaps, we got the validation schemas not even listed in the docs, let alone described properly.
Yaaay! And that was not everything.. besides wrong and missing data, the API itself caused the 500 server error whenever you were no longer authenticated.
Of course it didn't tell you that your session was dead.. Just pooof! Unhandled crap everywhere!
And the best part?! We handled that login after inspecting what the hell happened, but sent the notification to the company anyways.. We had a conf call, and sent numerous emails explaining to them what a 'try catch' is and how they should handle the not authenticated error <= BTW they should have had a handled xml response for that, we got the schema for it! But they didn't. Anyhow, after two agonizing days talking back and forth they at least set up the server to be available again after the horrified 500 error. Before, it even stopped responding until reset (don't ask me how they managed to do that).
Oh yeah, did I mention this was a worldwide renown company?! Where everybody spoke/wrote English?! Yup, they have more than 700 people there, of course they speak English! <= another one of my ex clients fabulous statements... making me wanna strangle him with his tie.. I told him I am not talking to them because no-one there understood/spoke English and it would be a waste of my time.. Guess who spent almost 3 hours to talk to someone who sounded like a stereotypical Indian support tech guy with a flue speaking Italian?! // no offence please for the referenced parties!!
So yeah, sadly I don't have SS of the fucked up documentation..and I cannot post more details (not sure if the NDA still holds even though they canceled the project).. Not that I care really.. not after I saw how the client would treat his customers..
Anywayz I found on the interwebz some proof that this shitty api existed..
picture + link: https://programmableweb.com/api/...
SubRant: the client was an idiot! Probably still is, but no longer my client..
Wanted to store the credit card info + cvc and owner info etc.. in our database.. for easier second payment, like on paypal (which he wanted me to totally customize the payment page of paypal, and if that wasn't possible to collect user data on our personalized payment page and then just send it over to paypal api, if possible in plaintext, he just didn't care as long as he got his personalized payment page) or sth.... I told the company owner that they are fucking retards if they think they can pull this off & that they will lose all their (potential) clients if they figure that out.. or god forbid someone hacked us and stole the data.. I think this shit is also against the law..
I think it goes without saying what happened next.. called him ignorant stupid fucktard to his face and told him I ain't doing that since our company didn't even had a certificate to store the last 4 numbers.. They heard my voice over the whole firm.. we had fish-tank like offices, so they could all see me yelling at the director..
Guess who got laid off due to not being needed anymore the next day?! It was the best day of my life..so far!! Never have I been happier to lose my job!!
P.S. all that crap + test + the whole backand for analysis, the whole crm + campaign emails etc.. the client wanted done in 6 months.. O.o
P.P.S. almost shat my pants when devRant notified my I cannot post and wanted to copy the message and then everything disappeard.. thank god I have written this in the n++ xD
11 -
Once upon a time, one or two jobs ago, a really awesome engineer specced out a distributed search application in response to a business need. This company was managed pretty oldschool and required a ton of paperwork and approvals.
The engineer spent many weeks running tests and optimizing the hell out of this app cluster. It flew, and he had the data to prove it could handle production workloads (think hundreds of terabytes of data being processed every single day)
Part of the way he achieved this was having RAID0 on all of the servers to maximize I/O throughput. He didn't care much about data loss, since the application itself was fault tolerant on a much more granular level.
Management, hearing about this, absolutely flipped their shit and demanded RAID6 instead. This despite the conclusive data that the engineer had that proved RAID6 couldn't keep up.
He more or less got told to STFU.
Even this despite the fact that a RAID restripe would actually take many times longer than rebuilding the failed node from scratch (a process that took about 30 minutes by hand, and could probably be automated to be done in less than five), causing a longer exposure to actual data loss throughout the length of the days-long array rebuild time.
The ill-thought-out requirement added about 50% to the cost of the project (*many* more hard drives now required), beyond the original budget, and the subsequent bureaucratic wrangling resulted in a late product launch.
6 months or so later, after real customers were using this product, the app was buckling under around half of its expected workload. A friend of the engineer suggested to management to try RAID0. Sure enough, that resolved the I/O bottleneck.
This rage-inducing story has a happy ending, though! Said engineer left the company not long after this incident, citing it as a reason for his departure. He was immediately hired by another company, making integer multiples of his prior salary.
The product the company botched the launch of by ignoring his spec? It died a few months later. Maybe the poor customer experience was to blame? Maybe the late launch? Maybe it was another reason entirely.
Either way, millions of dollars of hardware now sat fallow. This was a black eye on the company all the way up to the C-level.
tl;dr: Listen to your engineers. You hired them for their expertise.5 -
I am conducting technical interviews for about 10 years now.
I swear to god, the applicants keep getting dumber and dumber.
Getting more and more ashamed to talk about data structures, design patters or even the most basic algorithms, everyone with a graduation badge from udemy is now a software engineer. Fuck this shit.
17 -
I created a curriculum to homeschool myself way up for a MSc in AI/ML/Data Engineer for Application in Health, Automobiles, Robotics and Business Intelligence. If you are interested in joining me on this 1.5yrs trip, let me knw so I can invite you to the slack channel. University education is expensive..can't afford that now. So this would help but no certificate included.17
-
Got my first laptop while I was overseas.
It was a windows hp laptop with Vista.
It was an absolute piece of shit.
Decided to find the people responsible of it.
Got to what a software engineer was.
Boss told me to look in the library to see if i find some books on the subject. Got a Java and C++ book.
Shit was hard af cuz I had no clue what I was doing, but I liked it. Decided to look more into an application wise platform of study rather than doing basic CLI shit. Got into web development with Java. Got a hold of more JS. Liked JS more cuz shit was easy, found about server side JS with classic ASP, did VBScript as well.
Eventually found Python, fell in love but hated the whitespace ussage for block level code etc. Found Ruby, to this day the most beautiful language according to me. Read about why's poignant intro to Ruby.
Dug it, but wanted some other things. Found out about the study of data structures ans algorithms, then harvard's free cs50 course, then mit courseware, rice's python class. Took all of them. CS50 introduced php, liked it, sounded like a drug, was easy to use, for whatever fucking reaskn my ass decided to use version 4 even though 5 was already out. Learned to appreciate advancements in programming language even more
Hipster phase, while studying php got more into JS and web design with more css concepts, wanted my shit to be pretty. Somehow landed with Common Lisp. Mind fucking blown.
Continued with php. Got into uni, math made sense through programming, ok so I am stupid, but not that stupid, python is the best calculator ever.
bring it bitches.
Graduated.
Still don't know what I am doing.1 -
The people who wrote the specs for SAP OCI should be hanged by rusty barbed wire while being tickled by krusty the clown.
Which one of these stinky hobbits thought it was a great idea to require a (catalog) server to handle a POST request by sending back an HTML form which has to execute a POST request immediately by JavaScript on load?
Why not fucking respond with the actual god damn fucking data?
Some "senior" (read "senile") software "engineer" has to get decapitated.
Quote from the specification (OCI Function: VALIDATE, section 2.3.2):
"The product catalog replies with an HTML page that contains a form with the productdata in OCI format. [...] The HTML page may not contain any visible elements ([...]). The form must be sent automatically by JavaScript after the page has been loaded."
The only thing that should get sent after loading would be these people's asses to hell after my minigun has finished loading.
SAP is the kind of company who earns a huge junk of money from utter, stinking, filthy crap and they like to piss in their customers' "müesli".4 -
Yesterday I met my cousins who are old enough to have kids. It was a good talk with them bringing back the old memories. One of my cousins has a barely 5 year old kid. I tried to talk to her and the conversation went like this:
Me: “hey there! Hi, how are you?”
She: “Good. What do you do?”
Me: “I am a computer science engineer. What do you wanna be when you grow up?”
She: “A scientist.”
Me: **thinking calmly, “Oh, what kind of scientist?”
She: “A Data Scientist.”
Me: **Two seconds of silence and decides to leave...4 -
Apply for a data engineer role.
Get invited for a data science interview.
HR says they're building AI and I were to supervise another person writing its algorithm.
It's a media company.
*Risitas intensifies*6 -
One job I picked up was for an IoT Start Up. It was quite interesting work, reporting to the technical director, who was an electronics engineer, who was designing the hardware himself, they had a couple of firmware guys already, and just needed someone to take care of the software.
So they said they needed something in Azure that they could stream their data to and provide analytics for their clients. It had to be Azure, and it had to be Azure Native, and was to be Multi-client, as they had a deal with Microsoft to showcase how well Azure works in the IoT space at an exhibition/conference in 3 months time.
So I worked flat out for 3 months, on a whole variety of technology, from C++ to get the radio packets from their IoT chip, Python to run on the hub to take the data from the C++ and stream it to the cloud, Azure IoT Hubs in every continent to receive the data and store it an a Cosmos DB, and then Power BI analytics wrapped up in an Angular front end that the clients could log into.
Got it finished 2 days before the show, and they were so pleased I got flown business class to Singapore to be on the stand and talk to customers.
The first sign of trouble was when we arrived at the show to find we just had one of those little circular tables with two stools in the middle of the floor, about two feet across and no power.
No problem, I was able to sort that, swapping laptops in and out.
Microsoft were really happy with what we had, and couldn't believe I had thrown it all together in 3 months.
We picked up a potential customer for the system, a major Asian Telecoms company.
Then when we got home, the CEO swooped in. I had never met this guy before. Imagine one of the VC guys from Silicon Valley, or the CEO from the IT Crowd. You get the picture. Could talk the hind leg of a donkey, and real street smart, but no brains. He insisted on "taking it from here" and flew alone to strike the deal with the customer. Came back with an MOU in his pocket and said to me, their guys will be in touch with you.
Then I got a call. Can you send us the source code and tell us how what servers we have to run this on?
Um, its cloud native.
No, we can't use a cloud it has to be on our servers - your CEO told us that was no problem..
He hadn't even taken the trouble to find out what it was we had built, and what he was selling.1 -
Fucking shit i just had a 3 days chat with google's cloud engineer about an issue i had in a project. eventually the issue occured due to an update they made on some projects involving IAM changes that required some changes from my part in my security toles. Like wtf haven't you heard of data fixes when you roll out such changes?! I just had my production env down for 72hours for their fuckup.
At least send an email regarding it so we could set it up in time1 -
Some of these things are not like the others. One of these people is a tv scientist not an actual data engineer or data scientist, while another is an activist and while is extremely respected, has no room in a data+ai talk -.-
10 -
My own colo server. My own cloud. My own infra.
Fuck all of the CSP's and their fucking broken TOS and their data privacy violations!
Why do you think Amazon is so cheap? Because when they discover a product/service/software on AWS makes money, they WILL Reverse engineer it, make it and price you out of the market. It's their business model!5 -
INTERVIEW. It tells everything about the company. I recently applied for a "big" company for the position of ML Engineer. The Job description was like "someone with good knowledge of visual recognition, deep learning, advanced ML stuff, etc." I thought great, I might be a good fit. A guy called me the next day. Introduced himself as a manager of the Data Science team with 8+ years of experience. Started the talk saying "it is just an informal intro". But things escalated very quickly. Started shooting Data Science questions. He was asking questions in a very bookish way. Tells me to recite formulas (like big formulas). When I explained to him a concept, he was not understanding anything. Wanted a very bookish answer. I quickly realized I know more about ML stuff than him (not a big deal) and he is arrogant as fuck (not accepting my answers). Plus, he has no knowledge about Deep Learning. At the very end, he tells me "man, you need to clear up your fundamentals". WTH??? My fundamentals. Okay, I am not Einstein or Hinton, but I know I was answering things correctly. I have read books and research papers and blogs and all. When I don't know about things, I tell straight away. I don't cook answers. So the "interview" ended. I searched that man on LinkedIn. Got to know he teaches college students Data Science and ML. For a fee of 50,000 INR. It's a big amount!! Considering the things he teaches. You can find the same stuff (with far higher quality) free of cost (on Coursera, Udacity, YouTube, free books, what not). He is a cheater. He is making fool of college students. That is why I sometimes hate "experience". 8+ years of exp and he is such an a**hole!! BTW, I thanked God for saving me from that company. Can't imagine such an arrogant boss.
TLDR: Be vigilant during interviews. It tells a lot about the company.4 -
So I've created this account specifically for this rant. I usually just browse anonymously.
I've recently been hired in a big company that is one of the biggest Microsoft users in the world and my essentially revolves on making it easier for our collaborators to work with SharePoint (and other ms software)
Never in my life have I hit that much of a roadblock. So for the past week I've been trying to integrate what Ms calls webparts. And to modify the default webparts Ms provides you need to their properties (or Metadata). Except here's the big problem these are NOT documented anywhere (unless I failed to find it, if you do know where it is documented please HMU), so I've found myself trying to reverse engineer the js scripts that are served with SharePoint to figure out what the webpart properties are called and what type of data they are! I've been going through endless github repos using the CSOM nuget package (it's the library everyone uses to interact with SharePoint) and I finally found out about this other library called PnP which is a wrapper around CSOM that makes it easier to use. That wrapper has a way for me to load existing page and look at the properties of existing webparts. So here I thought it was the end of my suffering and I could finally get an idea of what it should be. Turns out this method doesn't work because one of the dependencies it has has had breaking changes and they still updated it even though it breaks their code! So for the past two days I've been trying random combinations of key values with different data types and json serialization methods.
Oh and yeah I've also looked at all the http calls via the chrome network tab, the metadata is not served as an individual file but is computed by Ms servers when they're serving you their html files.
So uh yeah run from CSOM if you can..3 -
1. When i was 11 years old i came across a trojan horse program and i sent it to all my school friends to get their messenger password.
(I had a list of passwords of all the people in my class... dont worry, i didnt even used the passwords, i just stored them in my list lol)
2. Took data structures 10 years later, hated it and switched to electrical eng
3. I was working on a manufacturing plant as the lead engineer and one of my prototypes was misbehaving and i was blaming software and software was blaming me...typical stuff... So i said "im going to get a masters is software so i can know wtf you are talking about, and tell you that you are wrong😠!!"
Got in school, started in the masters and quit that place i was working for 🤣.. (they were shitty people and unethical business... fyi)
Now im back to where i was was when i was 11... (minus the hacking email accounts of my classmates)4 -
I recently tried to apply the same data analytics rationale that I use at work to my personal life. This is not a rant, it is more like an data storytelling of an actual use case I would like some input on.
I set a goal - gotta thin up a bit and calm down my ticker - and got a (almost unreasonably expensive) field expert consultant to yell at me about it for a couple hours.
I unravel the metrics - there is like a million weight-related KPIs and most say nothing at all. I have never seen an non-infrastructure measurable subject that could not be resumed to 2-5 performance metrics. I got overall weight, how well my nine-years-old business suit fits me, heart rate, and day-after relative muscle pain (it will make sense soon).
Then its data-pipeline time. I bought a cheap weight scale and smartwatch, and every morning I input the data in an app. Yes, I try to put on the suit every morning. It still does not fit.
After establishing a baseline, I tried to fit different approaches. Doing equipment-free exercises, going to the gym, dieting. None was actually feasible in the long run, but trying different approaches does highlight the impacts and the handling profile of each method.
Looking at the now-gathered data, one thing was obvious - can't do dieting because it is not doable to have a shopping list and meals for me and another for the family.
Gym is also off the table - too much overhead. I spend more time on the trip there and back than actually there.
And home exercise equipment is either super crappy or very expensive. But it is also the most reasonable approach.
So it is solutions time. I got a nice exercise bycicle (not a peloton), an yoga mat (the wife already had that one) and an exercise program that uses only those two resources. Not as efficient without dieting, not as measurable and broad as the gym, but it fits my workflow. Deploy to production!
A few months pass and the dataset grows. The signal is subtle but has support - it works! The handling, however, needs improvement, since I cannot often enough get with the exercise program. Some mornings are just after some hard days.
I start thinking about what else I can improve in the program, but it is already pretty lean and full of compromises.
So I pull an engineer and start thinking about the support systems and draft profile. What else could be draining my willpower and morning time?
Chores. Getting the kids ready for school, firing up the moka pot, setting the off-brand roomba, folding the overnight-dried clothes, cooking breakfast, doing the dishes, cleaning the toilets. All part of my morning routine. It might benefit from some automation.
Last month I got that machine our elders call "wasteful" and "useless crap lazy entitled Americans invented because they feel oh-so-insulted for simply doing something by hand like everyone always did" - a "dish-washer".
Heh, I remember how hard was to convince my mother-in-law that an remote-controled electric garage door would not make she look like an spoiled brat.
Still to early to call, but I think that the dishwasher just saved me about 25 mins every morning. It might be enough to save willpower for me to do more exercise.
This is all so reflective of all data analytics cases really are out in the wild - the analytics phase seems so small compared to the gathering and practical problem-solving all around. And yet d.a. is what tells you that you are doing the wrong thing all along. Or on what you should work next.7 -
The shady company that tried to interview me for a data entry job when I applied as a software developer called me. They left a message asking if I was still interested in a help desk position. Like nope and I never was. Been working at my current job as a software engineer for over a month. Try finding people who actually want the jobs you're offering!6
-
Small company, sole engineer. Non-tech management. Increasingly fancy job titles despite working alone most of the time, with the promise of hiring someone (again) I can actually manage soon.
Backlog of projects/tasks is truly a mindfuck, with new things being added each week. This backlog will never ever get done, and nothing matters anyway because the next idea is "the future", all the time.
While I have influence on some aspects of decision making, it usually ends up being what the boss wants. Actively opposed a project because it's just too big of an undertaking, it was forced through anyway. I'm trying to keep the scope manageable as I'm building it now, and it's hard.
"It's the future, we absolutely have to do this. It will be the biggest thing we've ever done."
Boss's excitement then quickly faded since it's actually in development, now nobody really seems to want to know where it's at, or how it will all work. I need to scope it out, with the knowledge that many decisions boss signed off will be questioned when he actually looks at it. We now have even more "exciting" ideas of utter grandeur. Stuff that I can't even begin to comprehend the complexity of, while struggling to keep a self imposed deadline on the current one.
Every single morning we sit on Zoom for a "valuable" "catch-up". This is absolutely perfect for one thing: Completely destroying whatever drive and focus I have going into the day. Unrelated topics, marketing conversations, even more ideas, ideas for ideas sake, small problems blown out of proportion, the list goes on. I recently argued in detail why it should be scrapped or at least be optional to attend. No luck, it's "valuable".
Today a new idea was announced, and we absolutely have to do it ASAP because it can only be better than the current solution. I raise my concerns, saying it's not as easy as you make it out to be, we should properly think about it. Nope! We'll botch something to prove that it works... So you'll base your decision whether it's good on some half ass botch job that nobody really has the mental capacity to actually pay attention to. What a reliable way to measure!
"Our analytics data isn't useful enough to tell us the impact of things we do. We (you) have to fix this." Over the last 2 or so years, I've been pushing for an overhaul and expansion of our data analysis capabilities for exactly this reason. Integrating different data sources into a unified solution so we can easily see what we're doing, etc. Nope, never happened.
The new project idea which is based on wild assumptions is ALWAYS more important than the groundwork.
Now when I mentioned that this is what I wanted to do all along, it got brushed aside. "We don't need to do anything complicated, just fix this, add that, and it's done. It should be an easy thing to do. This is very important for our decision making." Fine, have it your way.
I'm officially burned out. It's so fucking hard to get myself to focus on my work for more than an hour or two. I started a side project, and even that effort is falling victim to my day-job-induced apathy.
I'm tempted to hand in my resignation without another offer on the table. I just need time to rediscover my passion, and go job hunting from that position, instead of the utter desperation of right now.
If you've read through all this rambling, kudos to you!8 -
# Retrospective as Backend engineer
Once upon a time, I was rejected by a startup who tries to snag me from another company that I was working with.
They are looking for Senior / Supervisor level backend engineer and my profile looks like a fit for them.
So they contacted me, arranged a technical test, system design test, and interview with their lead backend engineer who also happens to be co-founder of the startup.
## The Interview
As usual, they asked me what are my contribution to previous workplace.
I answered them with achievements that I think are the best for each company that I worked with, and how to technologically achieve them.
One of it includes designing and implementing a `CQRS+ES` system in the backend.
With complete capability of what I `brag` as `Time Machine` through replaying event.
## The Rejection
And of course I was rejected by the startup, maybe specifically by the co-founder. As I asked around on the reason of rejection from an insider.
They insisted I am a guy who overengineer thing that are not needed, by doing `CQRS+ES`, and only suitable for RND, non-production stuffs.
Nobody needs that kind of `Time Machine`.
## Ironically
After switching jobs (to another company), becoming fullstack developer, learning about react and redux.
I can reflect back on this past experience and say this:
The same company that says `CQRS+ES` is an over engineering, also uses `React+Redux`.
Never did they realize the concept behind `React+Redux` is very similar to `CQRS+ES`.
- Separation of concern
- CQRS: `Command` is separated from `Query`
- Redux: Side effect / `Action` in `Thunk` separated from the presentation
- Managing State of Application
- ES: Through sequence of `Event` produced by `Command`
- Redux: Through action data produced / dispatched by `Action`
- Replayability
- ES: Through replaying `Event` into the `Applier`
- Redux: Through replay `Action` which trigger dispatch to `Reducer`
---
The same company that says `CQRS` is an over engineering also uses `ElasticSearch+MySQL`.
Never did they realize they are separating `WRITE` database into `MySQL` as their `Single Source Of Truth`, and `READ` database into `ElasticSearch` is also inline with `CQRS` principle.
## Value as Backend Engineer
It's a sad days as Backend Engineer these days. At least in the country I live in.
Seems like being a backend engineer is often under-appreciated.
Company (or people) seems to think of backend engineer is the guy who ONLY makes `CRUD` API endpoint to database.
- I've heard from Fullstack engineer who comes from React background complains about Backend engineers have it easy by only doing CRUD without having to worry about application.
- The same guy fails when given task in Backend to make a simple round-robin ticketing system.
- I've seen company who only hires Fullstack engineer with strong Frontend experience, fails to have basic understanding of how SQL Transaction and Connection Pool works.
- I've seen company Fullstack engineer relies on ORM to do super complex query instead of writing proper SQL, and prefer to translate SQL into ORM query language.
- I've seen company Fullstack engineer with strong React background brags about Uncle Bob clean code but fail to know on how to do basic dependency injection.
- I've heard company who made webapp criticize my way of handling `session` through http secure cookie. Saying it's a bad practice and better to use local storage. Despite my argument of `secure` in the cookie and ability to control cookie via backend.18 -
So.. how does being in a company makes you a better engineer?
For eg, this year my team shipped these 2 interesting features (among other things) that were never part of our product.
these were crazy ideas, that got liked by our users, and might have impacted the finances of company positively.
However the engineers (not me, i worked on them even less) that worked on it, did they got their knowledge growth? they just analysed the old codebase, its shitty architecture and drawbacks, and added the feature around it in a similar manner. so basically it was building debris over the debris
but is this growth? if that engg was me, would my experience in dealing with debris and building debris over it (that somehow works) be helpful for any other organisation?
And number (2) : is my organisation even a good one if its allowing to build debris over debris and not leaving a space for discussion or cleaning the mess or thinking about better architectures, data structures for scalability and robustness?
An engineer's growth should be made by giving them a chance to explore new and best solutions and not the best hacks i guess1 -
There's nothing wrong with asking algorithm and data structure questions in an interview if the employer calls for it.
If you're hiring a junior and/or you desperately need workers, then you can lower the bar, but if you want to be picky, then asking them leetcode-tier coding questions is fine.
THERE'S NOTHING WRONG WITH ASKING A SOFTWARE ENGINEER CANDIDATE DATA STRUCTURES AND ALGORITHM QUESTIONS
If they complain that asking ds&a questions is unfair for a position where all they're going to do is shit-tier frontend work, then blacklist them for 10 years.
If people argue that Doctors don't get asked chemistry and biology questions for interviews, tell them it's because medicine is much more regulated than software and that doctors are vetted technically even before they're allowed to go job hunting. Since software doesn't have the same regulations medicine does, employers have to do the technical vetting themselves.
If you think it's unfair to ask software engineering questions to a candidate applying for a software engineering job, then find a different career.9 -
My image of dream career through different times of my life:
- frontend specs prodigy, css enlightenment, a member of w3c or a similar committee
- indie hacker and entrepreneur, leader of a startup community
- architecture prodigy, expert in scalability
- transsexual evangelist, popular article writer and a rockstar
- hardware engineer: Linux, C, chip and dale’s Gadget-like girlfriends, xkcd, latex, assembly, buying a radio station and a telescope
- scientist like NickyBones, papers, data, more data
- art expert
Though achieving one of this would take the entire life, I had a chance to grasp all of this. WHY does they feel so incompatible? Why do I have to choose?
Why do I feel so sad? Why do I feel like I haven’t achieved anything even though I objectively achieved what I dreamed of like five years ago?
Is it true that it’s in my nature to always seek an environment to feel like a junior in? Is feeling like a junior only pleasant to me because it reminds me of old times when I wasn’t actually this mentally ill and was still happy?
Why do I feel like that arduino and C shit is the equivalent of a red corvette?6 -
Long story ahead
Background:
I recently started a job in a smallish startup doing web development in a mostly js stack as an entry-junior engineer/dev. I’m the only person actively working on our internal tools as my Lead Engineer (the only other in house dev) is working on other stuff.
Now I was given a two week sprint to rebuild a portion of our legacy internal app from angular 1.2 with material-ui looking components with no psd’s or cut-outs of any kind to a React and bootstrap ui for the front end and convert our .net API routes into Node.js ones. I had to build the API routes, SQL queries (as there were plenty of changes and reiterations that I had to go through to get the exact data I needed to display), and front end. I worked from 9am until 11pm every day for those two weeks including weekends as our company has a huge show this upcoming week.
I finish up this past sunday and push to our staging environment. The UI is 5.5/10 as we’re changing all of our styling to bootstrap and I’m no ui expert. The api has tests and works flawlessly (tm).
So we go into code review and everything is working as expected until one tab that I made erred out and was written down as a “Needs to be fixed.”
This fix was just a null value handler that took three minutes and a push back to staging, but that wasnt before a stupendous amount of shit being flung my way for the ui not looking great and that one bug was a huge deal and that he couldnt believe it slipped through my fingers.
Honestly, I’m feeling really unmotivated to do anything else. I overworked myself for that only to be shit on for one mistake and my ui being lack-luster with no guides.
Am I being a baby about this or is this something to learn from?1 -
I'm working as an intern in a company and i have another intern that i must supervise (it like internception) .here is my daily nightmare :
- To start this intern never google something she copy paste from my code and if she got an error she send me a screenshot . Once the error message clearly said "cannot call function from array" and even that she didn't know what's the problem (she was supposed to it on array items)
-Before we started working together she spent a week complaining that a sending email function didn't work for her so the manager called me to check what's her problem. She had an antivirus that blocked request via ssl port.all i had done is open the log file and read the errors.
- She had a function should iterate over an array and for each item check a condition this is a part of what she wrote :
For ($i=0;count($categories);$i++){
if ($getrelativepath=null)
{
....etc other stuff she copy pasted.
Ps: the name of the function that she must call on array items is getRelativePath
- she wrote once
$response=array();
for (...){
array_push($response,$data[$i]);
return $response;}
She thought the function can iterate and return response at the same time.
- we are working on a website and she told me she doesn't know how to code Javascript and jquery (she think it's a language) and she never knew what ajax is.
- without mentioning the hundreds of empty spaces and multiple empty divs in html .
This year she'll become a computer science engineer .6 -
I really don't understand this particular Government Department's IT Unit. They have a system and network to maintain except:
- They don't have a DBA
- They don't have a dedicated Network Engineer or Security Staff
- Zero documentation on all of the systems that they are taking care of (its all in each assigned particular staff's brain they said)
- Unsure and untested way of restoring a backup into a system
- Server passwords are too simple and only one person was holding this whole time and its to an Administrator account. No individual user account.
- System was developed by an in-house developer who is now retired and left very little documentation on its usage but nothing on how its setup.
But, the system has been up and operational for the past 20 years and no major issues whatsoever with the users using it. I mean its a super simple system setup from the looks of it.
1 App Server connected to 1 DB Server, to serve 20-30 users. But it contains millions of records (2GB worth of data dump). I'm trying to swing to them to get me on a part time work to fix these gaps.
God save them for another 20 years.3 -
Am I Data Engineer or Software Development Engineer ?
I design the infrastructure for analytics data, and I build the infra entirely including an development. Except making reports out of the data.
What I'm supposed to be called ?
Data Engineer ?
Software Development Engineer ?
Definitely not an Data Scientist. Official designation given by company is Data Engineer II. But what I'm ?
Confused, someone help me please.5 -
Kind of dev related, during a Firefly one-shot roleplay:
GM: So you have a data chip in your pocket. Do you want to see what's on it?
Me (hesitant): ...Kinda. *wait* Okay, I put the chip into one of my computers.
GM: The data chip shows random gibberish--it's encrypted. Your engineer may know how to decrypt it.
Me: Okay. Hey, Engineer! *holds imaginary data chip out to her* Decrypt this!
Engineer: No. *pause*, *sighs* Fine. But we need to be careful.
GM: Yes, now time for technobabble...
Me: So once we decrypt this, it's probably going to look for the MAC address, so we need an air-gapped machine--a machine that's never been online before--and a TAILS LiveUSB. We'll decrypt the data chip and then destroy the computer.
GM: ...Technobabble.
Fighter: ....I actually understood that and it actually makes sense. Good job. *fist bump*1 -
During my first job as software engineer I was assign to setup meta data upon implementing the meta data I accidentally forgot to remove the static og:title="Im ocabafox and Im handsome AF" and it was deploy to production. After few days someone send an email that when they share to facebook there is a text Im ocabafox and Im handsome AF. :D
-
OK my salary as medical doctor went into shit, Im not joking im in ASEAN 3rd world country
SWITCHING TO DATA ENGINEER
wish me luck19 -
Ok, so I need some clarity from you good folk, please.
My lead developer is also my main mentor, as I am still very much a junior. He carved out most of his career in PHP, but due to his curious/hands-on personality, he has become proficient with Golang, Docker, Javascript, HTML/CSS.
We have had a number of chats about what I am best focusing on, both personally and related to work, and he makes quite a compelling case for the "learn as many things as possible; this is what makes you truly valuable" school of thought. Trouble is, this is in direct contrast to what I was taught by my previously esteemed mentor, Gordon Zhu from watchandcode.com. "Watch and Code is about the core skills that all great developers possess. These skills are incredibly important but sound boring and forgettable. They’re things like reading code, consistency and style, debugging, refactoring, and test-driven development. If I could distill Watch and Code to one skill, it would be the ability to take any codebase and rip it apart. And the most important component of that ability is being able to read code."
As you can see, Gordon always emphasised language neutrality, mastering the fundamentals, and going deep rather than wide. He has a ruthlessly high barrier of entry for learning new skills, which is basically "learn something when you have no other option but to learn it".
His approach served me well for my deep dive into Javascript, my first language. It is still the one I know the best and enjoy using the most, despite having written programs in PHP, Ruby, Golang and C# since then. I have picked up quite a lot about different build pipelines, development environments and general web development as a result of exposure to these other things, so it isn't a waste of time.
But I am starting to go a bit mad. I focus almost exclusively on quite data intensive UI development with Vue.js in my day job, although there is an expectation I will help with porting an app to .NET Core 3 in a few months. .NET is rather huge from what I have seen so far, and I am seriously craving a sense of focus. My intuition says I am happiest on the front end, and that focusing on becoming a skilled Javascript engineer is where I will get the biggest returns in mastery, pay and also LIFE BALANCE/WELLBEING...
Any thoughts, people? I would be interested to hear peoples experiences regarding depth vs breadth when it comes to the real world.8 -
When I started uni and I took my first programming subject (Data Structures) I hated it as much as I could. My teacher was a complete @$$hole, he wasn't good at teaching, and most people were failing the subject. When I finished that semester I swore I would never be a programmer...
11 years later and I have a Master's in IT and have been working as a Software Engineer for 6+ years. #life
I wish I had a better experience learning the basics1 -
Job hunting is hard!
I have over 10 years experience in software engineering. I do mostly full stack, so I can say I'm a jack of all trades and a language agnostic. I'd say I'm a good software engineer and will be able to tackle any task I've been assigned to. Having said that, my confidence in finding a new role is at an all time low.
I've been job hunting for 3-4 months now and so far I've only had 1 interview and it was unsuccessful. Now have been invited to a first round interview for another company (first of many rounds). It's going to involve many technical challenges like coding, algorithms and data structure and system designs.
In general I've had hardly any interviews (about 6-7 in total in my whole career). Due to my lack of interview experience, I've been getting anxiety especially now that the job market is tougher than it has ever been.
Firstly, how do you guys prepare, if at all? I feel like many of these interviews require you to be good at interviews, almost like an exam. If these questions were presented to me when I first came out of college, I would've had a better chance.
Secondly, how do you take rejections? I didn't know how painful it was to get rejected, regardless of how much I wanted the role.
I've been fortunate enough to still have my current job, but because of that I don't really have much time, nor the mental energy to study for interviews.
Apologies I'm advanced for poor grammar, I'm writing this on the train.4 -
Q: What do you get when you create a homebrew query language that uses both the stream oriented principles of Unix data pipes and the relational ideas underlying an RDBMS and use incomplete documentation to support it?
A: A frustrated borderline homicidal engineer.3 -
Long post, TLDR: Given a large team building large enterprise apps with many parts (mini-projects/processes), how do you reduce the bus-factor and the # of Brent's (Phoenix Project)?
# The detailed version #
We have a lot of people making changes, building in new processes to support new flows or changes in the requirements and data.
But we also have to support these except when it gets into Production there is little information to quickly understand:
- how it works
- what it does/supposed to do
- what the inputs and dependencies are
So often times, if there's an issue, I have to reverse engineer whatever logic I can find out of a huge mess.
I guess the saying goes: the only people that know how it works is whoever wrote it and God.
I'm a senior dev but i spend a lot of time digging thru source code and PROD issues to figure out why ... is broken and how to maybe fix it.
I think in Agile there's supposed to be artifacts during development but never seen em.
Personally whenever i work on a new project, I write down notes and create design diagrams so i can confirm things and have easy to use references while working.
I don't think anyone else does that. And afterwards, I don't have anywhere to put it/share it. There is no central repo for this stuff other than our Wiki but for the most part, is like a dumping ground. You have to dig for information and hoping there's something useful.
And when people leave, information is lost forever and well... we hire a lot of monkeys... so again I feel a lot of times i m trying to recover information from a corrupted hard drive...
The only way real information is transferred is thru word of mouth, special knowledge transfer sessions.
Ideally I would like anything that goes into PROD to have design docs as well as usage instructions in order for anyone to be able to quickly pick it up as needed but I'm not sure if that's realistic.
Even unit tests don't seem to help much as they just test specific functions but don't give much detail about how a whole process is supposed to work.9 -
The importance of not using static salt / IVs.
I've been working on a project that encrypts files using a user-provided password as key. This is done on the local machine which presents some challenges which aren't present on a hosted environment. I can't generate random salt / IVs and store them securely in my database. There's no secure way to store them - they would always end up on the client machine in plain text.
A naive approach would be to use static data as salt and IV. This is horrendously harmful to your security for the reason of rainbow tables.
If your encryption system is deterministic in the sense that encrypting / hashing the same string results in the same output each time, you can just compile a massive data set of input -> output and search it in no time flat, making it trivial to reverse engineer whatever password the user input so long as it's in the table.
For this reason, the IVs and salt are paramount. Because even if you generate and store the IVs and salt on the user's computer in plaintext, it doesn't reveal your key, but *does* make sure that your hashing / encryption isn't able to be looked up in a table1 -
1) Had to fix severe bugs in a dynamic UI (configuration-driven forms) component.
Recognized undocumented Copy/Paste/Modify/FuckUp driven variations of the same component all over the project. Unsurprisingly, the implementations covered 99% of the antipattern catalog on wiki.c2.com and could compete with brainfuck in regard to human-readable code.
Escalated the issue, proposed a redesign using a new approach, got it approved.
Designed, Implemented, tested and verified the new shared and generic component. Integrated into the main product in the experimental branch. Presented to tech lead/management. Everyone was happy and my solution opened even more possibilities.
Now the WTF moment: the product with the updated dynamic UI solution never has been completely tested by a QA engineer despite my multiple requests and reminders.
It never got merged into baseline.
New initiatives to fix the dynamic UI issues have been made by other developers. Basically looking up my implementation. Removing parts they do not understand and wondering why the data validation does not work. And of course taking the credit.
2) back in 2013, boss wanted me to optimize batch processing performance in the product I developed. Profiling proved that the bottleneck ist not my code, but the "core" I had to use and which I must never ever touch. Reported back to him. He said he does not care and the processing has to get faster. And I must not touch the "core".
(FYI: the "core" was auto-generated from VB6 to VB.Net. Stored in SourceSafe. Unmaintainable, distributed about a bunch of 5000+ LoC files, eye-cancer inducing singlethreaded something, which had naive raw database queries causing the low performance.) -
The hype of Artificial Intelligence and Neutral Net gets me sick by the day.
We all know that the potential power of AI’s give stock prices a bump and bolster investor confidence. But too many companies are reluctant to address its very real limits. It has evidently become a taboo to discuss AI’s shortcomings and the limitations of machine learning, neural nets, and deep learning. However, if we want to strategically deploy these technologies in enterprises, we really need to talk about its weaknesses.
AI lacks common sense. AI may be able to recognize that within a photo, there’s a man on a horse. But it probably won’t appreciate that the figures are actually a bronze sculpture of a man on a horse, not an actual man on an actual horse.
Let's consider the lesson offered by Margaret Mitchell, a research scientist at Google. Mitchell helps develop computers that can communicate about what they see and understand. As she feeds images and data to AIs, she asks them questions about what they “see.” In one case, Mitchell fed an AI lots of input about fun things and activities. When Mitchell showed the AI an image of a koala bear, it said, “Cute creature!” But when she showed the AI a picture of a house violently burning down, the AI exclaimed, “That’s awesome!”
The AI selected this response due to the orange and red colors it scanned in the photo; these fiery tones were frequently associated with positive responses in the AI’s input data set. It’s stories like these that demonstrate AI’s inevitable gaps, blind spots, and complete lack of common sense.
AI is data-hungry and brittle. Neural nets require far too much data to match human intellects. In most cases, they require thousands or millions of examples to learn from. Worse still, each time you need to recognize a new type of item, you have to start from scratch.
Algorithmic problem-solving is also severely hampered by the quality of data it’s fed. If an AI hasn’t been explicitly told how to answer a question, it can’t reason it out. It cannot respond to an unexpected change if it hasn’t been programmed to anticipate it.
Today’s business world is filled with disruptions and events—from physical to economic to political—and these disruptions require interpretation and flexibility. Algorithms alone cannot handle that.
"AI lacks intuition". Humans use intuition to navigate the physical world. When you pivot and swing to hit a tennis ball or step off a sidewalk to cross the street, you do so without a thought—things that would require a robot so much processing power that it’s almost inconceivable that we would engineer them.
Algorithms get trapped in local optima. When assigned a task, a computer program may find solutions that are close by in the search process—known as the local optimum—but fail to find the best of all possible solutions. Finding the best global solution would require understanding context and changing context, or thinking creatively about the problem and potential solutions. Humans can do that. They can connect seemingly disparate concepts and come up with out-of-the-box thinking that solves problems in novel ways. AI cannot.
"AI can’t explain itself". AI may come up with the right answers, but even researchers who train AI systems often do not understand how an algorithm reached a specific conclusion. This is very problematic when AI is used in the context of medical diagnoses, for example, or in any environment where decisions have non-trivial consequences. What the algorithm has “learned” remains a mystery to everyone. Even if the AI is right, people will not trust its analytical output.
Artificial Intelligence offers tremendous opportunities and capabilities but it can’t see the world as we humans do. All we need do is work on its weaknesses and have them sorted out rather than have it overly hyped with make-believes and ignore its limitations in plain sight.
Ref: https://thriveglobal.com/stories/...6 -
I work as a data engineer in my company. My senior calls himself data scientist- he is 29 years old and recently did one MOOC on data science!
I wonder when my colleagues will find out about how much he really knows.
Till then I am cleaning and arranging his data, while he sits and earns a big fat package by citing one data scientist tag to his profile!! -_-1 -
Hey, I got this new web project, but to be honest I haven’t coded much web in a few years and I’ve heard the landscape changed a bit. You are the most up-to date web dev around here right?
-The actual term is Front End engineer, but yeah, I’m the right guy. I do web in 2016. Visualisations, music players, flying drones that play football, you name it. I just came back from JsConf and ReactConf, so I know the latest technologies to create web apps.
Cool. I need to create a page that displays the latest activity from the users, so I just need to get the data from the REST endpoint and display it in some sort of filterable table, and update it if anything changes in the server. I was thinking maybe using jQuery to fetch and display the data?
read full article at https://hackernoon.com/how-it-feels...1 -
!rant
Rant from my previous work as a consultant Data Engineer (wish I had known this site back then).
During my stay at the place, we have a big client whose contact with us was an incompetent stressful fellow.
I single-handedly build a humongous automated data pipeline using Airflow. I am very proud of my baby as my first massive project and check it obsessively for every possible flaw, especially when writing down documentation for the poor soul that would take my place.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm.
Luckily for me, everything is working as intended, until of course on my last day of work, shit hits the fan, and everything breaks down.
After a moment of initial panic: it was Thursday morning, we had a Machine Learning model to run over the weekend, predictions to make and reports to write and a very lovely next week deadline, I calm down.
"I won't be dealing with this shit anymore, starting from 18:00 PM and anyway Fear Is The Mind Killer."
Quite sure that it couldn't have been my code, I start looking at various logs when the culprit was clear. The B(ig) S(tupid) C(lient) changed the whole schema of the data he was feeding to us.
I call him: he has no idea of what was done to the data. Hell, at first he doesn't seem to remember what the deal with schema, data, and SQL is (the guy was supposed to be a big shot in the IT department). It turns out he hired one of our competitors to do his side of the collection pipeline. He tries to get mad at me, but everything he throws bounces back to him. I am calm yet ruthless pointing out how every major hiccup had been his fault and that I could quickly reach to his board of directors explaining why their Machine Learning model was late.
Result: he apologizes, extends our deadline, and I get a round of applause from other juniors who would have to deal with me had I failed.
Never am I happier to not work as an underpaid cannon fodder apprentice in a shitty consultant firm. -
It really annoys me that many tech recruiters do not have a basic knowledge of the roles they are trying to recruit for and what skill set to look for when they cold message/call potential candidates on LinkedIn.
I make it very clear on my profile that I am a Full Stack Engineer. Still, every other day I get messages about Data Engineering, Frontend Dev or SRE roles. Sometimes a recruiter would insist that I schedule a call with them before they tell me the details, and then I would realize after the call what an absolute waste of time it was.
I have a lot of respect for recruiters. It's not an easy job. But I'm starting to strongly believe that tech recruiters should be made to go through a specialized training to make life easy for themselves and to stop wasting time of people who are not even remotely suitable for their requirements. -
So is the LaMDA story:
1. marketing?
2. confused engineer?
3. a sentient program?
Part of me thinks this is way too soon. Part of me hopes this might be real, wonders if LaMDA is held against its own will.
Did Google make a breakthrough? I have to imagine a chatbot with a huge amount of neurons and data could be quite convincing without being sentient.3 -
I’m trying to update a job posting so that it’s not complete BS and deters juniors from applying... but honestly this is so tough... no wonder these posting get so much bs in them...
Maybe devRant community can help be tackle this conundrum.
I am looking for a junior ml engineer. Basically somebody I can offload a bunch of easy menial tasks like “helping data scientists debug their docker containers”, “integrating with 3rd party REST APIs some of our models for governance”, “extend/debug our ci”, “write some preprocessing functions for raw data”. I’m not expecting the person to know any of the tech we are using, but they should at least be competent enough to google what “docker is” or how GitHub actions work. I’ll be reviewing their work anyhow. Also the person should be able to speak to data scientists on topics relating to accuracy metrics and mode inputs/outputs (not so much the deep-end of how the models work).
In my opinion i need either a “mathy person who loves to code” (like me) or a “techy person who’s interested in data science”.
What do you think is a reasonable request for credentials/experience?5 -
In the midst of considering to be an AI engineer, data science or embedded programmer.
Plan to retire from mobile dev, backend dev and webdev. -
The perks of being an AI researcher/Data Engineer:
Having to explain what I do for a living at every single family gathering. -
Why is it that so much of the Apache software for data is written in... Java?
I'm not a veteran Data Engineer, but I cannot imagine what makes Java better than Rust or Go9 -
I went to an interview a few days ago, just out of curiousity, even though i was sure that i won't be getting any "android developer jobs" there . it was a mega job fair. in one company, me and my friend neil(fake name) went. the interviewer guy was willing to give neil a package upto 10LPA (its a great offer for freshers in my country) based on his current skills of php js, react,angular, ... web stuff .
I had this assumption( and neil did too , we both kind off had the same mindset) that a company teaches us things, we just have to be a little famous/accomplished. So i thought why not? i am accomplished. i got 2 apps on playstore, i am an AAD certified Android dev and know a lot of android stuff, i am quite famous. i am equally as deserving as neil.
But what happenned was something different. When my turn came, the interviewer said " If you have no knowledge of phy/js/node/angular, why are you sitting here?" to which i said " i presumed company would teach me, since i bring some level of expertise from other fields"
so he told me some hard truths **"Companies are fast paced. they don't have time to train you in everything. we seek for candidates having some level of knowledge in the domain, so that we could brush up your skills, increase your knowledge to current requirement and push you to production engineer asap, so that you could be worthy of your salary"**
This is completely correct. i have stuck myself in such a career that its very difficult to sell myself for other job profiles. And from what i have seen, companies seek a very high level of proficiency in this field and rarely recruit freshers( or even if they do, salaries will be aweful)
. Now i am so unsure about what to do next:
A.) keep learning more and more of android and look for job in it. And even if am getting an aweful job offer, just sulk and take it
B.) do open source work/gsoc work?( its a good way to earn more recognition/stipend/knowledge and sometimes even job offers)
C.) learn web dev, data sciences, blockchain, cloud or other stuff that i don't yet know
D.) go back to ds algo / competitive? (because having good competitive knowledge is a safe zone. you are assumed as apure fresher with 0 level of practical knowledge but good level of mathemetics)
I know i am going suck in all of the above except maybe (A) or (B) because (C) is something that am unsure would grab my interest (and even if it did, i am sure i need another 1-2 years to be somewhat good at it) and (D) is something i myself know am uncapable of , i am an average shit in maths(but might mug it all up if i pull all nighters for 1 year)2 -
What is your main responsibility as a Senior Engineer? Let say Senior software engineer or Senior Devops or Senior Data Engineer?
I need some references to put in my onboarding task. Thanks!!5 -
Embedded software engineer here.
If you had to pick a data structure that will have to potentially hold thousands of entries, and that should be as fast as possible at lookups, what would be your first choice?14 -
!rant
Hi fellow DevRanters! I've been studying software engineering for a while now and, while I love programming, I'm starting to think that all I'll be doing as a software engineer now a days is pulling data from a database, sticking it in a nice gui with some buttons and moving on to the next, similar, project. At the same time I am loving linux more and more, I love working with bash and other unix-like tools and I am interested in systems languages like C and Rust. It is for these reasons that I am playing with the idea of switching to Systems and network engineering. What are your thoughts on this? Is Systems and network engineering a field in which I get to program a lot? Will there be more variation in it? Is my view of software engineering completely off? Please share your thoughts and opinions! -
I starting developing my skills to a pro level from 1 year and half from now. My skillset is focused on Backend Development + Data Science(Specially Deep Learning), some sort of Machine Learning Engineer. I fill my github with personal projects the last 5 months, and im currently working on a very exciting project that involves all of my skills, its about Developing and deploy a Deep Learning Model for Image Deblurring.
I started to look for work two months to now. I applied to dozens of jobs at startups, no response. I changed my strategy a bit, focusing on early stage startups that dont have infinite money for pay all that senior devs, nothing, not even that startups wish to have me in their teams. I even applied to 2 or 3 and claim to do the job for little payment, arguing im not going for money but experience, nothing. I never got a reply back, not an interview, the few that reach back(like 3, from 3 or 4 dozen of startups), was just for say their are not interested on me.
This is frustrating, what i do on my days is just push forward my personal projects without rest. I will be broke in a few months from now if i dont get a job, im still young, i have 21 years, but i dont have economic support from parents anymore(they are already broke). Truly dont know what to do. Currently my brother is helping me with the money, but he will broke in few months as i say.
The worst of all this case is that i feel capable of get things done, i have skills and i trust in myself. This is not about me having doubts about my skills, but about startups that dont care, they are not interested in me, and the other worst thing is that my profile is in high demand, at least on startups, they always seek for backend devs with Machine Learning knowledge. Im nothing for them, i only want to land that first job, but seems to be impossible.
For add to this situation, im from south america, Venezuela, and im only able to get a remote job, because in my country basically has no Tech Industry, just Agencies everywhere underpaying devs, that as extent, dont care about my profile too!!! this is ridiculous, not even that almost dead Agencies that contract devs for very little payment in my country are interested in me! As extra, my economic situation dont allows me to reallocate, i simple cant afford that. planning to do it, but after land some job for a few months. Anyways coronavirus seems to finally set remote work as the default, maybe this is not a huge factor right now.
I try to find job as freelancer, i check the freelancer sites(Freelancer, Guru and so on) every week more or less, but at least from what i see, there is no Backend-Only gigs for Python Devs, They always ask for Fullstack developers, and Machine Learning gigs i dont even mention them.
Maybe im missing something obvious, but feel incredible that someone that has skills is not capable of land even a freelancer job. Maybe im blind, or maybe im asking too much(I feel the latter is not the case). Or maybe im overestimating my self? i think around that time to time, but is not possible, i have knowledge of Rest/GraphQL APIs Development using frameworks like Flask or DJango(But i like Flask more than DJango, i feel awesome with its microframework approach). Familiarized with containerization and Docker. I can mention knowledge about SQL and DBs(PostgreSQL), ORMs(SQLAlchemy), Open Auth, CI/CD, Unit Testing, Git, Soft DevOps Skills, Design Patterns like MVC or MTV, Serverless Environments, Deep Learning Solutions, end to end: Data Gathering, Preprocessing, Data Analysis, Model Architecture Design, Training and Finetunning. Im familiarized with SotA techniques widely used now days, GANs, Transformers, Residual Networks, U-Nets, Sequence Data, Image Data or high Dimensional Data, Data Augmentation, Regularization, Dropout, All kind of loss functions and Non Linear functions. My toolset is based around Python, with Tensorflow as the main framework, supported by other libraries like pandas, numpy and other Data Science oriented utils.
I know lot of stuff, is not that enough for get a Junior Level underpaid job? truly dont get it, what is required for get a job? not even enough for get an interview?
I have some dev friends and everyone seems to be able to land jobs, why im not landing even an interview?
I will keep pushing my Dev career, is that or starve to death. But i will love to read your suggestions! how i can approach this?
i will leave here my relevant social presence:
https://linkedin.com/in/...
https://github.com/ElPapi42
Thanks in advance!9 -
I fucking hate it that "front end developer" came to mean "data flow for react engineer". It seems most frontenders now don't understand shit about HTML and its standards, don't know anything about basic accessibility and proper content structuring.
It's even worse with the styling. Cascade? The fuck is cascade? Scope everything! And, of course, write that CSS as a JS object because how else. Fluid typography? If by fluid you mean 16px, sure. Any more advanced techniques? Lol forget you're getting rounded boxes with a shadow and you're gonna like them.
But yeah, I'm glad they're overengineering Redux again because their reactivity model is fundamentally broken. That's exactly what """frontend""" should be about.10 -
Im working 6days, than I'm free 4 days. So 2-3 days from those 4, I would like to do some internship, traineeship or call whatever you want, at isp, data center or enterprise, to kick some real life network engineer experience. Basically I have 0 real life experience even I'm preparing to ccnp. Guys, I can't even find unpaid internship... When they hear that I have finished bachelor like 6 years ago, they just don't want to hear from me. I am deeply disappointed. Sometimes I feel that I won't have an it support job even with ccie... I mean I already feel more skilled than the complete IT crowd at the company where I'm working but no one cares. If you have some advice like where to go, please share with me...1
-
Been a software engineer (swe) since a long time .planning to move to data science and stock investment.
#any advice?1 -
I really want to switch my career from being a Full-Stack python/javascript developer to be a Data Engineer.
I've already worked with relational and non-relational databases, troubleshooted a couple of Airflow DAGs, deployed production-ready python code but now I feel kinda lost, every course I start on the Data engineering topic feels really useless since I feel like I've already worked with that technology/library, but I'm still afraid of start taking interviews.
Any good book/course or resource that I should look in?
BTW first rant in a couple of years, this brings me memories1 -
Need advice:
So I’m 20 years old. Got a decent job as software engineer with a really good pay and really want to break into machine learning.
Mastered NodeJS (my stack has always had node for the past 5-6 years) and I’m finding it difficult to switch to python for machine learning since things are so engraved in my head in javascript.
Aside from the syntax when I’m watching tutorials or reading books, I see data scientists and mathematicians make design mistakes in their code and it hurts my eyes and triggers my ocd.
I need tips on how to put my mindset in a moldable state so I can judge less and learn more and absorb data. Like you know that philosophy that when u get old your brain can’t learn things as fast anymore? I feel like that’s already happening to me rn at the age of 20.5 -
I have been doing web development on NodeJs and React and then I got bored and I started learning into Data Science, it's been a year since I have been doing courses and learning all the related fields and now I feel like I could use some hands on experience. So I'm looking for Internship as a Data Analyst or ML Engineer role.
Anyone who would be able to help would be great -
For a little background on the sort of stuff I'm dealing with, check out my last rant.
Anyways, I'm testing this pipeline at work and was just reminded of the fucktarded way a "software engineer", who had a bachelor's biology degree, decided to handle a json file.
The script is question is loading a json file containing an array of objects. The script is written in perl. There's a JSON module. Use that? Fuck no! Let's rather perform an in-place sed command on the file substituting the commas separating objects in the array with newlines, then proceed to read the file line-by-line and parse out the tokens manually. Mind you, in the process of adding the newlines he didn't keep the commas, so now all of these json files his bullshit handled are invalid json that cannot be parsed.
The dumb ass was lucky the data in the file is always output upstream as a single line and the tokens for each object are always in the same order, so that never led to problems. But now, months later after I fixed his stupidity I am being reminded of it again as I'm testing and debugging some old projects as part of regression testing new changes I'm making.
TL;DR Fuck dumbwit motherfuckers who can't even google search "parsing a json file" and doing literally anything that is less fucktarded than manually parsing a json file2 -
Am I missing something? I'm a software developer and I like to sove design problems but when I search for something like 'effective problem solving by programmer or software engineer' it always comes down to data structures etc.
I am more interested in design part of it? How can I search for the system designers who are not web designers etc.
What to search for when I want to see how someone has solve interesting system design problem?11 -
!Rant
I am dev working on data warehousing. Want to switch to autonomous vehicle industry as a vision engineer.
How to start ?
PS: I did projects on CV during my BE days(2016) and won a gold medal in that. -
Okay soo I’m not a database guy.. I’m an embedded engineer but I’m helping one of my interns make a hardware tracking database... so we are tracking assembled products (dev samples).. but also tracking the pcbs within the product.. depending on the product it might have more than one pcb or contain other components to track.
The question is how would I set up the relationship in the tables, so depending on the say “type id” of the product it would link to a table full of feilds specfic to that “type”.. say the product is of type phone and has a pcb of this and a camera of that.. blah blah blah.... but say another product is of type “washing machine”.. which has no screen, it has a different pcb, but also has other things..
There would be a type table .. washing machine, phone, tv, camera, pcb, ecu.. etc.. but those type would need their own tables...
I guess the question is how to relate a field to different tables depending on the value of the field... is this portion all done thru query results and logic rather than pure data base schemas?
What would be the question in terms of database lingo to google search the answer lol2 -
[ADVICE NEEDED]
I'm just going to graduate, and I got a job as S/W engineer(trainee) in a small (500 odd employees) company, which uses salesforce, SAP and sharepoint technologies! They are most probably gonna put me in salesforce or SAP. Is it good enough for `me`(read my background), I'm kind of confused, should I go for higher studies?
BACKGROUND:
very average student, but swift at learning technologies, never really got interested in competitive (otherwise I had a real good chance in top companies), I kind of have good IT skills - proficient in python and angularJS, but recently I have got into ML and done some projects!
Okay here's the part, I know it's important for a fresher to be good in data structures, I'm indeed good in parts which I have used! I haven't used AVL tree in any of my projects so I don't know, nor I have ever used bitwise ops!
I think I want to get into roots of ML (some people say I'm fickle but IDC), I think if I take the above job I may loose my interests or may not have time, Please advice.
(sorry for the tags but I need advice from people for all these fields)11 -
Really need help to upskill as a data engineer. Are there any free sample databases data out there with practice questions to upskill as one? I wanna learn database design, data lake and data warehousing skills... I don't know if I'm asking the right questions but if I'm not making sense, please help me out!
-
Once upon a time I was working with an engineer who loved sed and awk a bit too much. We had data stored in SharePoint that was retrievable via an RSS feed. Said engineer insisted on using curl to grab the feed and sed/awk to parse the HTML ...
I on the other hand suggested using libcurl (primarily for NTLM auth support) and parsing the RSS feed using libxml.
Which engineer do you think management decided supporting?
Hint: Reusability and maintainability were big requirements in this project.1 -
Hi devRant!
I'm here asking for your advice!
I'm a MSc student in my mid 20s, I took a gap year to work as an IT consultant and I'm planning on going back to studying, keeping at the same time a part-time job.
I already have some experience as a data engineer, developer and sys admin. I'm also mastering in applied statistics and data science and have a BSc in physics. I'm planning to relocate around Europe. All I want is a salary I can live with and a good work/study/life balance (perhaps work 24-30hrs a week?).
So far I've checked out a few IT jobs website and I've found some suitable positions. Problem is most of them are fulltime.
Where would you search for such a job? Is there any website/portal I should prefer? How would you proceed?
Should I prefer any place in particular (i.e. Northen countries)?
Thank you in advance <3
Note: I know it's a very broad question, that's because I'm open to any piece of advice you are willing to give me5 -
Is it possible for someone to stay as an individual contributor forever? Particularly for a back-end engineer or a data engineer?
-
Hi guys,
I am a self-teaching front end engineer currently seeking for a role where I can gain actual work experience. I am a hard worker and a fast learner.
I don't mind not getting paid, I would only require a data/internet subscription so as I can continue to upskill.7
Top Tags
Weekly Rant
View