Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
--- GitHub 24-hour outage post mortem ---
As many of you will remember; Github fell over earlier this month and cracked its head on the counter top on the way down. For more or less a full 24 hours the repo-wrangling behemoth had inconsistent data being presented to users, slow response times and failing requests during common user actions such as reporting issues and questioning your career choice in code reviews.
It's been revealed in a post-mortem of the incident (link at the end of the article) that DB replication was the root cause of the chaos after a failing 100G network link was being replaced during routine maintenance. I don't pretend to be a rockstar-ninja-wizard DBA but after speaking with colleagues who went a shade whiter when the term "replication" was used - It's hard to predict where a design decision will bite back and leave you untanging the web of lies and misinformation reported by the databases for weeks if not months after everything's gone a tad sideways.
When the link was yanked out of the east coast DC undergoing maintenance - Github's "Orchestrator" software did exactly what it was meant to do; It hit the "ohshi" button and failed over to another DC that wasn't reporting any issues. The hitch in the master plan was that when connectivity came back up at the east coast DC, Orchestrator was unable to (un)fail-over back to the east coast DC due to each cluster containing data the other didn't have.
At this point it's reasonable to assume that pants were turning funny colours - Monitoring systems across the board started squealing, firing off messages to engineers demanding they rouse from the land of nod and snap back to reality, that was a bit more "on-fire" than usual. A quick call to Orchestrator's API returned a result set that only contained database servers from the west coast - none of the east coast servers had responded.
Come 11pm UTC (about 10 minutes after the initial pant re-colouring) engineers realised they were well and truly backed into a corner, the site was flipped into "Yellow" status and internal mechanisms for deployments were locked out. 5 minutes later an Incident Co-ordinator was dragged from their lair by the status change and almost immediately flipped the site into "Red" status, a move i can only hope was accompanied by all the lights going red and klaxons sounding.
Even more engineers were roused from their slumber to help with the recovery effort, By this point hair was turning grey in real time - The fail-over DB cluster had been processing user data for nearly 40 minutes, every second that passed made the inevitable untangling process exponentially more difficult. Not long after this Github made the call to pause webhooks and Github Pages builds in an attempt to prevent further data loss, causing disruption to those of us using Github as a way of kicking off our deployment processes (myself included, I had to SSH in and run a git pull myself like some kind of savage).
Glossing over several more "And then things were still broken" sections of the post mortem; Clever engineers with their heads screwed on the right way successfully executed what i can only imagine was a large, complex and risky plan to untangle the mess and restore functionality. Github was picked up off the kitchen floor and promptly placed in a comfy chair with a sweet tea to recover. The enormous backlog of webhooks and Pages builds was caught up with and everything was more or less back to normal.
It goes to show that even the best laid plan rarely survives first contact with the enemy, In this case a failing 100G network link somewhere inside an east coast data center.
Link to the post mortem: https://blog.github.com/2018-10-30-...6 -
it's funny, how doing something for ages but technically kinda the wrong way, makes you hate that thing with a fucking passion.
In my case I am talking about documentation.
At my study, it was required to write documentation for every project, which is actually quite logical. But, although I am find with some documentation/project and architecture design, they went to the fucking limit with this shit.
Just an example of what we had to write every time again (YES FOR EVERY MOTHERFUCKING PROJECT) and how many pages it would approximately cost (of custom content, yes we all had templates):
Phase 1 - Application design (before doing any programming at all):
- PvA (general plan for how to do the project, from who was participating to the way of reporting to your clients and so on - pages: 7-10.
- Functional design, well, the application design in an understandeable way. We were also required to design interfaces. (Yes, I am a backender, can only grasp the basics of GIMP and don't care about doing frontend) - pages: 20-30.
- Technical design (including DB scheme, class diagrams and so fucking on), it explains it mostly I think so - pages: 20-40.
Phase 2 - 'Writing' the application
- Well, writing the application of course.
- Test Plan (so yeah no actual fucking cases yet, just how you fucking plan to test it, what tools you need and so on. Needed? Yes. but not as redicilous as this) - pages: 7-10.
- Test cases: as many functions (read, every button click etc is a 'function') as you have - pages: one excel sheet, usually at least about 20 test cases.
Phase 3 - Application Implementation
- Implementation plan, describes what resources will be needed and so on (yes, I actually had to write down 'keyboard' a few times, like what the actual motherfucking fuck) - pages: 7-10.
- Acceptation test plan, (the plan and the actual tests so two files of which one is an excel/libreoffice calc file) - pages: 7-10.
- Implementation evalutation, well, an evaluation. Usually about 7-10 FUCKING pages long as well (!?!?!?!)
Phase 4 - Maintaining/managing of the application
- Management/maintainence document - well, every FUCKING rule. Usually 10-20 pages.
- SLA (Service Level Agreement) - 20-30 pages.
- Content Management Plan - explains itself, same as above so 20-30 pages (yes, what the fuck).
- Archiving Document, aka, how are you going to archive shit. - pages: 10-15.
I am still can't grasp why they were surprised that students lost all motivation after realizing they'd have to spend about 1-2 weeks BEFORE being allowed to write a single line of code!
Calculation (which takes the worst case scenario aka the most pages possible mostly) comes to about 230 pages. Keep in mind that some pages will be screenshots etc as well but a lot are full-text.
Yes, I understand that documentation is needed but in the way we had to do it, sorry but that's just not how you motivate students to work for their study!
Hell, students who wrote the entire project in one night which worked perfectly with even easter eggs and so on sometimes even got bad grades BECAUSE THEIR DOCUMENTATION WASN'T GOOD ENOUGH.
For comparison, at my last internship I had to write documentation for the REST API I was writing. Three pages, providing enough for the person who had to, to work with it! YES THREE PAGES FOR THE WHOLE MOTHERFUCKING PROJECT.
This is why I FUCKING HATE the word 'documentation'.36 -
ARGH. I wrote a long rant containing a bunch of gems from the codebase at @work, and lost it.
I'll summarize the few I remember.
First, the cliche:
if (x == true) { return true; } else { return false; };
Seriously written (more than once) by the "legendary" devs themselves.
Then, lots of typos in constants (and methods, and comments, and ...) like:
SMD_AGENT_SHCEDULE_XYZ = '5-year-old-typo'
and gems like:
def hot_garbage
magic = [nil, '']
magic = [0, nil] if something_something
success = other_method_that_returns_nothing(magic)
if success == true
return true # signal success
end
end
^ That one is from our glorious self-proclaimed leader / "engineering director" / the junior dev thundercunt on a power trip. Good stuff.
Next up are a few of my personal favorites:
Report.run_every 4.hours # Every 6 hours
Daemon.run_at_hour 6 # Daily at 8am
LANG_ENGLISH = :en
LANG_SPANISH = :sp # because fuck standards, right?
And for design decisions...
The code was supposed to support multiple currencies, but just disregards them and sets a hardcoded 'usd' instead -- and the system stores that string on literally hundreds of millions of records, often multiple times too (e.g. for payment, display fees, etc). and! AND! IT'S ALWAYS A FUCKING VARCHAR(255)! So a single payment record uses 768 bytes to store 'usd' 'usd' 'usd'
I'd mention the design decisions that led to the 35 second minimum pay API response time (often 55 sec), but i don't remember the details well enough.
Also:
The senior devs can get pretty much anything through code review. So can the dev accountants. and ... well, pretty much everyone else. Seriously, i have absolutely no idea how all of this shit managed to get published.
But speaking of code reviews: Some security holes are allowed through because (and i quote) "they already exist elsewhere in the codebase." You can't make this up.
Oh, and another!
In a feature that merges two user objects and all their data, there's a method to generate a unique ID. It concatenates 12 random numbers (one at a time, ofc) then checks the database to see if that id already exists. It tries this 20 times, and uses the first unique one... or falls through and uses its last attempt. This ofc leads to collisions, and those collisions are messy and require a db rollback to fix. gg. This was written by the "legendary" dev himself, replete with his signature single-letter variable names. I brought it up and he laughed it off, saying the collisions have been rare enough it doesn't really matter so he won't fix it.
Yep, it's garbage all the way down.16 -
!rant
We just did a massive update to our prod db environment that would implicate damn near all system in our servers....on a friday.
Luckily for us, our DB is a badass rockstar mfking hero that was planning this shit for a little over a year with the assistance of yours truly as backup following the man's lead...and even then I didn't do SHIT
My boy did great, tested everything and the switch was effortless, fast (considering that it went on during working hours) and painless.
I salute my mfking dude, if i make my own company I am stealing this mfker. Homie speaks in SQL, homie was prolly there when SQL was invented and was already speaking in sql before shit was even set in spec, homie can take a glance at a huge db and already cast his opinion before looking at the design and architecture, homie was Data Science before data science was a thing.
Homie is my man crush on the number one spot putting mfking henry cavill on second place.
Homie wakes up and pisses greatness.
Homie is the man. Hope yall have the same mfking homie as I do5 -
On my former job we once bought a competing company that was failing.
Not for the code but for their customers.
But to make the transition easy we needed to understand their code and database to make a migration script.
And that was a real deep dive.
Their system was built on top of a home grown platform intended to let customers design their own business flows which meant it contained solutions for forms and workflow path design. But that never hit of so instead they used their own platform to design a new system for a more specific purpose.
This required some extra functionality and had it been for their customers to use that functionality would have been added to the platform.
But since they had given up on that they took an easy route and started adding direct references between the code and the configuration.
That is, in the configuration they added explicit class names and method names to be used as data store or for actions.
This was of cause never documented in any way.
And it also was a big contributing cause to their downfall as they hit a complexity they could not handle.
Even the slightest change required synchronizing between the config in the db and the compiled code, which meant you could not see mistakes in compilation but only by trying out every form and action that touched what you changed.
And without documentation or search tools that also meant that no one new could work the code, you had to know what used what to make any changes.
Luckily for us we mostly only needed to understand the storage in the database but even that took about a month to map out WITH the help of their developer ;)
It was not only the “inner platform” it was abusing and breaking the inner platform in more was I can count.
If you are going down the inner platform, at least make sure you go all the way and build it as if it was for the customers, then you at least keep it consistent and keep a clear border between platform and how it is used.12 -
After doing the work he requested as he wanted he was not happy. So i thought we sit and discuss what he didn't like. I was so wrong.
...
Boss: "...you know what I think you are: a fraud; Masquerading as a developer. The database design you have given is shit. The template I gave you I did in 1 hour. You took half the day."
He gave a simple template to use and he told me to come up with an ecommerce db design via downloading PrestaShop and seeing what is relevant to us.
Me: "what did I do wrong?"
Boss: "you think I don't know what PK means in database design? Why the fuck did you put this here."
Me: "can I expl..."
Boss: "I'm not finished, you been here half the month and what work have you to show for it..."
Me: "I have..."
Boss: "You shut up when I can speaking"
Me: "ok"
Boss: "You have no work to show for the time you have been here. I tell you what to do. I want someone who is proactive. My friend, you will do the work I tell you to do, you understand?"
Me: "yes but can I just say that I have been doing your work I have the contact the various developers as you..."
Boss: " You shut up when your boss is speaking. Can you do this work? (Slightly long pause)
Me: "I can do it. But, I have done the bits of the work you said I do. I was h..."
Boss "don't give me bullshit stories...you haven't done the work..."
Me: "But you have spoken"
Boss:" You know what Im giving you 1 weeks notice if you are not able to do the work. Can you do it?"
That moment!!! I was literally shaking I could have high fived his face with his laptop.
Me: "yes I can"
Boss: "Then get the fuck out of my sight and do it"8 -
Is it just me, or does nobody read their fucking email? Especially when I try explain how a bug isn't trivial to solve because its based on some shitty design decision, that the managers made, that is practically the core of the app. If YOU cant understand the logic with me explaining it to you in plain FUCKING ENGLISH, than how in the name of baby FUCKING jesus, do you expect me to communicate that to the most complicated machines that man has ever built?!? What in the actual fuck do you even do here?!? I could do your job blind-folded, with terminal access to the db, while a monkey was flinging shit at me!5
-
The IT head of my Client's company : You need to explain me what exactly you are doing in the backend and how the IOT devices are connected to the server. And the security protocol too.
Me : But it's already there in the design documents.
IT Head : I know, but I need more details as I need to give a presentation.
Me : (That's the point! You want me to be your teacher!) Okay. I will try.
IT Head : You have to.
Me : (Fuck you) Well, there are four separate servers - cache, db, socket and web. Each of the servers can be configured in a distributed way. You can put some load balancers and connect multiple servers of the same type to a particular load balancer. The database and cache servers need to replicated. The socket and http servers will subscribe to the cache server's updates. The IOT devices will be connected to the socket server via SSL and will publish the updates to a particular topic. The socket server will update the cache server and the http servers which are subscribed to that channel will receive the update notification. Then http server will forward the data to the web portals via web socket. The websockets will also work on SSL to provide security. The cache server also updates the database after a fixed interval.
This is how it works.
IT Head : Can you please give the presentation?
Me : (Fuck you asshole! Now die thinking about this architecture) Nope. I am really busy.11 -
I’m back for a fucking rant.
My previous post I was happy, I’ve had an interview today and I felt the interviewer acted with integrity and made the role seem worthwhile. Fuck it, here’s the link:
https://www.devrant.io/rants/889363
So, since then; the recruiter got in touch: “smashed it son, sending the tech demo your way, if you can get it done this evening that would be amazing”
Obviously I said based on the exact brief I think that’s possible, I’ll take a look and let them know if it isn’t.
Having done loads of these, I know I can usually knock them out and impress in an evening with no trouble.
Here’s where shit gets fucked up; i opened the brief.
I was met with a brief for an MVP using best practice patterns and flexing every muscle with the tech available...
Then I see the requirements, these fucking dicks are after 10 functional requirements averaging an hour a piece.
+TDD so * 1.25,
+DI and dependency inversion principle * 1.1
+CI setup (1h on this platform)
+One ill requirement to use a stored proc in SQL server to return a view (1h)
+UX/UI design consideration using an old tech (1-2h)
+unobtrusive jquery form post validation (2h)
+AES-256 encryption in the db... add 2h for proper testing.
These cunts want me to knock 15-20h of Work into their interview tech demo.
I’ve done a lot of these recently, all of them topped out at 3h max.
The job is middling: average package, old tech, not the most exciting or decent work.
The interviewer alluded to his lead being a bit of a dick; one of those “the code comes first” devs.
Here’s where shit gets realer:
They’ve included mock ups in the tech demo brief’s zip... I looked at them to confirm I wasn’t over estimating the job... I wasn’t.
Then I looked at the other files in the fucking zip.
I found 3 of the images they wanted to use were copyright withheld... there’s no way these guys have the right to distribute these.
Then I look in the font folder, it’s a single ttf, downloaded from fucking DA Font... it was published less than 2mo ago, the license file had been removed: free for Personal, anything else; contact me.
There’s no way these guys have any rights to this font, and I’ve never seen a font redistributed legally without it’s accompanying licence files.
This fucking company is constantly talking about its ethical behaviours.
Given that I know what I’m doing; I know it would have taken less time to find free-for-commercial images and use a google font... this sloppy bullshit is beyond me.
Anyway, I said I’d get back to the recruiter, he wasn’t to know and he’s a good guy. I let him know I’d complete the tech demo over the weekend, he’s looked after me and I don’t want him having trouble with his client...
I’ll substitute the copyright fuckery with images I have a license for because there’s no way I’m pushing copyright stolen material to a public github repo.
I’ll also be substituting the topic and leaving a few js bombs in there to ensure they don’t just steal my shit.
Here’s my hypotheses, anyone with any more would be greatly welcomed...
1: the lead dev is just a stuck up arsehole, with no real care for his work and a relaxed view on stealing other people’s.
2: they are looking for 15-20h free work on an MVP they can modify and take to market
3: they are looking for people to turn down this job so they can support someone’s fucking visa.
In any case, it’s a shit show and I’ll just be seeing this as box checking and interview practice...
Arguments for 1: the head told me about his lead’s problems within 20mn of the interview.
2: he said his biggest problem was getting products out quickly enough.
3: the recruiter told me they’d been “picky”, and they’re making themselves people who can’t be worked for.
I’m going to knock out the demo, keep it private and protect my work well. It’s going to smash their tits off because I’m a fucking great developer... I’ll make sure I get the offer to keep the recruiter looked after.
Then fuck those guys, I’m fucking livid.
After a wonderful interview experience and a nice introduction to the company I’ve been completely put off...
So here’s the update: if you’re interviewing for a shitty middle level dev position, amongst difficult people, on an out of date stack... you need people to want you, don’t fuck them off.
If they want my time to rush out MVPs, they can pay my day rate.
Fuuuuuuuuck... I typed this out whilst listening to the podcast, I’m glad I’m not the only one dealing with shit.
Oh also; I had a lovely discriminatory as fuck application, personality test and disability request email sent to me from a company that seems like it’s still in the 90s. Fuck those guys too, I reported them to the relevant authorities and hope they’re made to look at how morally reprehensible their recruitment process is. The law is you don’t ask if the job can be done by anyone.6 -
Oh boy. Gotta love having a team member (For a School project) be in charge of the Database functionality/design, who has almost no communication skills and basically no clue about how to store data in a data base.
Im talking dates stored as varchar(5), column sizes being way way to small, overall table design being rather terrible, no primary/foreign keys, pretty much... Actually no, everything was being stored as a varchar.
Not only that, but there was a hell lot of data that needed to be stored that wasn't even accounted for in the DB design. He made some code that could be used by our team members for queries, creating tables, inserting data (etc), almost 2,000 lines of it.... And basically nothing was fucking documented at all. I'm talking comments like "Insert data into cities table" and nothing else. More complicated functions had as much, or less documentation. Complete mess.2 -
20 minutes trying to convince my boss (lead dev) that dynamically create tables on DB based on file name to store uploaded json content isn't a good design approach and he is still convinced that 500 lines will really impact performance that much on one table and that this is the best design...
Based on this approach, he wants to create another table with the user who last modified and the "fk" (not sure if he even knows what this is cause none of the fks here point to anything...) would be the table name... now I know where those hideous tables we have here come from...
Don't know if I laugh or cry...3 -
I shit you not. This this a job qualifications qualifications entry level on LinkedIn.
7+ years working as part of a development team and with the following technologies:
Node.js Typescript and Java-based, microservice-driven applications using Spring Boot or similar framework
RESTful API design / microservice architectures
MongoDB or any other NoSQL DB
Message queues e.g. RabbitMQ, Kafka etc.
Modern MV*(MVC, MVVM, etc..) frameworks e.g. React, Angular, Vue etc.
JavaScript and design patterns, CSS and HTML
Modern CSS and view libraries e.g. RxJS, Angular Material, Typescript, JS ES6 etc.
Unit and UI testing using third party tools e.g. Jest, Cucumber, Groovy & Spock, etc.
Bachelor's degree in computer science or related field6 -
Rant!
Been working on 'MVP' features of a new product for the past 14 months. Customer has no f**king clue on how to design for performance. An uncomfortable amount of faith was placed on the ORM (ORMs are not bad as long as you know what you are doing) and the magic that the current framework provides. (Again, magic is good so long as you understand what happens behind the smoke and mirrors - but f**k all that... coz hey, productivity, right?). Customer was so focussed on features that no one ever thought of giving any attention to subtler things like 'hey, my transaction is doing a gazillion joins across trizillion tables while making a million calls to the db - maybe I should put more f**king thought into my design.' We foresaw performance and concurrency issues and raised them way ahead of the release. How did the customer respond? By hiring a performance tester. Fair enough - but what did that translate into? Nothing. Nada. Zilch. Hiring a perf tester doesn't automagically fix issues. The perf tester did not have a stable environment, a stable build or anything that is required to do a test with meaningful results. As the release date approached, the customer launched a pilot and things started failing spectacularly with the system not able to support more than 15 concurrent users. WTF! (My 'I told you so' moment) Emails started flying in all directions and the hunt for the scapegoat was on (I'm a sucker for CYA so I was covered). People started pointing in all directions but no one bothered to take a step back and understand what was causing the issues. Numero uno reason for transaction failure was deadlocks. We were using a proprietary DB with kickass tooling. No one bothered to use the tooling to understand what was the resource in contention let alone how to fix the contention. Absolute panic - its like they just froze. Debugging shit and doing the same thing again and again just so that management knew they were upto something. Most of the indexes had a fragmentation of 99.8% - I shit you not. Anywho, we now have a 'war room' where the perf tester needs to script the entire project by tonight and come up with some numbers that will amount to nothing while we stay up and keep profiling the shit out of the application under load.
Lessons learnt - When you foresee a problem make a LOT of noise to get people to act upon it and not wait till it comes back and bites you in the ass. Better yet, try not to get into a team where people can't understand the implications of shitty design choices. War room my ass!3 -
Worst thing you've seen another dev do? Here is another.
Early into our eCommerce venture, we experienced the normal growing pains.
Part of the learning process was realizing in web development, you should only access data resources on an as-needed basis.
One business object on it's creation would populate db lookups, initialize business rule engines (calling the db), etc.
Initially, this design was fine, no one noticed anything until business started to grow and started to cause problems in other systems (classic scaling problems)
VP wanted a review of the code and recommendations before throwing hardware at the problem (which they already started to do).
Over a month, I started making some aggressive changes by streamlining SQL, moving initialization, and refactoring like a mad man.
Over all page loads were not really affected, but the back-end resources were almost back to pre-eCommerce levels.
The main web developer at the time was not amused and fought my changes as much as she could.
Couple months later the CEO was speaking to everyone about his experience at a trade show when another CEO was complementing him on the changes to our web site.
The site was must faster, pages loaded without any glitches, checkout actually worked the first time, etc.
CEO wanted to thank everyone involved etc..and so on.
About a week later the VP handed out 'Thank You' certificates for the entire web team (only 4 at the time, I was on another team). I was noticeably excluded (not that I cared about a stupid piece of paper, but they also got a pizza lunch...I was much more pissed about that). My boss went to find out what was going on.
MyBoss: "Well, turned out 'Sally' did make all the web site performance improvements."
Me: "Where have you been the past 3 months? 'Sally' is the one who fought all my improvements. All my improvements are still in the production code."
MyBoss: "I'm just the messenger. What would you like me to do? I can buy you a pizza if you want. The team already reviewed the code and they are the ones who gave her the credit."
Me: "That's crap. My comments are all over that code base. I put my initials, date, what I did, why, and what was improved. I put the actual performance improvement numbers in the code!"
MyBoss: "Yea? Weird. That is what 'Tom' said why 'Sally' was put in for a promotion. For her due diligence for documenting the improvements."
Me:"What!? No. Look...lets look at the code"
Open up the file...there it was...*her* initials...the date, what changed, performance improvement numbers, etc.
WTF!
I opened version control and saw that she made one change, the day *after* the CEO thanked everyone and replaced my initials with hers.
She knew the other devs would only look at the current code to see who made the improvements (not bother to look at the code-differences)
MyBoss: "Wow...that's dirty. Best to move on and forget about it. Let them have their little party. Let us grown ups keeping doing the important things."8 -
Please use normalization. Don't store more than is intended in one field and use special characters to separate things. It would make future developers job easy.
7 -
4 years ago I made a personal goal/plan to be a full stack developer. Meaning a good understanding of any development between os level code and web/front end user experience.
Over the years this term 'full stack' has been abused greatly and now basically means 'a javascript developer that generally knows what they are talking about'.
So now, devRant collective I ask you. What do you call a developer with good skills in:
- os level code (c, c++ and os apis)
- database level tech (advanced querying and db aglo/modeling)
- software architecture
- application level (workflow and business logic)
- transport level (protocol design and usage)
- front end tech (graphics programming and event driven paradigm)
- user experience14 -
I don't know what to chose.
The fact that for three months, I had to design a 16-page catalog, when I have no experience and my job is web development;
The fact that I have to do SEO for the site, but that means for my boss that for a one-page long text, we have to find at least 60 (sixty! ) times the occurrences of the keywords;
The fact that when I finally have something interesting to do, the boss finds that it doesn't go fast enough and decide to drop the project even if making a whole new dynamic stock system with the db we have is something hard and long to do;
The fact that when I come to work five minutes late, my boss is at the verge on screaming on me, even if I come ten minutes early every other day;
The fact that when I'm coding, I need concentration, I don't need the boss to give me the phone to answer customers, stop everything I am doing and explain them what products we are selling;
The fact that I am paid the minimum wage for a trainee, and when there's no coffee anymore, we have to buy some ourselves because "you drink way too much coffee, you understand" (three a day, sorry for wanting to stay awake);
The fact that I have asked for one year how many days of vacation I still had, and the only answer they gave to me yet was: "Oh, we have to ask the accountant". I still don't know how many days I have left;
The fact that the site is made only by trainees since the beginning, so circa 2008, and the code is horrible but "it works, so don't touch it". The admin part is in CodeIgniter, the front in laravel 4.2, there are a lot of useless code but we can't touch it because the boss doesn't think it is worth the time.
I almost made a burn-out last year, my doc saw my state right before and made me stop for a week. I still have to work there 'till end of august, then I will have my diploma and find another company to work with. Now, I check everyday on my calendar.6 -
Ok now I'm gonna tell you about my "Databases 2" exam. This is gonna be long.
I'd like to know if DB designers actually have this workflow. I'm gonna "challenge" the reader, but I'm not playing smartass. The mistakes I point out here are MY mistakes.
So, in my uni there's this course, "Databases 2" ("Databases 1" is relational algebra and theoretical stuff), which consist in one exercise: design a SQL database.
We get the description of a system. Almost a two pages pdf. Of course it could be anything. Here I'm going to pretend the project is a YouTube clone (it's one of the practice exercises).
We start designing a ER diagram that describes the system. It must be fucking accurate: e.g. if we describe a "view" as a relationship between the entities User and Video, it MUST have at least another attribute, e.g. the datetime, even if the description doesn't say it. The official reason?
"The ER relationship describes a set of couples. You can not have two elements equal, thus if you don't put any attribute, it means that any user could watch a video only once. So you must put at least something else."
Do you get my point? In this phase we're not even talking about a "database", this is an analysis phase.
Then we describe the type dictionary. So far so good, we just have to specify the type of any attribute.
And now... Constraints.
Oh my god the constraints. We have to describe every fucking constraint of our system. In FIRST ORDER LOGIC. Every entity is a set, and Entity(e) means that an element e belongs to the set Entity. "A user must leave a feedback after he saw a video" becomes like
For all u,v,dv,df,f ( User(u) and Video(v) and View(u, v, dv) and feedback(u, v, f) ) ---> dv < df
provided that dv and df are the datetimes of the view and the feedback creation (it is clear in the exercise, here seems kinda cryptic)
Of course only some of the constraints are explicitly described. This one, for example, was not in the text. If you fail to mention any "hidden" constraint, you lose a lot of points. Same thing if you not describe it correctly.
Now it's time for use cases.
You start with the usual stickman diagram. So far so good.
Then you have to describe their main functions.
In first order logic. Yes.
So, if you got the point, you may think that the following is correct to get "the average amount of feedback values on a single video" (1 to 5, like the old YT).
(let's say that feedback is a relationship with attribute between User and Video
getAv(Video v): int
Let be F = { va | feedback(v, u, va) } for any User u
Let av = (sum forall f in F) / | F |
return av
But nope, there's an error here. Can you spot it (I didn't)?
F is a set. Sets do not have duplicates! So, the F set will lose some feedback values! I can not define that as a simple set!
It has to be a set of couples, like (v, u), where v is the value and u the user; this way we can have duplicate feedback values in our set.
This concludes the analysis phase. Now, the design.
Well we just refactor everything we have done until now. Is-a relations become relationships, many-to-many relationships get an "association entity" between them, nothing new.
We write down on paper every SQL statement to build any table, entity or not. We write down every possible primary key or foreign key. The constraint that are not natively satisfied by SQL and/or foreign keys become triggers, and so on.
This exam is considered the true nightmare at our department. I just love it.
Now my question is, do actually DB designers follow this workflow? Or is this just a bloody hard training in Pai Mei style?6 -
I'm curious..
When does programming suck for you, and when is it fun?
Like I hate programming, when I run into an obscure use case that opens up some serious errors with my some, or gasp, all, of my architecture and forces me to rethink everything - especially DB design, ugh.
I love programming when my architecture and DB design create naturally readable code and everything falls into place and I feel like a genius.
I guess, in short.... plan before you code?
And then, plan again.
But don't plan too much.
The love/hate of my programming life summed up right there I think.
How about you?10 -
Several years ago, I heard from a friend who was doing assignments for students on the side. Quite a hustle. His story began when he wanted to figure out why can't these students be able to draw their own database tables, relationships, UML, etc. That's what school has to be teaching them and then he was told that they were learning through MS Access. He goes and tell me that even though this is a lame way of teaching database design, its definitely easier to explain through hands-on and less typing mistakes, as according to the lecturer he met. Making the explanation more visually appealing and helpful for understanding.
OK I get it, but somehow that taught them the wrong way of database design from the beginning. I'd prefer getting them to start writing SQL commands from day 1 and play em at some DB VM. Keep em as real as it gets.
Now I have my own students asking for help in their assignment and also asked for tutoring lessons in web development. So I gave them the crash course in HTML, CSS and Javascript. I've asked them if they've used anything of what I taught them in school. They go and tell me that they've been taught web development through Wordpress. Oh WTF!? I havn't talked to their lecturer yet but it better be a really good explanation to teach these youngsters in a flawed and bloated PHP CMS framework for "web development".2 -
I'm writing a devrant like site, so a kind of forum that supports live chat under every article. Login will be just username and password to stay anonymous. Email is optional for password reset. Also it won't have password requirements. Who cares if user uses insecure password. I do like the devrant avatar thing. I will use the ducky generator instead. So everyone on the site is a custom duck. K-SASS prolly never expected his generator to be used anywhere. The requirement of this site is that it scales very well. I have db calls of 0.006s, this is for persistent data only and will be used by all site instances. I expect that it can handle many clients concurrent as long I do not return more than 30 rows or so. Events get handled by a self written pubsub server.
All sounds great and development goes fine. But why is this a rant? Because the same thing as always is biting me, I can't design a site at all. I know how but I don't have any feeling for design at all making me almost incapable of building an attractive site. The only thing I can 'design' is an application in bootstrap or smth. I spend so much time one design while I don't like to do it ironically. But looks of site is almost as important as an good working site. Good working site doesn't get used if looks bad in many casee. This is since the start of my career an issue and it sucks that I appearantly can't deliver a whole site on my own meeting my standards.
My backend work is top notch tho. Btw, this application is not to be an alternative for devrant. I do not think I can attract more users than it already has and I've seen two communities disappearing once because someone decided to make a new one, took half of community with him and both communities died after short while.
End product of this project is a working project, not a live site hosted somewhere. It's pure about mixing mostly self written tech to get the best performance. Reinventing wheel on many levels. I wanted maybe to do the site in C but decided that it's way to much work for the value. I change the site so rapid since I don't have decent plan that python aiohttp is the best choice in amount of writing it yourself and fast. It's very lightweight.
More a story than a rant, sorry27 -
Got some Freelance work(PHP) via my friend so i said ok lets see. Those were some college students and wanted their project to be deployed that their seniors built and none of them knew how to fix the issues so here i step in.
Nightmare of a design, no signs of framework, html/css looks like it was designed by a 10th grader, no prepared queries, every file uses db config seperately so a minor change would require me to edit those 50+ php files... overall a broken mess. FML.5 -
Four steps of professional development:
1. Simple and bad
2. Complicated and bad
3. Complicated and good
4. Simple and good
At CSS and frontend in general, I'm easy four, straight up. At architecture, I'm perhaps two in devops/docker/kubernetes/other crap and three at DB design. At electrical engineering and embedded stuff, I'm 1, no questions asked.
What are your rankings?1 -
Hey all, can you guys recommend some great books on database theory and design. Something aimed towards intermediate towards advanced. It doesn't have to be sql, I just want to learn how to make a kick a#@ db.
Thanks!!!!!!!
6 -
May's last week was very hectic. I had just finished my final exams and there were going to be semester project evaluations in that whole week.
@safiullah and me had decided to make a whole Social Network with all features in it, for the DB course project.
All other classmates were making small management systems like ticket booking and etc.
We thought that if we really wanted to learn DB concepts then we should come up with something different than a management panel.
Hence we did it. This was the first time we used a framework. Well, I had written that PHP framework while i was learning about how frameworks work and the way they are made. So it wasn't a big thing but it was something which could be used as a base for clean and organized code.
It took about a month of commits and pushes and it resulted in a very good social network. It had all the features and algorithms present in a starter social network.
For us students, we were happy to see what a fine job we had done. We learnt a lot and used new concepts.
When we went to the instructor, she asked us to sit down and show the project. @safiullah placed the laptop, and logged out from the social network so that he could show her a demo.
She exclaimed,"Why did you do it (Log out) ?"
He replied: "To show you how it works🤷🏻♂️"
She:"Get to the previous state and leave it"
Then she asked different questions like what was a post request in php and how it differed from get? what library for DB connection was used... etc.
We explained each and every step.
She saw the frontend design and said "You've just added text to the elements" as If we were showing her a theme demo with hard coded text accomplished by inspect element.
She did not take a look at any other page than the one we had shown her at start. She navigated to no other page and asked nothing about what total features were implemented and how they were done?
Then she said Thank You and we left.
After some days marks were uploaded in LMS and we were just two points above the average.
She took no look and gave us the least when our project was the best.
I'm 100 percent sure she thought that we were showing her a project copied from somewhere else. 🤣4 -
So we finished our requirement ( barely) for a new client. Next is data modelling and system design.
We started with data modelling. Unfortunately the lead developer does not know the difference between database and data modelling.
me: hey bro, we'll do the database and stuff later, now let's focus on data modelling.
him: (acting like he knows) yeah I have developed a sample design for the "data model".
me: no this is database design.
him: what's the difference?
me: dude, they're totally different. Okay, simple explanation data model is what you want to store, whereas DB design is how you store it.
him: So, if I am not wrong, it's implied that you know what to store if you are talking about how to store it.
me: but you don't know what it is you want to store yet. And one of them precedes the other.
him: Okay, let's start with DB design.
me: What?????? you want to build a house without a plan??? That's it for me I am done !!!
I left the project yesterday, later I heard that, the team members are coders, who think that developing a software is all about coding and fixing errors. -
So... I might have to build a survey and analysis tool to be used online nation wide (the final client is the government).
The bad news, it's probably going live in a week.
Even though 90% I wrote and tested in the last 3 years (matrixes,formulas,dB, Interface elements ) from previous runs, I have to handle fronted, databases, math, testing and design.
Over a daily changing methodology and session workflow (by my direct client).
No sleep for the next few days 😭6 -
Ok so first technical blog post/rant cuz I just reduced a lot of debt... Prolly gonna put this in an email to my boss (he says progress improvement is now a priority but there are some problems as listed below):
So last week, I spent a lot of time investigating db logs manually to figure out a prod issue: tiring, time consuming, and not very effective.
This week I built an app. It took a few days but having the time to design it correctly, it is very powerful.
So in order to really do process improvement, you need to have: dedicated the time, the problem solving mindset (the right people), and the understanding of what the problem is and why so you can build a good solution (time and people).1 -
More and more, I am getting frustrated/depressed from the attitude of our customers who complain, moan and get angry about issues in their infrastructure, while at the same time, refusing to pay more so the issues could be mitigated.
Like, a client's angry with us today for having one of their non-production-critical databases inaccessible for... Hmm... About 8 hours now (So a whole workday).
Like... I get it, some of your employees couldn't work with it offline, but like... What the hell do we do? You keep data from as far back as several years ago in there, without partitioning, without exports, in a mix of innodb and myisam, so when the DB crashes, and its replication has to be reset from zero, reimporting all the data takes hours upon hours, and importing .sql files just takes time.
Or another client who got angry when their app fell out of the internet, cuz one of their myisam-based log tables crashed, and had to be repaired, with data spanning several years back, meaning it took hours to fix...
The more I work with these "basic" and "simple" infrastructure designs that is *not* redundant, or HA, the more I wonder -- How do the big names out there do it? How do you design systems with fault tolerance so a single DB table crash doesn't lead to the whole app getting inaccessible?
We have... One, exactly one, client, who uses MariaDB with Gallera, and that cluster is *amazing*, it just keeps chugging along, without a care in the world. But it cost them quite a lot, as they had to buy 3 DB servers, instead of 1...1 -
How do you feel about not creating database tables for objects that only exist in relations?
For example, I have made a wiki engine. Because nothing on wiki pages can actually change, they aren't an entity. Revisions are an entity, and they refer to the title of the page which was changed. The same application also includes two non-version-controlled directed graphs between the pages (element of category and navigation log), which are represented by tables that link two titles. Of course the indexes are all set up so that it works like a foreign key would, but there is no Page or Article table. -
"Back-end" ahahahaha
Translating:
Back-end developer (São Paulo / Brazil)
Vacant job: Back-end developer
Assignments: Work as a Back-end developer, PHP, and MySQL. Develop tools/applications for the web with a DB.
Technical skills: Technical knowledge in PHP, Framework Code Igniter, HTML5, JavaScript (jQuery, Bootstrap, Json, and Ajax), responsive layouts and WordPress.
Projecting and modeling DB Knowledge. Good knowledge of usability, Cross-browser (IE7+, Chrome, Firefox, and Safari).
Desirable: Knowledge on Graphic Design and Front-end projects
1 -
Before get get source code for freelance job, the person who cantact me say the job is to continue the project for some update and tweak.
The UI from design is beautiful and he gave good explaination for the project and the update, continue to conversarion, negosiation and deal.
but he is not the IT guy and also the project is not his work or something that he do previosly. All the person who work on that project is already leave and not contactable.
And here that I get:
- source code
- domain cred.
And here what's missing:
- documentation
- .env file
- db backup / old db cred.
- server and hosting cred.
And after some hour of learning the code I find out that:
- latest commit was 2 year ago and different from production version.
- most of the branch is RnD.
- the code have many wtf/minute lol
And for now I still re-negotiate with the person who give me the project with 2 suggestion from me.
- continue with this code with condition, he need to search for the missing part at least backup db or documentation.
- recreate the project with more time
And here's one funny part of the code.
randomNumber(){
return 5 // this number was choose by dev team at random
}1 -
Programming at a job to me is no longer creating something fun and valuable; it's more like figuring out why shit doesn't work, con-stant-ly.
It' s like coming in to your desk every morning, dreading the day because there's yesterday's shit to fix. "Hmm, what shall today be like? Oh yes, troubleshooting why my database model doesn't work, redesign it completely and break my mind over db details. The next day? Having to redesign my classes to implement new patterns because apparently the current design isn't good enough." Even if you work on new deliverables, that's just new problems in disguise anyway.
Pleasant? Not really.
lol.3 -
In our application we now need the possibility to add visit hours to addresses. Each day has 3 possible visit moments with a from and to hour and some indicator, and a comment, like this:
monday_visit_from_1, monday_visit_to_1, monday_visit_ind_1, (repeat for 2 and 3), monday_visit_comment (repeat for all 7 days)
For "performance" they (db admin) decided to add all the fields of all the days as separate columns in the address table, 70 in total.
Besides being a horrible design it is a fucking pain to work with, like find if an address can be visited on a given day and time we need to check a subset of columns based on the day and the 3 moments in this day.
Is this really more performant than an extra table to hold the visit moments and is this something that gets done in more places?1 -
Bullshittery continues. This time around, absolutely innocent, clamav is root cause. For once not incompetent idiot, but piece of software. IDK if that makes me happy or upset.
So our email server that I configured and took care of died. RIP. Damn, better put it back together ASAP. So Im under pressure, while still pissed at everything that I ranted before (actually my last 2 rants were throttled, and in total all of that happened past 60 minutes but devrant rate limiting) I start auditing logs. You imagine, we kindda need it NOW, and it's second time last month clamav is pulling stunts and MTA refuses (properly) to work without antivirus. So pressurized, I look at logs, what the fuck went wrong.
clamav deamonize() failed - cannot allocate memory
Hmm. Intresting, but sounds like bullshit. I know server is quite micro becouse they wanted to save on costs as much as possible, but it has well over half a gig free ram just before it crashes (like 800MB) with that message. Is it allocating almost gig in one call or what? Looked carefully at trusty htop while it was starting, and indeed, suddenly it just dies with quite a bit of ram free, almost as much as it weights already. And I remember booting it up when I was configuring it, and it had fair bit of headroom.
Google, help me friend... Okay, great, so apparently at some point clamav loads virus DB into ram (dafuq?), and than forks, which causes spike of 2x the ram usage, and than immidietely frees it up.
Great, that sounds like great design decision... At least I know, I can just slap on SWAP file, restart it and call it a day.
It worked, swap file is almost empty (used 15megs, 900 megs free ram, whatever).
That leaves me wandering, who figured out to load DB to ram? That means pretty much that clamav will eat a little bit more ram each vir db update, and that milisecond "double ram" spike will confuse innocent people who just wanted to run clamav and it worked last *long period of time* and now crashes without warning without any changes to configuration.
Maybe there is logical explanation, I want to know it.8 -
I need an advice!
I'm a back-end dev with 5yrs of experience.
Our team initially started with 7 back-end engineers, and 1 developer was acting as the "tech lead". I was happy as an individual contributor and I enjoyed it a lot. I learned a lot of things.
After 1 year, our team got downsized. All other BE devs got replaced by 2 new engineers - one with 7 yrs of experience who fckin doesn't even know how to google and drop a constraint in DB, and another with "13 years" of experience who's a credit-grabber and all talk.
Now here's my problem. I feel that I've been "unofficially" given the role of a lead developer - the one who needs to lead code reviews, mentor others, decide on the higher level design, chase people for deployment approvals, managing 3rd-party dependencies, and forced to become the "coordinator".
This stresses me and burns me out. I just want the peace of becoming an individual contributor.
What can I do at this point?3 -
I can't stand when people spend a single day familiarizing themselves with a new technology or concept and then come to the conclusion that's it doesn't work and really the old way is great. Not saying all new things are better. In fact, I'm probably more in favor of tried and true methods than shiny new methods. But one day? Really? That's all it took? In this particular case it's code-first DB development. Again, I'm not a fan myself really. But I have a co-worker who said creating tables and and schemas is much harder using code-first instead of DB first. I mean... Neither are hard. I personally think it's easier for basic things like tables and schemas but either way it's not hard. Now SQL triggers and index's all that fun stuff? Yeah code first is probably more complicated (I'm clearly not a database expert or anything). But a day? Really? You know enough to force a design paradigm on the whole company now? Wtf.3
-
I work in a small team. As the senior dev I tens to focus on important tasks that shape the core of the product but some times I can’t divide my self when there are multiple tasks at hand, so I pass some tasks to the an other mid level dev.
So the task was to create an automation in order to CD (continuously deliver) an order from WHMCS of the (git versioned) product to customers UAT, PROD envs.
To get a background this is an old guy with “constricted” experience in PHP/jQuery/Joomla/Wordpress.
So when we were breaking up the tasks he told me he would like to implement this so i gave him the task as i was busy with core features.
I was like what could go wrong? I know he doesn’t know much about CI/CD but he can read right? He will google right? He will search for CI/CD solutions that do this out of the box right? He will design on paper or what ever and do small POCs right? He will design the flow first before starting the implementation right? RIGHT?
So fast forward to today I had a call with him this morning about some DB staff. And he wanted to show me his progress…
His solution is:
(parentheses is my brain)
1. Customer completes WHMCS order (perfect)
2. Web Hook 🪝 action (YES)
3. cpanel gets source and “automatic!” Init, all using pure PHP code ignoring the usage of the current framework (ok… something is missing)
4. cpanel web hooks(?) WHMCS to send email to customer with the envs initial setup page(?)
5. Customer opens link and adds setup info (ok fuck, fuck, fuck)
(Ok stay cool composed, lets ask some questions maybe he thought it all in a cool way I can’t get my mind around)
Me: So how are you gonna get the correct version from the repo to the env and init the correct schema?
Dev: I haven’t thought about it yet.
Me: Are we gonna save each version to a file system then your code is going to fetch them?
Dev: I haven’t really thought about it we will see. But look on customer init user setup I implemented a password strength validation and it also checks if the password is the same.
So after this Pokémon encounter I politely closed teams. Stood up drank some (a lot) coffee ☕️. Put out the washed laundry while reflecting on life’s good things, while listening to classical music 🎼 .
Then I sat on my office chair drank some more coffee, put some linking park starting with in that order:
“Numb” then “What I’ve Done” and ended with “In the end, it does really fucking matter” -
Architecture for Java REST API going to build/port from existing NodeJS one.
So Spring Boot + *
Lots of concurrent requests and large MongoDB calls. Current APIs use like 4GB memory for each instance because they don't use stream/pipe the response. Hold all data in memory and then return it all at once to user.
And well we expect more load in the future, so want to do this the right way.
So my understanding since this morning, is there's the blocking? MongoClient, (find* returns List) and now a Reactive MongoClient which is very async and like JS promises. Based on Pub, Sub model.
But the downside of JS promises was callback hell.
So actually 2 questions.
1. For each request, the db call done using the same MongoClient/db connection such that if there are 2 requests one would block the other?
2. Reactive Mongo would be non-blocking by design so would be better to support streamed responses?8 -
JPA my friend ... JPA why are you like this? JPA why do hate me so much? JPA, let's have a word ...
How come you are so far away from real-world problems, so cumbersome to use, so ugly (criteria API), so wrong and inconsistent?
Oh, what it's all your parents fault? Oh come, on that can't be, right? Did you have a bad childhood?
Your parent's were fucking crack-smoking maniacs which didn't know a single bit about actual databases?
They design you as an API without actually trying you out in the wild? And then they patched up together with some essential DB stuff, like friggin indexes? Not even tried to make this API consistent nor really functional?
Oh poor, you little JPA ... -
I started looking into building my Android app but wanted to see if I could get a refresher on a few things. The starter template for the Nav Layout isn't exactly functional.
So first question is anyone know any resources like an actually functioning demo project.
Also I need DB access but want to open any db file given the *.db path and the DAO should be persistent, share across all fragments/activities. What would be the best design, way of doing that in Android though.
I don't think you can pass the object between activities but what about fragments. I'm thinking the main app opens the DB and then can pass a DAO Interface to all the fragments to use?2 -
Visualize the entire complexity of the content within the project so that you know what data users will need to access, and compartmentalize those in to separate modules that you can build on over time. Think about any limitations with accessing that data (does the user have that role, what if the data is accessed simulateously, how to handle the same user accessing from different devices etc).
Think about the devices being used - is it going to be a website, an app, both? How best then to access the data? Direct access to a database, or an API system?
Then think about the front-end design and how to simplify the view right down as much as you can. Again, break it down in to modules.
Then decide on the technology you want to use, and what libraries would help simplify things.
These days I like to use JSON API's to access DB content because app and web technologies change quite often but the API will be accessible to whatever I use to build it.
For websites I love using Laravel, which simplifies the back-end tasks, and mdbootstrap which simplifies the front-end tasks and looks "appy".
Top Tags
Weekly Rant
View