Got hired as an SSRS engineer. Walked into work day 1 thinking I was a bad ass with my SQL joes2pros knowledge and quickly (and i mean quickly) got smacked down with what real SQL procs look like........should've renamed my title to SSRS reverse-engineer. Good times.

I had spent the last year working on a online store power by woocommerce with over 100k products from various suppliers. This online store utilized a custom API that would take the various formats that suppliers offer their inventory in and made them consistent. Now everything was going swimmingly initially, but then I began adding more and more products using a plug-in called WP all import. I reached around 100k products and the site would take up to an entire minute to load sometimes timing out. I got desperate so I installed several caching plugins, but to no avail this did not help me. The site was originally only supposed to take three to four months but ended up taking an entire year. Then, just yesterday I found out what went wrong and why this woocommerce website with all of these optimizations was still taking anywhere from 60 to 90 seconds to load, or just timing out entirely. I had initially thought that I needed a beefier server so I moved it to a high CPU digitalocean VM. While this did help a little bit, the site was still very slow and now I had very high CPU usage RAM usage and high disk IO. I was seriously stumped the Apache process was using a high amount of CPU and IO along with MYSQL as well. It wasn't until I started digging deeper into the database that I actually found out what the issue was. As I was loading the site I would run 'show process list' in the SQL terminal, I began to notice a very significant load time for one of the tables, so I went to go and check it out. What I did was I ran a select all query on that particular table just to see how full it was and SQL returned a error saying that I had exceeded the maximum packet size. So I was like okay what the fuck...
So I exited my SQL and re-entered it this time with a higher packet size. I ran a query that would count how many rows were in this particular table and the number came out to being in the millions. I was surprised, and what's worse is that this table belong to a plugin that I had attempted to use early in the development process to cache the site. The plugin was deactivated but apparently it had left PHP files within the wp content directory outside of the actual plugin directory, so it's still executing scripts even though the plugin itself was disabled. Basically every time I would change anything on the site, it would recache the whole thing, and it didn't delete any old records. So 100k+ products caching on saves with no garbage collection... You do the math, it's gonna be a heavy ass database. Not only that but it was serialized data, so when it did pull this metric shit ton of spaghetti from the database, PHP then had to deserialize it. Hence the high ass CPU load. I had caching enabled on the MySQL end of things so that ate the ram. I was really desperate to get this thing running.
Honest to God the main reason why this website took so long was because the load times made it miserable to work on. I just thought that the hardware that I had the site on was inadequate. I had initially started the development on a small Linux VM which apparently wasn't enough, which is why I moved it to digitalocean which also seemed to not be enough, so from there I moved to a dedicated server which still didn't seem to be enough. I was probably a few more 60-second wait times or timeouts from recommending a server cluster to my client who I know would not be willing to purchase it. The client who I promised this site to have completed in 3 months and has waited a year. Seriously, I would tell people the struggles that I would go through with this particular site and they would just tell me to just drop the site; just take the money, just take the loss. I refused to, this was really the only thing that was kicking my ass. I present myself as this high-and-mighty developer like I'm just really good at what I do but then I have this WordPress site that's just beating the shit out of me for a year. It was a very big learning experience and it was also very humbling as well, it made me realize that I really don't know as much as I think I might. It was evidence that there is still so much more to learn out there, I did learn a lot from that experience especially about optimizing websites the different types of methods to do that particular lonely on the server side and I'll be able to utilize this knowledge in the future.
I guess the moral of the story is, never really give up. Ultimately things might get so bad that you're running on hopes and dreams. Those experiences are generally the most humbling. Now I can finally present the site that I am basically a year late on to the client who will be so happy that I did not give up on the project entirely. I'll have experienced this feeling of pure euphoria, and help the small business significantly grow their revenue. Helping others is very fulfilling for me, even at my own expense.

Anyways, gonna stop ranting. Running out of characters. If you're still here... Ty for reading :')

Well it's a bit long but worth reading, two crazy stories in one rant:

So there are 2 things to consider as being my first job. If entrepreneurship counts, when I was 16 my developer friend and I created a small local music magazine website. We had 2 editors and 12 writers, all music enthusiasts of more or less our age. We used a CMS to let them add the content. We used a non-profit organization mentorship and got us a mentor which already had his exit, and was close to his next one. The guy was purely a genius, he taught us all about business plans, advertising, SEO, no-pay model for the young journalists (we promised to give formal journalist certificates and salary when the site grows up)
We hired a designer, we hired a flash expert to make some advertising campaigns and started filling the site with content.

Due to our programming enthusiasm we added to the raw CMS some really cool automation: We scanned our country's radio charts each week using a cron job and the charts' RSS, made a bot to search the songs on youtube and posted the first search result as an embedded video using some reg-exps. This was one of the most fun coding times I've had. Doing these crazy stuff with none to little prior knowledge really proved me I can do anything with the power of will.

Then my partner travelled to work in an internship in the Netherlands and I was too lazy to continue it on my own and it closed, not so surprisingly for a 16 years old slacker boy.

Then the mentor offered my real first job. He had a huge forum (14GB of historical SQL) but it was dying, the CMS version was very old and he wanted me to upgrade it to the latest. It didn't seem hard at first, because there were very clear instructions in the CMS website on how to do that. However, the automation upgrade scripts didn't work well because the forum owners added some raw code (not MVC plugins but bad undocumented code) and some columns to the SQL tables. I didn't give up and decided to migrate between the versions without the scripts. I opened a new CMS and started learning by heart all of the database columns so I can make a script to migrate between the versions. The first tests ran forever because processing 14GB of data on a single home computer is not a task meant to be done. I didn't give up. I made an old forum and compared the table structures and code with my mentor's. I think I didn't exhaustively finish this solution, the task was too big on my shoulders and eventually I gave up. I still owe thanks for that mentor for teaching me how to bare with seemingly (and practically) impossible tasks, for learning not to fear from being a leader and an entrepreneur and also for paying me in time even though I didn't deliver anything 😂

Well here I go my first rant.

A little bit of background:
So I started working my first job a little over a month ago. found devrant about a week in. I was lucky that at a very young age I found programming and liked it (about 6 or 7). I went to college just to get a degree (bachelors of game development).

The job that was a "Great" opportunity that would be bad to let slip by (not a game dev job sadly). Well during the interview they asked me simple thing like what programming languages I know and some simple stuff like that, they never did ask me to demonstrate my knowledge though. Then they went to the weirder questions.
Do you know SQL? yeah at a very base level.
Do you know Excel? I mean I used is a bit, but not very much.
Etc.
A few of the questions felt a little out of place for the field, But it was the only "programming job" that would hire an experienced junior developer, so I took it. Guess I should have asked more questions.
Now I'm here at a job to help replace someone who is retiring. He wasn't a programmer really, but he wrote some code out of necessity well his platform of choice was VBA in Excel. Oh, and that's not the best part, he also dealt with mistakes that happen in the lab (electronics shit). So when ever there is a fuck up I have to go figure out how to search a poorly designed database (that is constantly changing), and today is the day he leaves, so no more help after today. My biggest fear currently is that I wont be able to fill a request that someone makes and I'll be the reason the company is losing money. And with all the stress/burn out that's building up I haven't been working on personal projects, which being my main source of entertainment might be making me depressed. Even when I do work up the effort to work on my projects I don't get very much entertainment. (If anyone has a suggestion for this that would be helpful.)

TIL: Even if the job is a great opportunity don't stop searching and ask a lot of questions.

Thought I would be clever to batch insert a bunch of data into the db with sql, with the catch that the data of a specific column should have a zero-padded number as prefix, counting up from 0. Ask AI for suggestion. Generates some SQL beyond my basic understanding. No way I'm gonna run that! Can't be bothered for an explanation, don't have the time. I realize my assumption to SQL my way through failed right from the start. Instead I let AI generate a python script to create a CSV with the desired data. A few iterations of testing, I take the result file and import it into the db. Done.