Oh, man, I just realized I haven't ranted one of my best stories on here!

So, here goes!

A few years back the company I work for was contacted by an older client regarding a new project.

The guy was now pitching to build the website for the Parliament of another country (not gonna name it, NDAs and stuff), and was planning on outsourcing the development, as he had no team and he was only aiming on taking care of the client service/project management side of the project.

Out of principle (and also to preserve our mental integrity), we have purposely avoided working with government bodies of any kind, in any country, but he was a friend of our CEO and pleaded until we singed on board.

Now, the project itself was way bigger than we expected, as the wanted more of an internal CRM, centralized document archive, event management, internal planning, multiple interfaced, role based access restricted monster of an administration interface, complete with regular user website, also packed with all kind of features, dashboards and so on.

Long story short, a lot bigger than what we were expecting based on the initial brief.

The development period was hell. New features were coming in on a weekly basis. Already implemented functionality was constantly being changed or redefined. No requests we ever made about clarifications and/or materials or information were ever answered on time.

They also somehow bullied the guy that brought us the project into also including the data migration from the old website into the new one we were building and we somehow ended up having to extract meaningful, formatted, sanitized content parsing static HTML files and connecting them to download-able files (almost every page in the old website had files available to download) we needed to also include in a sane way.

Now, don't think the files were simple URL paths we can trace to a folder/file path, oh no!!! The links were some form of hash combination that had to be exploded and tested against some king of database relationship tables that only had hashed indexes relating to other tables, that also only had hashed indexes relating to some other tables that kept a database of the website pages HTML file naming. So what we had to do is identify the files based on a combination of hashed indexes and re-hashed HTML file names that in the end would give us a filename for a real file that we had to then search for inside a list of over 20 folders not related to one another.

So we did this. Created a script that processed the hell out of over 10000 HTML files, database entries and files and re-indexed and re-named all this shit into a meaningful database of sane data and well organized files.

So, with this we were nearing the finish line for the project, which by now exceeded the estimated time by over to times.

We test everything, retest it all again for good measure, pack everything up for deployment, simulate on a staging environment, give the final client access to the staging version, get them to accept that all requirements are met, finish writing the documentation for the codebase, write detailed deployment procedure, include some automation and testing tools also for good measure, recommend production setup, hardware specs, software versions, server side optimization like caching, load balancing and all that we could think would ever be useful, all with more documentation and instructions.

As the project was built on PHP/MySQL (as requested), we recommended a Linux environment for production. Oh, I forgot to tell you that over the development period they kept asking us to also include steps for Windows procedures along with our regular documentation. Was a bit strange, but we added it in there just so we can finish and close the damn project.

So, we send them all the above and go get drunk as fuck in celebration of getting rid of them once and for all...

Next day: hung over, I get to the office, open my laptop and see on new email. I only had the one new mail, so I open it to see what it's about.

Lo and behold! The fuckers over in the other country that called themselves "IT guys", and were the ones making all the changes and additions to our requirements, were not capable enough to follow step by step instructions in order to deploy the project on their servers!!!

[Continues in the comments]

CR: "Add x here (to y) so it fits our code standards"
> No other Y has an X. None.

CR: "Don't ever use .html_safe"
> ... Can't render html without it. Also, it's already been sanitized, literally by sanitize(), written by the security team.

CR: "Haven't seen the code yet; does X change when resetting the password?"
> The feature doesn't have or reference passwords. It doesn't touch anything even tangentially related to passwords.
> Also: GO READ THE CODE! THAT'S YOUR BLOODY JOB!

CR: "Add an 'expired?' method that returns '!active'?"
> Inactive doesn't mean expired. Yellow doesn't mean sour. There's already an 'is_expired?' method.

CR: "For logging, always use json so we can parse it. Doesn't matter if we can't read it; tools can."
CR: "For logging, never link log entries to user-readable code references; it's a security concern."
CR: "Make sure logging is human-readable and text-searchable and points back to the code."
> Confused asian guy, his hands raised.

CR: "Move this data formatting from the view into the model."
> No. Views are for formatting.

CR: "Use .html() here since you're working with html"
> .html() does not support html. It converts arrays into html.

NONE OF THIS IS USEFUL! WHY ARE YOU WASTING MY TIME IF YOU HAVEN'T EVEN READ MY CODE!?

dfjasklfagjklewrjakfljasdf

Thus really cracked me up 😂😂

Oh boy I got a few. I could tell you stories about very stupid xss vectors like tracking IDs that get properly sanitized when they come through the url but as soon as you go to the next page and the backend returns them they are trusted and put into the Dom unsanitized or an error page for a wrong token / transaction id combo that accidentally set the same auth cookie as the valid combination but I guess the title "dumbest" would go to another one, if only for the management response to it.

Without being to precise let's just say our website contained a service to send a formally correct email or fax to your provider to cancel your mobile contract, nice thing really. You put in all your personal information and then you could hit a button to send your cancelation and get redirected to a page that also allows you to download a pdf with the sent cancelation (including all your personal data). That page was secured by a cancelation id and a (totally save) 16 characters long security token.

Now, a few months ago I tested a small change on the cancelation service and noticed a rather interesting detail : The same email always results in the same (totally save) security token...
So I tried again and sure, the token seemed to be generated from the email, well so much about "totally save". Of course this was a minor problem since our cancelation ids were strong uuids that would be incredibly hard to brute force, right? Well of course they weren't, they counted up. So at that point you could take an email, send a cancelation, get the token and just count down from your id until you hit a 200 and download the pdf with all that juicy user data, nice.

Well, of course now I raised a critical ticket and the issue was fixed as soon as possible, right?
Of course not. Well I raised the ticket, I made it critical and personally went to the ceo to make sure its prioritized. The next day I get an email from jira that the issue now was minor because "its in the code since 2017 and wasn't exploited".

Well, long story short, I argued a lot and in the end it came to the point where I, as QA, wrote a fix to create a proper token because management just "didn't see the need" to secure such a "hard to find problem". Well, before that I sent them a zip file containing 84 pdfs I scrapped in a night and the message that they can be happy I signed an NDA.

One thing when working with a ton of data:
If there is a slight, infinitesimal probability that something will be wrong, then it will 100% be wrong.

Never make assumptions that data is consistent, when dealing with tens of gigabytes of it, unless you get it sanitized from somewhere.

I've already seen it all:
* Duplicates where I've been assured "these are unique"
* In text fields that contain exclusively numeric values, there will always be some non-numeric values as well
* There will be negative numbers in "number sequences starting with 1"
* There will be dates in the future, and in the far far future, like 20115 in the future.
* Even if you have 200k customers, there will be a customer ID that will cause an integer overflow.

Don't trust anything. Always check and question everything.

HEYYY!!
Glad to see ya all, how have ya been?

Gosh, it has been forever since the last time! I feel like I forgot about this platform too much, it feels good to have a place full of wonderful people to speak to, and you don't see those everywhere. I'm sorry I haven't been here much, it was mostly due to me not being able to practice programming much and thus falling back on tech stuff.

BUT - that period is now over. Maybe.
I'm gonna be more active on here, in the past recent years I've seen how bad most social medias turned out to be, with a few very special exceptions. I think devRant deserves more activity, so for better or for worse (hopefully better), I'm back!

I think my biggest problem right now is the need for a better PC, one Italians would call "a PC with the controcazzi", lol. A good one, is what I'm sayin. But would ya look at that, thr moment I start searching up for one, a friggin pandemic takes place and prices skyrocket! Ain't that fun. XD

I would probably have found an awesome PC build by now if I knew jack shit about hardware, but unfortunately I was always more into software than hardware. ^^"
So if anybody has any idea, I mean, I'm open to suggestions~ they'd be very appreciated, and thanks in advance. <3

But enough about that - how are you holding up? I hope you're doing good.
Misadventures and bad stuff happen, but I promise you we're all gonna get out of it soon. In the meantime, always remember to drink water, eat properly, keep yourself sanitized, exercise, and do things you love doing. That's what life is about. I'm looking forward to hearing from y'all, once again. 💙

Keep fighting the good fight, and kick ass! And chew gum, too.

From a dev and eCommerce management perspective, Groupon is THE WORST. They just launched a brand new marketplace site but STILL don’t have an API I can connect with to do fulfillment. Their CSV format is useless for our shipping company. Their data inputs are not sanitized and standardized enough to be predictable for purposes of transformation via scripting. The exceptions and edge cases are infinite. So I’m STILL stuck having to take time out of my day to manually copy and paste and correct order data into the proper format to FTP to the fulfillment company. Oh, and I don’t yet know if the new marketplace does this, but the old one used to suspend vendor accounts for...get this...selling TOO MUCH! How is Groupon still in business?

Here is a gem I found when looking at the previous offshore team's database.

So apparently they didn't know that SQL has an ALTER TABLE command to add new columns. So they created a brand new table, version 2, THEN migrated all the data over, every single time a new field was needed.

Then of course they had to update all their code that previously looked at the original table and the clients had to resync data onto the tablets as well.

Maybe they thought it was a good solution since they don't know what database versioning is (something they also manually implemented) or that ORMs exist.

**Sanitized the table names but kept the general structure, casing, etc

CREATE TABLE [dbo].[TVP_NameHere] AS TABLE(
[NameTime] [datetime] NULL,
[NameId] [int] NULL,
[somethingId] [int] NULL,
[fooId] [int] NULL,
[Time] [int] NULL
)

CREATE TABLE [dbo].[TVP_NameHereV002] AS TABLE(
[NewColumnHere] [int] NULL,
[NameTime] [datetime] NULL,
[NameId] [int] NULL,
[somethingId] [int] NULL,
[fooId] [int] NULL,
[Time] [int] NULL
)