0

Context: data warehouse

A colleague gets notified that his historisation produces duplicates. He checks the tables, sees that something has gone wrong ... and asks me for help.

I'm so sorry but I had to laugh so hard. Turns out everything crashed cause the value to be historised had changed back to an earlier version (which can happen). And now I'm here wondering how someone who calls himself a senior dev can create a historisation without taking care of gaps and islands. Seriously, how can you not think about that?

Comments
  • 2
    So you found out that that dev is actually a human?
  • 1
    @Oktokolo No I found out that someone got promoted to a senior position without knowing basic problems. Everyone makes mistakes, that's normal, but if you get promoted for just being long enough the company, that's a problem.
  • 0
    @CodingTripledad Maybe, he was good enough implementing other stuff. Even the most experienced devs make silly mistakes and botch their architecture sometimes.

    Also, "senior" is mostly a position honoring work experience - in older times you could only get it by having enough years of work experience. Most will still have serious bugs when implementing something they didn't implement before.

    The only problem here might be, that the senior didn't test a case which they should have expected to happen.

    Btw: I don't think, there should be gaps in a history. And values obviously aren't unique. But timestamp-value pairs for a field of an object are. I have no idea what the meaning of "island" would be here.

    I also suspect, that the beast started out as an immutable audit history and feature creep at some point required mutability, but instead of creating a new thing for that, the old thing lost its immutability (and therefore its usefulness for actual auditing).
  • 1
    @Oktokolo I'd be happy to say "yes" to your first point. Trust me, I'd be. Devs are humans, but if you push changes without proper testing, you deserve a rant ;)

    Honestly it's a pretty common DWH problem. For instance Jack Doe today, tomorrow Jack Smith and after divorce Jack Doe again. So you have Doe as islands and Smith as gap. A good historisation will take care of this and write three entries with normal timestamps. A bad historisation will try to put the Does together, then produce strange results or crash. Just like you'd verify user input in every other language, you must verify that in a DWH environment.
Add Comment