Honest question. When do you consider yourself a "Big data engineer"?

Today I managed to create a system that collects historical metrics from monitoring tools every 5 minutes and do all sorts of crazy transformations to make them ingestible by grafana Mimir in OTLP protocol. Doing 600gb a dat, millions of active time series, .... And I still feel it's, "small"

Thoughts?

My data is bigger than yours.

@electrineer are you a big data engineer?

You're doing transformations on gigabytes of data. That's big.

Hamilton. That's the only case I think that someone working with code is justified in calling themselves an engineer. Doesn't matter if it is data or software or whatever.

And I am getting very suspicious about the claim that someone is an engineer when they have never visited a materials class.

NeatNerdPrime, if your real first name is not Margarete, I don't consider you an engineer.

labels don't make any sense to me

people call themselves anything and it means nothing

I'd be more avoidant of labeling myself because of the amount of people who didn't do stuff yet label themselves and now you're beside them in everyone's head as associated peers of the same ilk

I've seen anecdotally that it's not big until it's terabyte scale. But you're pretty close.