!!rant

When I worked at a previous job, they only gave out decent titles (and salaries) to upper management. Everyone else... well... I was the Domain/Sysadmin, responsible for the domain and both DCs, upgrading the physical network (plus recabling it: the MDF was a *disaster*), as well as all backups, migrations, printers, servers, and workstations/lappys in the building, plus pushing software, antivirus, updates, security policies, etc. I had complete access to everything, and ofc was responsible for everything. Nothing on my network caused anyone (else) any trouble except one particular printer I wasn't able to replace. Also, nothing new appeared on my network without me noticing and tracking it down.

But my official title? "IT Assistant".
I made $11/hr.

Worth it? Take a flying leap into an overflowing outhouse during the height of a Vegas summer if you even begin to think so.

I eventually managed to switch to a developer position, and (after several attempts) got a ~$5/hr raise. The girl they replaced me with in IT with some ditz who had never installed an OS before, didn't know what the BIOS was, and couldn't figure out why a monitor... plugged into itself... wasn't working. Things went downhill from there.

What an absolute fucking disaster of a day. Strap in, folks; it's time for a bumpy ride!

I got a whole hour of work done today. The first hour of my morning because I went to work a bit early. Then people started complaining about Jenkins jobs failing on that one Jenkins server our team has been wanting to decom for two years but management won't let us force people to move to new servers. It's a single server with over four thousand projects, some of which run massive data processing jobs that last DAYS. The server was originally set up by people who have since quit, of course, and left it behind for my team to adopt with zero documentation.

Anyway, the 500GB disk is 100% full. The memory (all 64GB of it) is fully consumed by stuck jobs. We can't track down large old files to delete because du chokes on the workspace folder with thousands of subfolders with no Ram to spare. We decide to basically take a hacksaw to it, deleting the workspace for every job not currently in progress. This of course fucked up some really poorly-designed pipelines that relied on workspaces persisting between jobs, so we had to deal with complaints about that as well.

So we get the Jenkins server up and running again just in time for AWS to have a major incident affecting EC2 instance provisioning in our primary region. People keep bugging me to fix it, I keep telling them that it's Amazon's problem to solve, they wait a few minutes and ask me to fix it again. Emails flying back and forth until that was done.

Lunch time already. But the fun isn't over yet!

I get back to my desk to find out that new hires or people who got new Mac laptops recently can't even install our toolchain, because management has started handing out M1 Macs without telling us and all our tools are compiled solely for x86_64. That took some troubleshooting to even figure out what the problem was because the only error people got from homebrew was that the formula was empty when it clearly wasn't.

After figuring out that problem (but not fully solving it yet), one team starts complaining to us about a Github problem because we manage the github org. Except it's not a github problem and I already knew this because they are a Problem Team that uses some technical authoring software with Git integration but they only have even the barest understanding of what Git actually does. Turns out it's a Git problem. An update for Git was pushed out recently that patches a big bad vulnerability and the way it was patched causes problems because they're using Git wrong (multiple users accessing the same local repo on a samba share). It's a huge vulnerability so my entire conversation with them went sort of like:

"Please don't."
"We have to."
"Fine, here's a workaround, this will allow arbitrary code execution by anyone with physical or virtual access to this computer that you have sitting in an unlocked office somewhere."
"How do I run a Git command I don't use Git."

So that dealt with, I start taking a look at our toolchain, trying to figure out if I can easily just cross-compile it to arm64 for the M1 macbooks or if it will be a more involved fix. And I find all kinds of horrendous shit left behind by the people who wrote the tools that, naturally, they left for us to adopt when they quit over a year ago. I'm talking entire functions in a tool used by hundreds of people that were put in as a joke, poorly documented functions I am still trying to puzzle out, and exactly zero comments in the code and abbreviated function names like "gars", "snh", and "jgajawwawstai".

While I'm looking into that, the person from our team who is responsible for incident communication finally gets the AWS EC2 provisioning issue reported to IT Operations, who sent out an alert to affected users that should have gone out hours earlier.

Meanwhile, according to the health dashboard in AWS, the issue had already been resolved three hours before the communication went out and the ticket remains open at this moment, as far as I know.

Is this learning job cpu intensive or memory intensive?

I don't know and I don't give a flying fuck, because it's 6:20pm and I have not found any of my favorite servers free to rerun this shit the whole fucking week, so this server (which I have actually killed before, btw) can suck a dick and do its fucking job.

🎤🖐️

I teach military students how to operate a UAS control station and these things are as complicated as computers get. ATCA's and the most crazy stupid network setups I've ever seen. I have one month to teach a guy that's never even had a laptop... Basically giant server room that can fall out the sky from the push of a button.

proxying youtube

today I thought writing a quick project, a youtube proxy server, as in, you browse localhost:<PORT> and youtube comes in the response.

this is not rocket science as proxy servers have around for a long time.

I thought it'd be interesting to code it in userland, as opposed to "systemland".

And 50 lines of code and some minor hurdles later I see youtube "running" in localhost.

Although youtube didn't just work as usual since the videos don't actually come from youtube.com, but from googlevideo.com instead. And my browser, expectably, enforces CORS and forbids any requests to it.

At that point I started to think of ways to somehow proxy googlevideo.com too. But the solutions are not at all trivial.

Then I thought what was the payoff of all this. I tried to proxy serve youtube out of curiosity, and sure thing, you can do it.

But what problem would proxying youtube solve? Maybe I should think in a fuller way what are the problems I have with youtube.

One issue I have is the exposure, discoverability. To explain it, let's say I have been watching a very, very big amount of videos as of today.

Personally I would expect youtube to understand very well by now what my tastes are, what do I want to watch and what I do NOT want to watch.

Notice that I am very black and white, and I do not have much interest in watching certain types of videos.

It could be true that if my expectations of how youtube should work became reality then youtube recommendations would become polarizing or echo chambering.

But that is my decision though, and the problem with youtube is that it's seemingly forcing a single recommendations algorithm onto everyone.

Some people are more open minded and want to watch EVERYTHING, and a lot of people don't.

But users aren't deciding what they should get recommended. Youtube is making that decision for them. And it sure feels like it's trying to maximize ad revenue.

I for one don't give two flying fucks about pranks or diva youtubers. Yet youtube is adamant in presenting some of these to me.

Now, trying to come up with a solution for this is really non trivial. It would definitely require some youtube mining, or some kind of network so as to not get rate limited when mining, and even then you still have to think of how a good recommendation system would work.

I think the implementation of all that would be too much for me (time and skill wise). But I think it's fun to at least try to outline how recommendations could work.

I would very much prefer that when youtube recommended something, at least it has some number of confidence meaning how much would I like that video, so at least I know what to expect.

It should also have some indicators like what is the mood of the video. As in, sometimes I watch youtube in the mood of learning, like programming videos, but most of the time I watch to get entertained.

These ideas are just brainstorms and could be terrible on reproduction, but I'd like to hear what ideas can some of the people here can come up with.