Soo.. over the span of 1 month, x2 MAC servers have died (hardware failure). I'm in progress of setting up the third one.

What a dumb fucking system MacOS is!!! wtf...

I've spent 4+ hours trying to figure out why the fuck `git clone` is getting stuck and timing out our CI runs.

Turns out, on MacOS you cannot run git clone via ssh IF you have that user logged in via GUI (e.g. VNC). Git is getting stuck at

11:51:26.084245 run-command.c:759 trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git-credential-osxkeychain store

and just chilling there doing nothing, waiting for the user to click on a Cancel button in GUI.
Logged out my VNC session and git magically started working.

What a dumb system... Or perhaps I'm misunderstanding this "security" feature...?