Darn it, I was having such a good day. Just sitting over here in sysadmin land watching the Java devs tear their hair out over the Log4j vulnerability, when someone just had to ask me about the Jenkins servers my team maintains.

Jenkins doesn't use Log4j! What a relief!

Jenkins does, however, have third-party plugins, some of which use Log4j. And thus my relief was short-lived and now I'm also tearing out my hair trying to patch this shit.

This is exactly what people warned against with container formats. Bundle up all shit in containers, bypass your distro's package manager, because it's so convenient!

But but but what if a security issue... - Shut up, won't happen, we'll think about later. Oh, here, great idea: let's just give the chaos a fancy name and call it devops! Worked already great with agile in dev, right?

@Fast-Nop to be fair, while containerized apps are more rigid when it comes to library updates, they are encapsulated in a single unit by default. If a single app gets exploited - only that one will suffer.

Containers limit the vulnerable scope.

Now imagine if 7 different apps were running on the same bare-metal server. That's 7x more vulnerabilities. And it only takes 1 of them to get exploited to hack all the 7 apps. And the server. And probably your company network too.

Ofc all this could be prevented if sysadmin was given enough time to setup the box right: default-deny fw, selinux, different users per app, different limits, chroot,... But then again we're getting very close to what containers do automatically. Except we'd be doing manually and waste days instead of minutes to spin it all up and maintain

@netikras I don't think that, in this case, a Java application is actually containerised by default just by itself.

Debian shipped the security fix already last week, but people are still looking for impacted apps, let alone have them fixed, although the attacks are full-on. There's just no way that this is more secure.

@Fast-Nop there are probably security issues in some container usage but this has nothing to do with containers.
If you put 1000 Java apps on a single patched machine you still have 1000 apps that are vulnerable.
Why? Because dependency management is not handled by the OS package manager.

Also our way of development would not be possible without containers and like anything you put in the landscape you need to maintain. We apply layer scanning and security patching is almost fully automated. This too would be almost impossible without the deployment setup containers made possible for us...

@hjk101 The OS does provide the package liblog4j2-java via the package manager, and applications that use this package have been patched automatically already a week ago. That's what you get when you don't bypass your distro's package manager.

@Fast-Nop Java applications almost never use the distro package manager for dependencies. They're typically baked into the jar archive itself to make the application as self-contained as possible.

@EmberQuill Correct, and that's exactly the problem. It's a bug, not a feature.

@Fast-Nop no it's not and again not a container issue. On my os it's not even called liblog4j-java. You probably want a container to deploy this consistently @EmberQuill is right and it is not a bug. Relying on os package manager for your dependencies just does not happen, with good reason.

The os only provides libs necessary for the packages that you can install trough the os package manager. For the rest and correct version for your app or has no solution. This is not new, Perl's cpan is a good example. It's wealth of modules has never been in the package manners

Distros are often slow with fixing minor security issues in libs and have buggy versions for years.

@hjk101 I already wrote that Debian had already fixed that lib one week ago. People who bypass their package manager are those who still are in trouble.

However, your conclusion is that bypassing the distro manager increases security because distros are slow to fix that.

Really? That's comedy gold right there.

@Fast-Nop people who don't use Java are in trouble including company. We use a few external services two of them where too slow to respond. Putting our data at risk.

We don't use package manager yet we use renovate and several other tools. Vulnerable container image layers also get redeployed with the latest version. So for the os layer we do use the package manager to stay up to date. Also about 80% of our in-house services is not customer facing and has no access to the internet (default setting).

Again the OS package manager was not meant for managing development dependencies and it fails there.

@Fast-Nop and it's not comedy gold, wish it was. Even your precious debian liblog4j-java has a DoS vulnerability already fixed in the official lib v2.17.0 until yesterday.

@hjk101 The OS package manager was meant to address that, and it doesn't fail. What failed was application devs who decided to just bypass it: ship today, worry later, who cares anyway.

Oh, and the DoS vulnerability was already the fix for the RCE, and what you still don't get is that the bypassers aren't dealing with a DoS - they are still facing the RCE.

Also, tell me: which application exactly included the fix faster than Debian? Can you back up your claims with actual facts?

@Fast-Nop using an OS package manager for development dependencies would be an absolute fucking nightmare, no thank you.

I can already imagine the bug reports that are all distro-specific because Ubuntu, Debian, Arch, and Fedora all ship different versions of the same dependency with breaking changes introduced between versions. And you might as well forget about cross-platform compatibility because nobody wants to deal with the absolute garbage that Mac and Windows have instead of a sane package manager...

@EmberQuill Strangely enough, that does work nicely via the package manager with other applications that are not in Java. It's Javaland that is so fucked up.

@Fast-Nop and Perl (cpan) and PHP (composer) and Rust (cargo) and Go (go cli) and Python (pip) and the list goes on. What you call a pypass we call a necessity. Lets agree to disagree here. This is going nowhere.

@hjk101 Btw., you failed to present evidence for applications being patched faster than Debian, so let's also agree that your claim was BS.

@Fast-Nop this hits home top hard. Damn..