I gotta learn to use virtual environments

You sure do

I made a bash alias to create and activate them, just makes the process easier

Yeah. It's not hard on Unix based OSes tho

@ScriptCoded

I'm on Microshit

it's like 3 commands mate!

python -m venv venv

source venv/bin/activate

deactivate

learn it, it's useful as all fuck

Try poetry, it's not perfect but still better then manual handling of v(irtual)env / dependencies

@Hazarth I do this aaaaalll the time. Becomes second nature.

Global package versions are a mistake and conflating executable program management with library management is a mistake, and both are so obvious I find it hard to believe we made these mistakes multiple times.

@lbfalvy Can you elaborate....

My brain broke reading that sentence.

@IntrusionCM Globally addressable executables are monolithic by design, there can be exactly one version installed at a time, they can have state and they must own that state to such an extent that they can safely modify it upon updating to match the current program version. Since they're addressed by program name, it makes no sense to dedicate any significance to version numbers.

Libraries exist purely in the way executables depend on them, several versions may be present concurrently, they must obtain the location of their owned resources through the executables (or at least allow the program to specify it) and they must operate with the assumption that any supported past or future version may appear and disappear spontaneously on the machine. Denying this on the language level is denying LTS releases (because there's no guarantee another executable doesn't depend on the latest release), version dependencies (because the only way to avoid false conflicts that prevent your executable from coexisting with others is to preemptively allow overly general dependency versions), and consequently also supervised dependency update procedures and breaking API changes within a given package.

Time and time again this gets messed up. Npm is designed to manage libraries, so it will happily install multiple versions of React or Webpack in a project if other packages depend on unreconcilable version ranges. The solution is peer dependencies, a "weak link" that influences version selection but cannot own its dedicated instance of the library in question. This works decently well until packages such as Storybook start to wrap Webpack and React. It makes sense for Storybook to own Webpack and React (at least it does according to the authors, I disagree) but it also opens the door to multiple coexisting versions of these packages that actually act a lot like monolithic dependencies rather than libraries - at least with respect to a given project - in that they tend to just crash with a cryptic error when encountering another version of themselves.

@lbfalvy I'm still having problems following your stream of consciousness....

1) By global executable I'd guess you mean e.g. a a binary at a specific location without any form of package management?

A virtualenv is a mixed bag, as there are still prerequisites on the host system, but I guess that's true for any language. I'm not aware of a language that can compile itself without any dependency at all.

2) The first half is already pretty "inconsistent". An executable can be dependent on a library, but a library can exist without an executable.

Even further, there's the question of what type of library - usually static and dynamically exist, where static linking allows to build a "fat' library with no (linked) dependencies.

3) Denying this on the language level... Okay there I lost it.

But "allowing overly general dependency versions" is bad idea.

Same for e.g. allowing multiple versions of a library to exist in an environment. Very very very bad idea.

That's where virtualenv shines in my opinion - it allows to create a nearly fully isolated environment consisting of runtime / python plus packages in a fixed version with no multiple versions allowed.

Which is how it should be.

One must take care of API compatibility, always... This is not just the job of package management, but of the developer, too.

When you want library X which requires library Y in version 1.2… your project must not depend on library Y in any other version than 1.2.

Any attempt to circumvent this or "to work around these limitations"… be it shadowing of namespaces, cross linking versions or other stuff, only leads to a massive pile of problems and an exponential cost of maintenance plus the exponential rise of security problems.

Your comment regarding NPM just came up as I finished typing my answer.

NPM is not package management at all.

It's more the example of "how you should never ever design any form of package management TM".

@IntrusionCM How would allowing multiple versions of, say, ncurses, jsoncpp or fftw be a bad idea? These aren't isolated examples, your typical library works with input, output, possibly a readonly config file.

@lbfalvy depends on how you do it - and where.

https://devmanual.gentoo.org/genera...

One of the things I like about Gentoo is that it has a lot of documentation.

On a <<<system>>> level it will be unavoidable. Proper slotting and thus library naming solves this.

Note that this is not given. Many libraries don't properly version name during the build process, Gentoo / other distributions sometimes go so far and <provide> (as in replace the current) _working_ build system of a project as it's totally bonkers.

On a <<<system >>> level it's already hard to support the slotting - it's a necessity, but a thing noone likes as it requires a lot of maintenance. When upstream doesn't release new versions and security becomes a concern, pulling the plug is always hard as pulling the plug means unwinding the packages depending on it.

TLDR: Necessary evil. Don't do it unless you have good reasons for it.

Id it's compiled statically, totally fine, too - as the generated file has no dynamic library dependencies.

But we are talking about <<<projects>>> here. Not a system level.

On a project level, avoid this - the burden of maintenance and the security implications of supporting multiple APIs is always the wrong choice.

Transitive dependencies are always hard, but they become way simpler if one dependency can only have exactly one version.

... and transitive dependencies will be the majority of dependencies.

@IntrusionCM I still think coexisting versions shouldn't be categorically ruled out on a language level; Gentoo's slots are a great example I didn't know about before, however I think Node's technique of marking the accepted version range in the dependent and automatically combining versions can provide some much needed flexibility at times, especially with packages that don't follow semver due to the parasitic mycelia of marketing that grows deep into everything that costs money.

I think categorically making these decisions on the language level is bad because you can't realistically account for every scenario. Some libraries might for example change their interface with every major and also become massively popular leading to a situation where a combination of versions for a given set of packages might not exist st. all common dependencies can be unified to a single version. Of course you can say that people should just hurry up and support the latest, but why would I have to release a new version of my large and complex library just to accept an update for my config parser that doesn't influence my usage at all but requires me to change the way I interact with it?

@lbfalvy You can express the trouble - and the reason for me being a hardliner regarding the library stuff - by a simple "sort of" binary tree.

Project - actual project
Dep - short for dependency

Project
- dep A - dependencies of A
- dep B - dependencies of B
...

The leaf nodes are missing.

Cause the dependencies of A have themselves possible dependencies.

So the tree grows further with each dependency added - either directly to the Project or one of the transitive dependencies via the Projects dependencies.

What a package manager does is usually to flatten this tree, so that only project as a root node exists and all dependencies are resolved.

Resolved is the key word here.

If a transitive dependency e.g. conflicts with a dependency of the project, the package manager _should_ raise alarms and terminate.

_Should_ as some package managers (e.g. sbt < 1.4, node) don't do this.

Node - as we already said - duplicates the libraries to resolve conflicts.

This means that the tree grows exponentially - which is exactly the reason why node_modules is extremely large and NPM is / was slow.

The binary tree represents something very fundamental - as it lists each dependency, it directly is a representation of any package you have to include in migration planning and that you should be aware of regarding maintenance.

Transitive dependencies are a pain in the ass.

Think of e.g. supply chain attacks - like poisoned NPM packages.

Your project might not depend on a poisoned package directly, but indirectly via transitive dependencies. So the security hole exists - but only if you really look at the full dependency tree and not at the dependencies of your project.

That's the reason I'm against multi version... It seems innocent at first, but in reality it is an exponential cost.

Cause each version of the library bring in different dependencies - at worst these dependencies even conflict each other.

That's why maintaining slots is a pain in the arse, too.