- a reasonable package manager with lockfiles and transitive dependency versions
- package org namespaces
- steps towards hermetic builds with less linking against system libraries and dependencies
- cleanup and slimming down of the standard library. Removal of cruft. Less "batteries included" so people lean more on packages and enforcing good practices with packages.
"make depending on system python an antipattern" is a social problem, not a technical one. uv's virtualenvs are not special (aside from not bootstrapping pip; bootstrapping it is already optional, although it's the default); the standard library `venv` writes `pyvenv.cfg` files that tell Python whether or not to use the system `site-packages`.
Pip has attempted to resolve and install transitive dependencies since even before 2020. That's just when they introduced a new, backtracking resolver (which can be slow because it is thorough and simple-minded about its backtracking).
"hermetic builds" sounds to me like it's referring to packages that don't depend on system binaries being present, by the magic trick of separately providing them (even if redundant). That's... just a wheel.
But also: currently, Python 4 is out of the question, and even if it were in play, "we should take this opportunity to integrate package management into the language" would be completely out of the question. That's just not how Python's politics work internally. There is generally pretty broad agreement anyway that the reasons packaging problems are hard to solve in Python (in the places where they actually are) are: 1) people don't agree on the design for a solution; 2) all that code in other programming languages that people want to interface with from Python (which, to be clear, is not something they're trying to get rid of); 3) backwards compatibility concerns. Making a clean break is not going to help with the first two and is directly opposed to the third.
> An organization would parent all of its packages under its namespace.
If you're talking about what you write in the code to import it, nothing prevents anyone from doing this. They just don't, because PyPI is a free-for-all by design.
> A project should blow up if you attempt to use system python. It should not work.
Why?
> `python main.py` should handle all of this for you.
How could it, given the issues you've identified?
> You keep seeing this come up because it's still a systemic issue with the ecosystem. Python needs to enforce this top down.
A very large fraction of Python users (whom you'll likely never hear from because they have their own ways of sharing code amongst themselves) actively don't want that kind of enforcement.
uv is still pretty young, a year and a half since initial public release? It's gotten a lot of traction but yes, it hasn't eaten the "market" for python package management, quite yet. I'd bet that there are users and developers of those ML projects that use uv to manage their environment, but aren't checking in a lockfile (depending on what kind of project it is, it might not make sense to check in the lockfile, I guess).
A project should blow up if you attempt to use system python. It should not work.
Disagree here - an environment is an environment. Yes, the system python environment is kind of shit, but it does provide one - if a package cares about where exactly it gets dependencies, other than "here's a directory with packages", it's doing something wrong. I'm not sure how you'd enforce that, as nice as it sounds from the point of view of getting system python to go away.
`python main.py` should handle all of this for you.
Just to be clear - I agree with you. My impression of the python packaging ecosystem has been that it's kind of shit, various tries have been made at fixing it with various tools, all of them have had issues with one workflow or another. But things are now trending in a much more positive direction.
The tragedy of Python 3 is that they made the community go through a billion dollar migration but didn't tackle any of the hard stuff. And the reason they didn't was the Perl 6 debacle. So it's all Larry Wall's fault.
Python 3 took the exact same path as Larry Wall's Perl 6. But unlike Perl, it was able to come out largely unscathed.
Python could do with more major version upgrades, but small scoped upgrades that only change tiny, manageable pieces. Forcing a good package manager on all Python developers would be a good candidate for this.
> They do not enforce it. It's not about "can do". It's about defaults and enforcing stricter standards.
How exactly do you want to "enforce" an "optional" runtime type check for an interface, that is different from opting in by calling `isinstance`?
For that matter, `TypeError` exists and is raised non-optionally by all sorts of things.
> And in no world are Python builds and dependencies solved. It's a major headache.
For your specific set of dependencies, perhaps. Lots of people are using Python just fine. A large majority of what I'm interested in using would install fully (including transitive dependencies) from pre-built wheels on my system; and of those wheels, a large majority contain only Python code that doesn't actually require a build step. (Numpy is the odd one out here.)
In fact, of the top 10 largest packages in my system's `dist-packages` (all stuff that came provided with my Linux distro; I respect PEP 668 and don't let Python native packaging tools touch that environment), at least 9 have wheels on PyPI, and 4 of them have `none-any` wheels. (And of course, tons of the smaller ones are pure Python - there's simply no room for a compiled binary there.)
Look, is correct to understand that the BEST way, the BEST refactoring of all, is to change to a language + libraries that fix core issues (similar how you can't outrun a bad diet you can't outrun bad semantics)
BUT, also change it means lose a lot of nice things.
For example I move from python to F# and then Rust, for what I say that is now the best overall decision at all, BUT I miss dearly Django (and instant run and REPL)
IF I could get Django (that means transitively use python) and get better type system and perf from it, I could consider it and surely use it, like "python get as swift or go but with rust algebraic types, whatever it means to get here and yet is Django and python. ok?
Is unreasonable? but sincerely, it will nice if the best ideas get retrofitted in other languages because, well, I wish everyone get as more nice things as possible instead of keep with suboptimal options (that is what I say: breaking languages is not something that must be forbidden forever)
Is unreasonable? (I say again) could be, but I think is a good thing to explore. Maybe is not that much, and if at the end things get better, great!
Personally, I do; my only interactions with Python are unwilling (work or AI-related code). Given that I don't have much of a choice in the matter, I'd like to see Python improve as a language and as an ecosystem. I hope that one day I will not feel dread upon encountering it in the wild.
Setuptools comes to around 100kloc now (per a naive search and `wc` of the .py files in a new environment) and installs without a hitch for everyone. Pip, nearly double that. Yes, both of those are heavily using vendoring, but because of bootstrapping problems because they are the default build tool and package installer respectively. (And quite a bit of devendoring would be possible if some system maintainers didn't insist on "building from source" when there is only Python code. Yes, technically pre-compiling .py to .pyc can be considered "building", but pip would already do that during installation.) If they didn't have to worry about those issues (or the additional Pip issue of "how do you get the packages for Pip's dependencies if you don't already have Pip?"), they would install and run just fine from a series of wheels that are all pure Python code.
For that matter, one of those dependencies, `pyparsing`, is around 10k lines installed (and the repository is much larger with tests etc.). That's with it not having any dependencies of its own.
There is no real barrier anywhere around the 1k loc mark. The problems large projects run into have more subtle causes.
- runtime optionally type checked interfaces
- make depending on system python an antipattern
- a reasonable package manager with lockfiles and transitive dependency versions
- package org namespaces
- steps towards hermetic builds with less linking against system libraries and dependencies
- cleanup and slimming down of the standard library. Removal of cruft. Less "batteries included" so people lean more on packages and enforcing good practices with packages.