.venvThis PEP sets out to help make the discovery of a project’s virtual environment easier for tools by providing a default location as well as a way for a project to point to its preferred virtual environment when it differs from the default location.
Imagine you are on your Mac laptop and you double-click your desktop shortcut
to launch Emacs (feel free to substitute “Mac” and “Emacs” with your preferred
OS and editor, respectively). You open the directory for your project in Emacs.
Now, how is Emacs (or any other tool for that matter) supposed to know where
the virtual environment for your project is? There’s no possible detection of
an activated virtual environment via the VIRTUAL_ENV environment variable
as you didn’t launch from a terminal. You potentially could scan all
subdirectories for a pyvenv.cfg file to find the virtual environment,
but that assumes the virtual environment is kept locally with the project and
that there is only one of them and not several to ambiguously choose from. What
are tools like code editors which need access to the virtual environment being
used to provide functionality like auto-complete to do when there is currently
no standardized way to tell anyone where the virtual environment is? Currently,
tools like editors have to hard-code a search algorithm for every tool that
they choose to support. As well, they can document any conventions they
support, but that assumes you or the tool you use to manage your virtual
environments follow those conventions.
And this is not a hypothetical issue. The author of this PEP was the dev manager for Python support in VS Code for 7 years and saw firsthand the user struggles and constant feature requests involving trying to find one’s preferred/default virtual environment for a project.
This issue is also not restricted to code editors. Other tools have a need to access the virtual environment to know what is installed. One example is type checkers which need access to the packages that are installed to appropriately gather type annotations for 3rd-party code in order to type check the user’s code.
The goal of this PEP is to provide a specification for tools which create/manage virtual environments a way to tell other tools where the (default) virtual environment for a project is. In the case of a project which has multiple virtual environments, this PEP is meant to allow for specifying the default or “active” virtual environment so users are not forced to make a choice of virtual environment to use if one does not want to make such a decision (e.g. at first launch of their code editor). Please note this PEP neither condones nor discourages having multiple virtual environments for a single project; it is neutral as to whether having a single virtual environment or multiple ones is good or bad.
This PEP does not define what the “root of a project” means, but the assumption
is it is the directory one would open in their code editor to work on a
project’s code. This could be the directory where the project’s
pyproject.toml lives, or potentially the top directory of a monorepo.
The virtual environment for a project MAY be in a directory named
.venv (i.e. .venv/pyvenv.cfg will exist which can be used to
detect the existence of a virtual environment) in the root of the project.
In all other situations where placing a virtual environment at the project root
in a .venv directory is not possible or desirable, a .venv
file SHOULD be written in the project root instead. The file MUST contain
at least a single line recording the path to the directory of the
virtual environment (i.e. the directory containing pyvenv.cfg). The
file MUST be encoded in UTF-8. If \r\n or \n are contained in the file
then the path is considered from the start of the file until the first newline
in the file with the rest of the file’s contents ignored but reserved for
future use cases, otherwise the path is the entire contents of the file. There
are NO requirements on path formatting (i.e. a POSIX path is not required). The
path MAY be relative to the .venv file. Tools SHOULD verify that the
directory the file points to exists before using it.
Tools looking for a virtual environment SHOULD look for the .venv
directory or file and handle them appropriately. Tools SHOULD NOT prefer one
format over another when searching for a virtual environment (e.g. if a tool
looks up through parent directories for a virtual environment, it shouldn’t
look for a directory first and then a file; the first thing found with the
.venv path name should be chosen). Sharing the same path name for both
the directory and file means there is no precedence issue within the same
directory. If the found .venv is a symlink, it does NOT alter how the
resolved file is treated; symlinks SHOULD be treated as if they were a regular
entry in the file system.
This PEP proposes some changes to the venv module to go along with the
above recommendations:
DEFAULT_NAME: str global that’s set to ".venv".read_redirect_file(project_root: os.PathLike|str) -> pathlib.Path[str]
function for getting the path from a redirect file in
project_root / DEFAULT_NAME. Raises an exception if the location
recorded in the redirect file does not exist.venv.EnvBuilder gains
write_redirect_file(project_root: os.PathLike, env_dir: os.PathLike) -> None
and an equivalent write_redirect_file() function for the module. The
function and method will create a redirect file at
project_root / DEFAULT_NAME that points to env_dir.venv.EnvBuilder.create() and venv.create() gain a keyword-only
project_root: os.PathLike | None parameter that will write out a
.venv file to that directory via
EnvBuilder.write_redirect_file(). If the value for env_dir ends in
DEFAULT_NAME and project_root points to the parent directory of
env_dir then write_redirect_file() will not be called.venv.EnvBuilder.create() and
venv.create() get a default value of DEFAULT_NAME.-m venv CLI will gain a default value for its ENV_DIR argument of
DEFAULT_NAME (it’s currently an error not to provide the argument).-m venv CLI will gain a --project-root option that mirrors the
new parameter to venv.EnvBuilder.create(). It will be an error to use
the option when multiple ENV_DIR arguments are provided.executable(dir: os.PathLike, name: str = DEFAULT_NAME, *, traverse: bool = False) -> pathlib.Path
will be added; it will look for a virtual environment in dir at name
(directory or redirect file; defaults to DEFAULT_NAME) and return the
path to the python executable for the virtual environment, raising
an exception if the path to a virtual environment is not found or the
virtual environment is somehow corrupted. If traverse is true, then
traversal through the parent directories of dir to look for
DEFAULT_NAME as a file or directory will be done and will stop at the
first DEFAULT_NAME found closest to dir.With regard to committing a .venv file to version control, it MAY be
done when the location of the virtual environment is considered static to a
project once it is set up. For instance, some projects that use tox have a
“dev” environment defined in their configuration that ends up at .tox/dev.
Setting a .venv file to point to that virtual environment and checking
in the file is reasonable. The same goes for a project that is only worked on
within a container where the location of the virtual environment is controlled
and thus static on the file system. The guidance of NOT committing your actual
virtual environment to version control is unchanged by this PEP.
There are three aspects to where a virtual environment is placed. The first is whether the virtual environment is local to the project or stored globally with other virtual environments. Keeping the virtual environment local means that it is isolated and unique to the project. As well, it means that if you delete the project you also delete the virtual environment. If you store the virtual environment globally then you can share it among multiple projects and delete all virtual environments at once by deleting the directory that contains them all. Keeping virtual environments global also means it won’t be backed up automatically if a project is stored e.g. in a directory automatically backed up to remote storage where you pay based on how much storage you use.
Another aspect is the directory name used for the virtual environment
(although this really only affects local virtual environments). If one views
virtual environments as more of an implementation detail, a directory name
starting with . seemingly makes sense to mark it hidden or de-emphasized
in various tools such as shells and code editors. But hiding it can make
accessing the directory harder via tools that don’t expose paths starting with
a ..
Lastly, there’s whether you have one virtual environment at a time or many. Having only one can minimize disk space for some tools and keeps it simple by not trying to manage multiple virtual environments. Having multiple virtual environments, though, means not having to constantly recreate virtual environments when e.g. needing to test against multiple Python versions.
This PEP takes a two-pronged approach to making virtual environments easily
discoverable while supporting all aspects mentioned above. First, this PEP
suggests putting the virtual environment in the .venv
directory of the project (this can be a hardlink, symlink, etc.). This name
has been chosen due to preexisting tool support:
.gitignore which ignores .venv.gitignore which ignores .venv.gitignore which ignores .venvBut for various reasons (from personal preference to preexisting tool defaults),
the .venv directory in the project root may not work. In those cases, a
.venv file which points to the virtual environment by default
should be provided in the project’s root directory (i.e. the same location as
specified above for the .venv directory). This file should point to the
virtual environment to use by default; there can be other virtual environments
for the project, but the .venv file should point to the virtual
environment to be used if no preference is specified. While a symlink for
.venv could serve the same purpose, not all file systems support
symlinks. As well, situations like automatic backup of a directory to a cloud
backup solution require a level of indirection so that backup tools don’t
implicitly follow into a virtual environment and back it up.
The .venv file is meant to represent the virtual environment a workflow
tool is expected to use that is external to the one that wrote the
.venv file (e.g. Hatch wrote the file and VS Code is going to read it).
This means that a workflow tool shouldn’t update the .venv file when
running a test suite through multiple versions of Python. But if the workflow
tool has a command to control what virtual environment is used when running
Python, then the file should be updated as necessary to match what environment
the workflow tool would use (e.g. .venv should always match what
virtual environment ‘hatch run’
would use). This is not expected to cause a “noisy neighbour” problem as it’s
not expected to change that rapidly.
The format for the .venv redirect file is for ease of use. Allowing
newlines in the file makes it easy to create or edit the file in a code editor
that automatically adds newlines to the end of a file. Only reading up to the
first newline, if one exists, also allows for adding more data to the file in
the future. It also allows for easy shell scripting to read the file, e.g.
head -n 1 .venv | tr -d '\n' or Get-Content .venv -TotalCount 1.
Having tools check for the existence of the path before using it is to prevent
tools from being tricked into e.g. blindly passing the file contents into
subprocess.run(..., shell=True).
Speaking to various tool maintainers about this PEP:
Note
Any tool without a link to an expression of (no) support gave that information privately, but with permission to state publicly.
For the virtual environment location aspect of this PEP, the backwards
compatibility concern would be over some alternative use of .venv. But
due to the current usage already in the community, the likelihood of an
alternative usage is probably small. This will likely lead to tools showing an
error message when a .venv file is used, though. While the error message
would likely be around .venv being a file and thus not explaining why
there’s a file, it just prevents any tool from overlooking the .venv file and
blindly creating another virtual environment.
The other possible backwards compatibility concern is the new default value for
the -m venv CLI. But since it’s currently an error not to specify the
directory, the impact should be minimal.
Not checking the contents of a potentially malicious .venv file and
passing it to a shell process (e.g. subprocess.run(..., shell=True)) would
be a serious security concern. This is why this PEP says tools MUST make sure
the path is valid before using it.
Setting a .venv file to a path that isn’t a virtual environment is only
a concern if the arguments the user provided to the executable were also a
concern. That would require the user to craft appropriate arguments
on top of using the malicious .venv file.
For new users, they can be told that python -m venv creates a virtual
environment in .venv, and that any other tool that creates a virtual
environment on their behalf can do the same.
For experienced users, they should be taught the default location for a
project’s virtual environment is at the root of the project in .venv.
If the currently active virtual environment lives elsewhere, a .venv
file will be there to tell them where to find the virtual environment.
The proposed code changes to venv can be found at
https://github.com/brettcannon/cpython/tree/venv-location-pep. A diff showing
the changes can be seen at
https://github.com/brettcannon/cpython/compare/main…brettcannon:cpython:venv-location-pep.
.venvSome people either don’t like that .venv is hidden by some tools by
default thanks to the leading ., or don’t like venv as an
abbreviation. Since there doesn’t seem to be a clear consensus on an
alternative, a different name doesn’t fundamentally change any semantics,
existing tools seem to already support .venv, one can still use a different
name for an environment thanks to redirect file support as proposed by this
PEP, and the author of this PEP prefers the name, .venv was chosen.
Discussing alternative names was viewed as bikeshedding.
While the redirect file format is designed for future usage, this PEP could
choose to just use that space now instead of in some future PEP. The extra data
in the file could record other virtual environments that the project has.
Optionally, the path could be separated from a labelled name by a \t. The
default virtual environment would be allowed to be listed in the labelled
section if an explicit label was desired. Another option would to record such
data in a JSON/JSONL trailer in the file.
Thanks to everyone who participated in an earlier discussion on this topic at https://discuss.python.org/t/22922/. Thanks to Cary Hawkins of Hatch, Randy Döring of Poetry, Frost Ming of PDM, Bernát Gábor of virtualenv & tox, Vinay Sajip of venv, and Zanie Blue of uv for feedback on the initial draft of this PEP.
venv.executable() be configurable for the virtual environment
nameThis document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。