Python issue when installing libraries - error: externally-managed-environment

When attempting to install python libraries on my OSX

pip3 install pandas, numpy, matplotlib

I'm seeing the following responses

error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
   xyz, where xyz is the package you are trying to
   install.
   If you wish to install a Python library that isn't in Homebrew,
   use a virtual environment:
   python3 -m venv path/to/venv
   source path/to/venv/bin/activate
   python3 -m pip install xyz
   If you wish to install a Python application that isn't in Homebrew,
   it may be easiest to use 'pipx install xyz', which will manage a
   virtual environment for you. You can install pipx with
   brew install pipx
   You may restore the old behavior of pip by passing
   the '--break-system-packages' flag to pip, or by adding
   'break-system-packages = true' to your pip.conf file. The latter
   will permanently disable this error.
   If you disable this error, we STRONGLY recommend that you additionally
   pass the '--user' flag to pip, or set 'user = true' in your pip.conf
   file. Failure to do this can result in a broken Homebrew installation.
   Read more about this behavior here: <https://peps.python.org/pep-0668/>
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

The above error your seeing indicates that your Python environment is managed by Homebrew, which restricts the use of pip to install packages directly into the system Python environment. This is to prevent conflicts between Python packages installed via Homebrew and those installed via pip. This is in line with the guidelines of PEP 668, which aims to ensure system integrity when multiple package managers are used.

How you can resolve this and successfully install the packages?

Option 1: Use a Virtual Environment

Using a virtual environment is a best practice for Python development because it allows you to manage dependencies for your projects without affecting global site packages. Here’s how you can do it:

Create a Virtual Environment

python3 -m venv myenv

Activate the Virtual Environment

source myenv/bin/activate

Install Packages

Once the virtual environment is activated, you can safely install packages using pip without affecting the global environment:

pip install pandas numpy matplotlib

Deactivate the Virtual Environment

Once you're done working in the virtual environment, you can deactivate it:

deactivate

Option 2: Use pipx for Application Installation

If you're installing Python applications rather than libraries, and you want them isolated from your project environments, use pipx:

Install pipx

brew install pipx
pipx ensurepath

Install Python Applications Using pipx

For example, if you were installing a tool like Jupyter

pipx install jupyter

Option 3: Configure pip to Allow System-Wide Changes (Not Recommended)

As mentioned in your error message, you can configure pip to ignore the externally-managed environment constraint, but this is generally not recommended because it can lead to a broken system or conflicts with Homebrew's Python. If you still want to proceed with this, you can do it by:

Using the --break-system-packages flag with pip when installing

pip install --break-system-packages pandas numpy matplotlib

Or, permanently in your pip configuration (not recommended):

Add break-system-packages = true to your pip.conf file (usually found in /etc/pip.conf or $HOME/.config/pip/pip.conf).

The wrap up

The safest and most recommended approach is to use a virtual environment for Python library installations. This keeps your system clean and your Python libraries managed per project, preventing any conflicts with the system Python managed by Homebrew.