When working with Python, a programmer often encounters situations where she needs to install packages not contained in the Standard Library. In such situations, she must install modules from online repositories using packager installers.
The goal of this article is to help beginners develop a working knowledge of pip (acronym for “PIP Installs Packages”) as quickly as possible while defining all the prerequisite jargon along the way. In particular, this article aims to make the content of the pip documentation as accessible as possible for beginners by using easier words and emphasizing practical examples.
This lesson is part of our Python Dependency Management – PIP Commands course on the Finxter Academy.
What is pip?
PIP (“Pip Installs Packages”) is the official package managing software for Python which installs packages from PyPI (“Python Package Index”). PyPI contains over 300,000 packages as of November 2021 which is much larger than similar package repositories for Python. PIP allows users to install and uninstall packages, manage dependencies, keep archives of wheel files, amongst many other things.
The purpose of this article is to develop a “working knowledge” of PIP that may become useful while working on Python projects at a basic to intermediate level. In particular, we will talk about the most useful parts of the PIP documentation and provide explanations so to make the system more accessible to the beginner. The article will assume that the user is working on MacOS, but the commands for Windows can be obtained through minor modifications.
Note on Pip vs. Conda
A popular alternative to PIP is Conda, which is a package managing software aimed for data analysis. We will highlight three key differences to give you a sense of which you may prefer to use. For a more extensive discussion, see the official Anaconda blog page or StackOverflow.
1) Virtual Environments. Virtual environments are isolated Python environments used for Python projects. Because many Python projects depend on having specific versions of packages installed in the environment, projects may be broken when globally installed packages are updated. To prevent this, virtual environments are created so that projects can be run in the same environment, every time they need to be implemented.
PIP has several virtual environment builders such as virtualenv and venv. (See Chris’s article for a more detailed discussion.) In contrast, Conda has a built-in virtual environment manager. (This can be managed through a GUI if you install Anaconda Navigator.) In this respect, Conda may be easier to use for beginning coders.
2) Availability of Packages. As noted before, PyPI boasts over 300,000 packages in contrast to around 7000 packages in the Anaconda repository. Although PyPI packages can be installed through Conda, they often lead to complications, and mixing the two should generally be avoided. (For more details, see the official Anaconda blog page). Many popular Python packages (numpy, matplotlib, and pandas to name a few) are available through Conda, but when working on Python projects, it is not uncommon for developers to come across packages that are only available through PyPI.
3) Languages. While PIP only deals with Python packages, Conda can install packages written in other languages such as R or C. This is because Conda is aimed toward data science tasks.
Part I: How to install packages using pip?
In this section, we will look at how to install packages and manage dependencies using pip.
pip install
To install packages on pip from PyPI, open up your terminal and use the command:
pip install matplotlib
🛑 Note: pip is replaced with python -m pip in the PIP documentations. -m flag searches the sys.path for the pip module and executes it as an executable file. Some systems require that you use python -m pip. For this article, we will just use pip.
The install command installs all of the package’s dependencies, which is to say it installs all the necessary packages for the desired package to install properly. For instance, matplotlib requires numpy, packaging, pyparsing, cycler, amongst many others whereas NumPy has none. Dependency resolution is a major topic in using pip.
There are various other sources from which you can install packages.
Requirement Files. Requirement files are .txt files that allow users to install packages in bulk, possibly with specifications such as package versions. (See the “Example” in the PyPI documentation to get a sense of what the contents of the file should look like.) Many of the pip commands have options that make outputs suitable for requirement files.
You can use the pip install command to install from requirement files. To do this, navigate to the appropriate directory on the terminal (using the terminal command cd). Then use the following PIP command:
pip install -r requirements.txt
Instead of navigating to the directory on terminal, you could use the absolute path of the file:
pip3 install -r /Users/username/Desktop/requirements.txt
VCS Projects. Many Python packages are available through VCS repositories (such as GitHub) as well. The following example is if you wanted to install Django from their GitHub repository:
pip install git+https://github.com/django/django.git#egg=django
Wheel and Tarball File. The pip install command can also be used to install from local wheel (.whl) and tarball (.tar.gz) files. (Read this Medium article and StackOverflow post on their differences.)
The syntax is similar to before. Navigate to the directory where the files are located using the change directory (cd) command on terminal. For example, to install the tea package from a whl file, use:
pip install tea-0.1.6-py3-none-any.whl
To install the tea package using tarball, use:
pip install tea-0.1.6.tar
pip uninstall
The uninstall command is fairly self-explanatory. It allows users to uninstall packages. For instance, if you were to uninstall the tea package using pip, then use:
pip uninstall -y tea
You can (optionally) add -y as above to prevent the program from asking for confirmation.
To uninstall multiple packages at once, you can list the packages in a requirements.txt file (much like we did for pip install), and use the following command:
pip uninstall -r requirements.txt
pip check
The check command allows users to check for any broken dependencies, i.e. if there are any packages that depend on other packages that are not installed in the environment. The syntax is as follows:
pip check
pip show
The show command lists all the relevant information for a particular package. For instance, if you want to know where Django is installed on your device or if you want to know its package dependencies, you can use:
pip show django
For which you can get the output:
Name: Django
Version: 3.0
Summary: A high-level Python Web framework that encourages rapid development and clean, pragmatic design.
Home-page: https://www.djangoproject.com/
Author: Django Software Foundation
Author-email: foundation@djangoproject.com
License: BSD
Location: /Users/user_name/Library/Python/3.8/lib/python/site-packages
Requires: pytz, sqlparse
Required-by:
pip list
To list all the packages available in your environment, use the pip list command:
pip list
For which you may get the output:
Package Version
-------------- -------
pip 19.2.3
setuptools 41.2.0
setuptools-scm 6.3.2
six 1.15.0
sqlparse 0.4.2
tea 0.1.6
tomli 1.2.2
tzlocal 3.0
wheel 0.33.1
What if the user wanted to uninstall all the packages except the bare essentials? You can obtain a list of packages that are not dependencies of installed packages using:
pip3 list --format freeze --not-required
The option “--format freeze” puts the list in a format compatible with a requirements.txt file:
pip==19.2.3
setuptools-scm==6.3.2
six==1.15.0
sqlparse==0.4.2
tea==0.1.6
wheel==0.33.1
Now the user can copy the above into a requirements.txt file, delete the names of files that the user wants to keep, and use
pip uninstall -r requirements.txt
to uninstall all the rest.
pip freeze
The freeze command outputs a list of packages installed in the environment in a package suitable for requirement files. The syntax is as follows:
pip freeze
The freeze command is useful for copying all the packages from environment A to environment B. First run freeze in environment A, and store the contents in a requirements.txt file:
pip freeze > requirements.txt
The file gets stored in the current directory (which you can check using pwd command on terminal). Then go to environment B. (If A and B are virtual environments, deactivate A and activate B on terminal using commands from whichever virtual environment manager is being used.) Then install the packages in the requirements file using install:
pip install -r requirements.txt
Part II: Distribution Files
In this section, we will discuss how to download and manage distribution files for Python packages.
Distribution files are compressed files containing various files necessary to implement the Python library. See this medium article on an extensive discussion on the different types of distribution files. We just need to know the following in order to understand the rest of this section:
Wheels. (.whl) Wheel files are essentially zip files containing everything necessary to install packages in your local environment. They are generally faster to download and install compared to tarballs. For more details, see this article from RealPython.org and this article from PythonWheels.com.
A “built” distribution file is in a format that is ready to install, thereby making the whole installation process faster.
Tarballs. (.tar.gz) Tarballs are types of source distributions that contain both python codes and codes for any extension modules for the package.
Wheel files a