Poetry for Dependency Management

Ever struggled with Python dependency conflicts? So have I, until I discovered this tool.

Meet Poetry 🌟

Poetry is a Python dependency management tool.

As I add more packages to my projects, Poetry deftly resolves any dependency conflicts—goodbye, dependency hell (yes, TensorFlow and Numpy, I’m looking at you 👀 )

And the best part? Poetry creates a lockfile that ensures reproducible environments across different operating systems. For instance, while I work on MacOS, I can seamlessly share my project environment with colleagues on Linux.

While I’ve been a fan of conda for its one-stop environment setup, I’m starting to appreciate Poetry’s reproducibility. Now, my approach for new AI projects combines the best of both worlds—conda for the Python version, and Poetry for managing everything else.

Here’s a list of Bash commands I run when setting up a new Python project. Feel free to incorporate them into your Makefile 😁

$ conda create -n myenv python=3.12  # Create a virtual env with Conda or PyEnv

$ pip install poetry # Install Poetry in your virtual env

$ poetry init # Creates a basic pyproject.toml file in the current directory.

$ poetry add langchain openai # Adds dependencies to pyproject.toml file.

$ poetry update # Get and installs latest versions of dependencies, automagically.

You could also incorporate these commands into a Makefile to save time. I like to use Makefiles as they save developer a few seconds, and it accumulates as you code longer. The code below displays my Makefile, and anyone who wants to work on this propioject could just run make setup and make install.

setup:
	@echo "Creating a new Python environment called 'myEnv'..."
	@conda create -n myEnv python=3.12 -y

install:
	@echo "Installing Poetry, a Python package manager..."
	@pip install poetry

	@echo "Installing packages with poetry..."
	@poetry install --no-root

After running this command poetry add langchain openai, the pyproject.toml file looks like this.

[tool.poetry]
name = "projectName"
version = "0.1.0"
description = ""
authors = ""
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
openai = "^1.14.3"
langchain = "^0.1.14"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"