Let's Talk About the Python Debugger

If I had to choose one tool that massively changed my approach to writing Python code, it would be the debugger. PDB, because that’s what we’re talking about, is Python’s built-in debugger.

So far, I mostly used it within VSCode, whose support for tools like this is simply fantastic. But then I thought it would be nice to get to know it from a more down-to-earth perspective, using the best interface for a computer, which is of course the terminal :P

Personally, I use Neovim, and that’s what I’ll base this article on.

But first — how do we get started?

python -m pdb <file_name>.py

After launching it, we’ll end up in an interactive environment. At this point, it’s worth learning a few basic commands:

That should be enough to get started.

Now we’d need some script to test, right? For example, I have a script used to generate JWT tokens. You can find it on Github

That’s the code I’ll be working with now.

First, let’s prepare everything:

python3 -m venv venv
source venv/bin/activate
pip install pyjwt
wget <gist_address> -o jwt_generator.py

And now let’s launch the debugger:

python -m pdb jwt_generator.py --username andrzej --user-level admin --valid-for 15

At the beginning, you’ll see something like this:

> /home/Andrzej/projekty/blog_examples/python_debugger/jwt_generator.py(1)<module>()
-> import argparse

The program starts by importing the argparse module.

Okay, let’s take a look around this code. First, list:

(Pdb) l
  1  -> import argparse
  2     import jwt
  3     from datetime import datetime, timedelta, timezone
  4     from pathlib import Path
  5     import sys
  6
  7     MAX_MINUTES = 30
  8     DEFAULT_SECRET_FILE = "secret.txt"
  9
 10     def load_secret(path: str) -> str:
 11         try:

Everything checks out — the program stopped at the first line.

By entering the n/next command a few times, we’ll eventually reach the main() function:

And after a few more times, an error will appear: the secret file is missing.

(Pdb) n
> /home/Andrzej/projekty/blog_examples/blog_examples/python_debugger/jwt_generator.py(73)<module>()
-> main()
(Pdb) n
Error reading secret file: [Errno 2] No such file or directory: 'secret.txt'
SystemExit: 1

Notice that we have a SystemExit here, but the program still hasn’t terminated.

After a few more next executions:

> /mnt/c/Users/Andrzej/Documents/projekty/blog_examples/python_debugger/jwt_generator.py(73)<module>()
-> main()
(Pdb) n
--Return--
> /mnt/c/Users/Andrzej/Documents/projekty/blog_examples/python_debugger/jwt_generator.py(73)<module>()->None
-> main()
(Pdb) n
SystemExit: 1
> <string>(1)<module>()->None
(Pdb) n
--Return--
> <string>(1)<module>()->None
(Pdb) n
The program exited via sys.exit(). Exit status: 1
> /mnt/c/Users/Andrzej/Documents/projekty/blog_examples/python_debugger/jwt_generator.py(1)<module>()
-> import argparse

And that’s how we start from the beginning again. To make it easier, I’ll add that using c/continue once would have had the same effect. The program crashes.

We don’t want the program to crash, so let’s add the secret.txt file:

echo "adf8e10h515nbu91b34j51p0j4nb51bn34jn96n01n6nn460186b1b04h614n61pn0" > secret.txt

When we return to the debugger now:

> /mnt/c/Users/Andrzej/Documents/projekty/blog_examples/python_debugger/jwt_generator.py(1)<module>()
-> import argparse
(Pdb) c
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhbmRyemVqIiwibHZsIjoiYWRtaW4iLCJpYXQiOjE3NzYyNjE0NDUsImV4cCI6MTc3NjI2MjM0NX0.YQsWyKS1ZIcEAQU6MEWkJCIHK9IvmcV7lYvbt75Gb6A
The program finished and will be restarted

And now we have a sample generated token.

So now let’s do something more interesting — let’s try to see what happens along the way.

We’ll need a few more commands:

Using the command b 12, we set a breakpoint on line 12, which is where the secret is read from the file.

Using the c command now, the program reaches that exact line.

-> return Path(path).read_text().strip()
(Pdb) list
  7     MAX_MINUTES = 30
  8     DEFAULT_SECRET_FILE = "secret.txt"
  9
 10     def load_secret(path: str) -> str:
 11         try:
 12 B->         return Path(path).read_text().strip()
 13         except Exception as e:
 14             print(f"Error reading secret file: {e}", file=sys.stderr)
 15             sys.exit(1)
 16
 17     def parse_args():

Let’s inspect how the file reading works.

Remember that by entering an expression, you execute it:

(Pdb) Path(path)
PosixPath('secret.txt')
(Pdb) Path(path).read_text()
'adf8e10h515nbu91b34j51p0j4nb51bn34jn96n01n6nn460186b1b04h614n61pn0\n'
(Pdb) Path(path).read_text().strip()
'adf8e10h515nbu91b34j51p0j4nb51bn34jn96n01n6nn460186b1b04h614n61pn0'

The secret value is correct.

But let’s assume I don’t know what PosixPath is, so let’s look inside that object:

(Pdb) Path(path).__dir__
<built-in method __dir__ of PosixPath object at 0x7b6990aabb80>
(Pdb) Path(path).__dir__()
['__module__', '__doc__', '__slots__', '_accessor', '__new__', '_make_child_relpath', 
'__enter__', <...> '__sizeof__', '__dir__', '__class__', '_flavour']

Of course, I shortened the above list of methods for readability.

Okay, now we know that PosixPath has quite a lot of methods.

Since we just reminded ourselves about magic methods, I already know that I’d like to inspect the docstring:

(Pdb) pp Path(path).__doc__
('Path subclass for non-Windows systems.\n'
 '\n'
 '    On a POSIX system, instantiating a Path should return this object.\n'
 '    ')

And once again, everything checks out.

More Advanced Features

Interact

The interact command in PDB allows you to launch an interactive Python interpreter session containing the current set of program variables. It’s useful when experimenting with code on the fly and solving less obvious bugs.

Just remember that the set of currently available variables depends on where you start interactive mode.

For example, launching interact on the breakpoint at line 12, as described earlier, allows us to inspect the path parameter from the load_secret function, but from this place we won’t see the args variable defined at the beginning of main() even though load_secret is called later.

-> import argparse
(Pdb) b12
*** NameError: name 'b12' is not defined
(Pdb) b 12
Breakpoint 1 at /home/andrzej/projekty/blog_examples/python_debugger/jwt_generator.py:12
(Pdb) c
> /home/andrzej/projekty/blog_examples/python_debugger/jwt_generator.py(12)load_secret()
-> return Path(path).read_text().strip()
(Pdb) interact
*interactive*
>>> path
'secret.txt'
>>> args
Traceback (most recent call last):
  File "<console>", line 1, in <module>
NameError: name 'args' is not defined
>>> 

You can exit interactive mode using exit() or Ctrl+D.

Alias

Defining aliases can speed up your workflow, especially when you frequently use the same commands.

It’s worth carefully reading the documentation because unfortunately the behavior of this command is quite unintuitive (at least for me).

Using %1, %2, we refer to parameters passed to the alias.

Final Thoughts

I have to admit that debugging code from the terminal is surprisingly pleasant. No editor plugins, no configuration — it just works.

Wonderful :)

Of course, the fact that you have to constantly switch between the debugger and the editor is annoying, but I still think it’s worth knowing the tools you use, instead of just mindlessly clicking GUI icons (VSCode, I’m looking at you :P).

To wrap up, in one of the upcoming posts I want to show step-by-step integration of PDB with Neovim, which realistically reduces the number of reasons to switch to VSCode :)

Have fun!

Useful links

• Official documentation of PDB