Improve the default excepthook formatting for multi-line and exception notes

[FFY00]

Proposal:

Currently, it is possible to add notes to exceptions, see BaseException.add_note, but this feature is underutilized.

One of the reasons for that, I believe, is the poor formatting in the default excepthook:

Current Output

(single note)

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     raise Exception('error')
... except Exception as e:
...     e.add_note('some context')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('error')
Exception: error
some context

(with multiple notes)

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     raise Exception('error')
... except Exception as e:
...     e.add_note('some context')
...     e.add_note('some context 2')
...     e.add_note('some context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('error')
Exception: error
some context
some context 2
some context 3

I think if we formatted it better, making the output easier to understand, we could start seeing better adoption of this feature.

(with multiple notes)

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     raise Exception('error')
... except Exception as e:
...     e.add_note('some context')
...     e.add_note('some context 2')
...     e.add_note('some context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('error')
Exception: error
├── Note: some context
├── Note: some context 2
└── Note: some context 3

---

In addition to this, multiline exception messages could also be improved.

Current Output

(multi-line string, with no notes)

$./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> raise Exception('some error\n\nsome context\nsome context (cont.)\n\nsome extra context')
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    raise Exception('some error\n\nsome context\nsome context (cont.)\n\nsome extra context')
Exception: some error

some context
some context (cont.)

some extra context

(multi-line string, with multi-line notes)

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
>>> try:
...     raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
... except Exception as e:
...     e.add_note('some context\nsome context (cont.)')
...     e.add_note('some context 2\n\nsome extra context 2')
...     e.add_note('some context 3\nsome context 3 (cont.)\n\nsome extra context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
Exception: some error

some error text
some error text (cont.)

some more error text
some context
some context (cont.)
some context 2

some extra context 2
some context 3
some context 3 (cont.)

some extra context 3

The first example IMO isn't optimal, but it's okay enough, but the second example is very confusing. We could format it something like:

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
>>> try:
...     raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
... except Exception as e:
...     e.add_note('some context\nsome context (cont.)')
...     e.add_note('some context 2\n\nsome extra context 2')
...     e.add_note('some context 3\nsome context 3 (cont.)\n\nsome extra context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
Exception: 
│ │ some error
│ │
│ │ some error text
│ │ some error text (cont.)
│ │
│ └ some more error text
│
├── Note:
│   │ some context
│   └ some context (cont.)
│
├── Note:
│   │ some context 2
│   │ 
│   └ some extra context 2
│
└── Note:
    │ some context 3
    │ some context 3 (cont.)
    │ 
    └ some extra context 3

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[iritkatriel]

Interesting idea. I have a vague recollection that we deliberately kept it free form so that you can format the notes however you want. (Maybe you don't want them to appear as separate notes?). Of course anyone can write their own traceback formatter.

CC @Zac-HD.

[FFY00]

The proposal still allows for some level of custom formatting, it would only clash with instances where this kind of structural formatting is being done on the note message directly, which could also mess up 3rd party formatters anyway.

I think it would be reasonable to define the notes as free-text fields. Structural formatting would be left to the excepthook.

[Zac-HD]

I disagree that the current formatting is poor. Most notes are a single line of context, something like "while doing $operation", and would not benefit; and a few are structured multi-line notes like Hypothesis' failing-example reports.

I further think that changing how exceptions are formatted should have a high bar given the variety of tools built on the current behavior; if you want to pursue this I think the proposal should have a PEP.

[FFY00]

While the current formatting is okay-ish for single-line notes, I don't think it's very suitable for multi-line notes.

I think the current implementation is curated for a specific use-case, which is adding small context clues to exceptions in exceptions groups, but this is a generic API, so I think making improvement to its usability on other scenarios makes sense.
For example, in GH-127598, I would like to start adding "hints" to what may have caused import errors. These hints will likely be multi-line explanations with suggestions on how to fix the issue, and in some cases, we may want to add multiple hints to one exception. If we don't have a way to distinguish between notes, the output becomes very confusing.

I personally don't think this should call for a PEP, considering that we would only be changing the excepthook, which is customizable anyway, so relying on its output formatting is fragile, and PEP 678 specifically calls against using structured data in notes.

However, given PEP 678's specification, I do think it would make sense to ask the SC if we need a PEP for this change or not.

When an exception is displayed by the interpreter’s builtin traceback-rendering code, its notes (if there are any) appear immediately after the exception message, in the order in which they were added, with each note starting on a new line.

@Zac-HD could you elaborate on exactly how tools are relying on the current implementation, and how would changing the formatting as proposed would impact them? This would help clarify the extent of real-world impact, and maybe help find a compromise in the exact formatting, so that we could have a balanced point between all mentioned use-cases.

[iritkatriel]

I would do something like this to distinguish between notes:

>>> e = ValueError(42)
>>> e.add_note('---\nnote 1\nmultiline')
>>> e.add_note('---\nnote 2\nwhat a mess')
>>> 
>>> traceback.print_exception(e)
ValueError: 42
---
note 1
multiline
---
note 2
what a mess

[FFY00]

Yeah, that's the only option right now, but I don't think it's a good long-term solution. Structuring data like this relies on the audithook implementation, and doesn't work well with multiple note sources (eg. user code adding its own notes).

In the proposal, I suggested the usage of a "Note:" prefix to identify notes, but I would be okay with just using the brackets, as a compromise, if that helps with concerns.

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> try:
...     raise Exception('error')
... except Exception as e:
...     e.add_note('some context')
...     e.add_note('some context 2')
...     e.add_note('some context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('error')
Exception: error
├─ some context
├─ some context 2
└─ some context 3

$ ./python
Python 3.14.0a2+ experimental free-threading build (heads/gh-127429-dirty:cc4f57a74d0, Dec  1 2024, 07:06:51) [GCC 14.2.1 20240910] on linux
>>> try:
...     raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
... except Exception as e:
...     e.add_note('some context\nsome context (cont.)')
...     e.add_note('some context 2\n\nsome extra context 2')
...     e.add_note('some context 3\nsome context 3 (cont.)\n\nsome extra context 3')
...     raise
...
Traceback (most recent call last):
  File "<python-input-0>", line 2, in <module>
    raise Exception('some error\n\nsome error text\nsome error text (cont.)\n\nsome more error text')
Exception: 
│ │ some error
│ │
│ │ some error text
│ │ some error text (cont.)
│ │
│ └ some more error text
│
├┤ some context
│└ some context (cont.)
├┤ some context 2
││ 
│└ some extra context 2
└┤ some context 3
 │ some context 3 (cont.)
 │ 
 └ some extra context 3

Though, I have to say, for multi-line notes, I think the space between them makes a big difference in readability.