Exception Handling in Python: Catch Narrowly, Log Everything

Exception handling in Python means using try, except, else and finally to deal with errors deliberately instead of letting them crash your program. Done well, it catches only the failures you can handle and records everything else. Done badly, it catches too much and does nothing with what was caught.

The bad version is easy to write. A bare except: pass also catches KeyboardInterrupt, so Ctrl-C can stop working, and it silently discards the AttributeError from a simple typo. This guide is for Python programmers writing scripts, pipelines or services. It covers each clause, which exceptions to catch, custom exceptions, chaining, exception groups and logging, with example output from Python 3.14.

How does try, except, else and finally work?

for user_input in ["42", "abc"]:
    try:
        value = int(user_input)
    except ValueError as err:
        print(f"Not a number: {err}")
    else:
        print(f"Parsed {value}")        # runs only if no exception
    finally:
        print("Always runs")            # cleanup, always

Output:

Parsed 42
Always runs
Not a number: invalid literal for int() with base 10: 'abc'
Always runs

Four clauses, each with a job:

• try: the code that might fail. Keep it small.
• except: how to handle a specific failure.
• else: the success path. It keeps non-risky code out of try.
• finally: cleanup that must happen either way.

The Errors and Exceptions chapter of the Python tutorial covers each clause, and the try statement reference has the exact rules.

Why should you catch exceptions narrowly?

It is the single most important habit in exception handling in Python:

import logging

logger = logging.getLogger(__name__)

# terrible
try:
    result = compute(data)
except:
    pass

# bad
try:
    result = compute(data)
except Exception:
    result = None

# good
try:
    result = compute(data)
except (ValueError, KeyError) as err:
    logger.warning("compute failed for %s: %s", data.id, err)
    result = None

A bare except: matches every exception. That includes KeyboardInterrupt and SystemExit, which the built-in exceptions documentation deliberately places under BaseException rather than Exception so that ordinary handlers do not catch them. As a result, Ctrl-C can stop working. A bare except also swallows typos: misspell an attribute name inside the try and the resulting AttributeError is silently discarded, leaving you with wrong behaviour and no clue why.

Catch what you can actually handle. Let everything else propagate to something that logs it.

Python 3.14 also accepts except ValueError, KeyError: without parentheses, but only when there is no as clause. The parenthesised form works on every supported version, so it remains the safer choice for shared code.

How much code should go inside a try block?

As little as possible: only the line or lines that can raise the exception you plan to handle. Otherwise you catch errors you never anticipated:

# too broad - which line raised?
try:
    config = load_config(path)
    conn = connect(config.db_url)
    rows = conn.query(sql)
    return transform(rows)
except Exception as err:
    logger.error("something failed: %s", err)

# better - the guard is specific
try:
    config = load_config(path)
except FileNotFoundError:
    logger.error("config missing at %s", path)
    raise

conn = connect(config.db_url)
rows = conn.query(sql)
return transform(rows)

In the first version, a bug in transform produces the same log line as a missing file. In the second, only the thing you anticipated is caught.

Which built-in exceptions will you meet most?

These are the exceptions you will see most often, with the message Python 3.14 prints for each trigger:

Frequently the cleaner answer is to avoid the exception entirely:

# instead of catching KeyError
value = config.get("timeout", 30)

# instead of catching FileNotFoundError for an optional file
from pathlib import Path
if Path(path).exists():
    ...

However, note the race condition in that last pattern. For files, catching FileNotFoundError is often genuinely safer than checking first, because the file can disappear between the check and the open.

When should you create a custom exception?

Define your own when callers need to distinguish your failures from library ones. Start with one base class for the module, then subclass it:

class DataPipelineError(Exception):
    """Base for everything this module raises."""

class SchemaMismatchError(DataPipelineError):
    def __init__(self, expected, actual):
        self.expected = expected
        self.actual = actual
        super().__init__(f"Expected columns {expected}, got {actual}")

class SourceUnavailableError(DataPipelineError):
    pass

The base class matters: callers can catch DataPipelineError to handle anything from your module, or a specific subclass for targeted handling.

try:
    run_pipeline()
except SchemaMismatchError as err:
    alert_data_team(err.expected, err.actual)
except DataPipelineError:
    logger.exception("pipeline failed")
    raise

Attaching structured data to the exception, such as expected and actual above, is far more useful than embedding it only in the message string. Custom exceptions are ordinary classes, so the same rules about inheritance and composition in Python apply.

How do you re-raise an exception properly?

A bare raise inside an except block re-raises the current exception with its traceback intact:

try:
    process(record)
except ValidationError:
    metrics.increment("validation_failures")
    raise                      # traceback preserved

When wrapping in your own exception type, use from to keep the cause:

try:
    conn = psycopg.connect(url)
except psycopg.OperationalError as err:
    raise SourceUnavailableError("Cannot reach warehouse") from err

The traceback then shows both, separated by the line "The above exception was the direct cause of the following exception:". The tutorial's section on exception chaining explains the difference between explicit and implicit chaining.

Use from None deliberately when the original is genuinely noise:

raise ConfigError("Invalid timeout") from None

To add context without changing the exception type, attach a note (Python 3.11+):

def parse_row(n, raw):
    try:
        return int(raw)
    except ValueError as err:
        err.add_note(f"while parsing row {n}")
        raise

parse_row(17, "x")

The traceback ends with the original message followed by the note:

ValueError: invalid literal for int() with base 10: 'x'
while parsing row 17

How do you handle several errors at once?

Raise an ExceptionGroup and handle it with except*. Since Python 3.11, ExceptionGroup bundles unrelated exceptions, and except* handles each type in the group separately. This is useful for validation passes and concurrent tasks, where you want every failure rather than only the first:

errors = []
for raw in ["10", "x", None]:
    try:
        int(raw)
    except (ValueError, TypeError) as err:
        errors.append(err)

try:
    if errors:
        raise ExceptionGroup("batch failed", errors)
except* ValueError as eg:
    print("bad values:", len(eg.exceptions))     # bad values: 1
except* TypeError as eg:
    print("wrong types:", len(eg.exceptions))    # wrong types: 1

Unlike ordinary except clauses, more than one except* clause can run for the same group. Plain except still works for everyday code, so reach for groups only when you genuinely have several failures to report.

When should you use finally for cleanup?

Use finally for cleanup that must run whether or not an exception occurred, including on return, but prefer a context manager when one exists:

def read_rows(path):
    f = open(path, encoding="utf-8")
    try:
        return list(f)
    finally:
        f.close()          # runs even though we returned

Never put return, break or continue inside finally. It silently discards any exception in flight, and from Python 3.14 the compiler emits a SyntaxWarning for it.

In practice a context manager is better:

def read_rows(path):
    with open(path, encoding="utf-8") as f:
        return list(f)

Write your own for resources that need cleanup:

import time
from contextlib import contextmanager

@contextmanager
def timed(label):
    start = time.perf_counter()
    try:
        yield
    finally:
        logger.info("%s took %.2fs", label, time.perf_counter() - start)

with timed("model training"):
    train(model, data)

The timing is logged even if training raises. @contextmanager is itself built on generators, which iterators and generators in Python covers in depth.

How do you log an exception with its traceback?

Call logger.exception inside the except block. According to the logging documentation, it logs at ERROR level and adds the exception information automatically:

try:
    risky()
except Exception:
    logger.exception("risky() failed")     # includes full traceback
    raise

By contrast, logger.error(str(err)) throws away the traceback, which is the part you need when debugging a failure later.

A worked example: collecting failures in a data pipeline

A pattern that comes up constantly is to process everything, collect failures, and report at the end:

import logging

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
logger = logging.getLogger(__name__)

records = [
    {"id": 1, "amount": "250"},
    {"id": 2, "amount": "abc"},
    {"id": 3},
    {"id": 4, "amount": "75.5"},
]

def transform(record):
    return {"id": record["id"], "amount": float(record["amount"])}

results, failures = [], []

for record in records:
    try:
        results.append(transform(record))
    except (ValueError, KeyError) as err:
        failures.append({"id": record.get("id"), "error": repr(err)})

if failures:
    logger.warning("%d of %d records failed", len(failures), len(records))

print(results)
print(failures)

Output:

WARNING 2 of 4 records failed
[{'id': 1, 'amount': 250.0}, {'id': 4, 'amount': 75.5}]
[{'id': 2, 'error': 'ValueError("could not convert string to float: \'abc\'")'}, {'id': 3, 'error': "KeyError('amount')"}]

One bad row does not kill the job, and you get a record of exactly what went wrong rather than a silent gap in the output. Note repr(err) rather than str(err): a KeyError message on its own is just 'amount', which tells a reader nothing about the type.

Common mistakes with exception handling in Python

• Bare except: or except Exception: pass. Both hide bugs. Catch specific types and log what you catch.
• A huge try block. You cannot tell which line failed, and you catch errors you never anticipated.
• Logging str(err) without the traceback. Use logger.exception inside the handler.
• Raising a new exception without from. The original cause becomes harder to trace. Use raise New(...) from err.
• return inside finally. It overrides the try block's return value and swallows exceptions.
• Using exceptions for normal control flow. A missing optional key is not exceptional; dict.get is clearer.

Related reading

The role of programming in data science covers code quality more broadly. User-defined functions covers structuring the code you are protecting. For errors around file input and output, see reading and writing files in Python.