Programming for Data Science: How Much Code Do You Really Need?

Programming for data science needs to be competent, not expert. You must write SQL and Python that runs twice: code someone else can rerun to get your numbers, read without you in the room, and defend in an interview. You do not need software architecture, design patterns or systems programming.

The bar is easy to misjudge in both directions, and both mistakes cost months. Overshoot and you study patterns no analysis role uses. Undershoot and you share a notebook that only works if its cells run in the order you once clicked them.

This guide is for career changers and early-career analysts asking how good at coding they need to be. It defines the bar level by level, shows what reproducible code looks like, and explains where AI assistants help and where they quietly introduce errors such as data leakage.

How much programming for data science do you need?

You need to write code that runs twice. That sounds trivial, yet it is the whole thing. It means someone else can take your project, run it, and get your numbers. That in turn requires paths that are not hardcoded to your desktop, a stated environment, a fixed random seed, cells that execute top to bottom, and no manual step you forgot to write down.

Most analysis code fails this test. Passing it consistently puts you ahead of a surprising share of working data scientists.

You do not need design patterns, deep inheritance hierarchies, microservice architecture, or the ability to optimise at the assembly level. Those are software engineering skills — adjacent and occasionally useful, but not your job.

The table below shows how that bar rises as you progress.

What code does a data scientist actually write?

SQL, constantly. It is programming that people forget to call programming: joins, aggregations, window functions and CTEs. You will write more SQL than Python in most roles, and it is what interviews test first.

Data transformation. In pandas that means loading, filtering, groupby, merging, reshaping and handling missing values. This is the bulk of the work.

Analysis and modelling. Usually you assemble library calls rather than implement algorithms. Knowing which function to use, and why, matters far more than being able to write the algorithm from scratch.

Small utilities. These are functions you write once and reuse, such as a loader, a cleaner or a plotting helper. This is where the step up from beginner to competent happens.

Occasionally, production code. Wrapping a model in an API or writing a scheduled job. Many teams hand this to engineers; even so, being able to do it yourself makes you considerably more useful.

Why do notebooks cause so many problems?

The single most common code-quality failure in data science is not bad algorithms. Instead, it is the notebook that only works if you run the cells in the order you happened to run them three weeks ago.

You know the symptoms: a variable defined in a cell you later deleted, cell 14 run before cell 9, a file path pointing to a folder that no longer exists, and a result you cannot reproduce but are fairly sure was right.

The fix is cheap and habitual.

1. Restart and run everything. Restart the kernel and run all cells, top to bottom, before you share anything. If it fails, it was already broken — you just had not noticed.
2. Move reusable code into modules. Anything you use more than twice belongs in a .py file that you import. Notebooks are for exploring; modules are for things that must keep working.
3. Pin the environment. Create a virtual environment per project with Python's built-in venv module, and record the package versions you used.
4. Keep paths relative and configuration at the top. Then the project runs on any machine, not just yours.
5. Commit to Git. Our Git and GitHub guide covers enough to be useful in an afternoon.

What does reproducible code look like?

Here is the idea in miniature. The function fixes its random seed, and the assertions check both an assumption about the data and the reproducibility itself.

import random


def sample_customers(customer_ids, k, seed=42):
    """Draw the same random sample every time the analysis is rerun."""
    rng = random.Random(seed)
    return sorted(rng.sample(customer_ids, k))


customer_ids = list(range(1000, 1100))
assert len(set(customer_ids)) == len(customer_ids), "customer IDs are not unique"

first_run = sample_customers(customer_ids, 5)
second_run = sample_customers(customer_ids, 5)
assert first_run == second_run, "sample is not reproducible"
print(first_run)

Output:

[1003, 1014, 1035, 1081, 1094]

Create the generator with random.Random() instead, with no seed, and the two runs will almost certainly differ, so the second assertion fails. The same principle applies in libraries: scikit-learn's guide to controlling randomness explains how the random_state parameter decides whether results repeat from one run to the next.

Which habits mark a competent analyst-programmer?

Functions over copy-paste. The third time you paste the same transformation, make it a function. It is the single highest-return habit in analysis code.

Meaningful names. df2 and temp are how you lose an hour next month, whereas orders_by_region costs nothing.

Assertions on your assumptions. assert df['user_id'].is_unique catches a whole category of silent errors that otherwise surface as a wrong number in a board deck.

Handle failure explicitly. Data pipelines break because the source changed. Our exception handling guide covers doing this without swallowing the errors you needed to see.

Comment the why. Not what the line does — the reader can see that. Explain why the filter excludes 2023, or why this column is cast to string.

Where do AI coding assistants fit?

They help most with the tedious parts: boilerplate, transformations, plotting syntax you can never remember, and docstrings. Adoption is now mainstream. In the 2025 Stack Overflow Developer Survey, 84% of respondents were using or planning to use AI tools in their development process.

However, they are dangerous at the statistical parts. They will write a train-test split that leaks, choose accuracy on imbalanced data, or implement a time series validation that peeks at the future — and it will all look confident and idiomatic. The same survey found that the top frustration, cited by 66% of respondents, was AI output that is "almost right, but not quite".

Leakage is the classic example. scikit-learn defines data leakage as using information at training time that would not be available at prediction time, which produces overly optimistic scores. An assistant will not reliably warn you about it, so you have to recognise it yourself.

Therefore, the floor for what you must understand has gone up, not down. You can write more code with less recall, but you need more judgement to review it.

Who can get by with less code?

Not everyone needs this full bar. If you are aiming at reporting or BI analyst roles, excellent SQL, a dashboard tool and light Python will usually carry you. On the other hand, if you want to move towards ML engineering later, aim above the "strong" row in the table, because testing, packaging and deployment become the job.

Related reading

Start with our Python fundamentals series: building blocks, functions and NumPy. For the wider picture, see what data science is and why it matters, and for the rest of the stack read tools and techniques for data science.