March 29, 2026

Becoming AI QA: Jupyter Notebook + Python

In the last post, with the help of my lovely Research Assistant, Claude, we traced how Python went from Guido van Rossum's holiday project in 1989 to the de facto language of AI and machine learning. 

Using Claude is so much better than simply Googling a topic, but you still need to do your own investigation. Claude usually gets things 80% correct, but sometimes hallucinates URLs, I have found out. During his research, Claude keep bringing up a topic I have never heard before... Jupyter notebooks... What, is that a typo? 

What Is a Jupyter Notebook?

According to the Project Jupyter official documentation, a Jupyter Notebook is a web-based interactive computing platform. The notebook combines live code, equations, narrative text, visualizations, and interactive dashboards into a single shareable document.

The file format is .ipynb -- short for "IPython Notebook," a holdover from the tool's origins.

A notebook is organized into cells. Each cell is either:


The Jupyter Ecosystem: Notebook, Lab, and Hub

Three main products have come out of the project:

Jupyter Notebook is the original web application -- a streamlined, document-centric interface. The current version is Notebook 7, built on top of JupyterLab, per the official Jupyter Notebook CHANGELOG on GitHub.

JupyterLab is the next-generation UI, enabling panels, terminals, notebooks, and text editors to be arranged side by side in the same browser window. Official docs: https://jupyterlab.readthedocs.io/

JupyterHub is for multi-user deployments -- classrooms, corporate data science teams, research groups. Official docs: https://jupyterhub.readthedocs.io/

Jupyter AI was announced via the official Jupyter Blog post "Generative AI in Jupyter" (August 2023). The extension brings generative AI into notebooks, enabling code generation, error correction, content summarization, and full notebook generation from natural language prompts.


How To Get Started With Jupyter

To try Jupyter, the fastest way would be to just go to https://docs.jupyter.org/en/stable/start/ to try it free in your browser. "When running the examples on the Try Jupyter site, you will get a temporary Jupyter server running on mybinder.org which you can use to play around until you close your browser session".

Google Colaboratory (Colab) claims to be the fastest way to start using Jupyter. According to the Google Colaboratory official FAQ, Colab is a hosted Jupyter Notebook service that requires no setup to use and provides free access to computing resources including GPUs and TPUs. Notebooks are stored in Google Drive and shareable like a Google Doc.

Anaconda is the standard local install path. It bundles Jupyter, NumPy, pandas, and most of the scientific Python ecosystem in a single installer.

According to the Anaconda "Our History" page, Anaconda was founded in 2012 by Peter Wang and Travis Oliphant, with a stated conviction: "open-source tools had the potential to transform how organizations work with data."

The company launched under the name Continuum Analytics, based in Austin, Texas. According to a Weights & Biases podcast interview with Peter Wang, Wang described the original mission as bringing open-source scientific Python -- particularly NumPy -- to the world of business computing. As Wang put it: "We put the company together to promote the use of Python and to advance the state of the art for Python for a broader set of data analysis needs."

Travis Oliphant is the primary creator of NumPy and a founding contributor to SciPy. According to the HandWiki biography of Travis Oliphant, from 2012 to 2017 he served as CEO of Continuum Analytics, during which the company received a $6M DARPA contract to develop GPU-accelerated extensions to Python for high-performance computing, and raised $24 million in a Series A funding round in 2015.

The rebrand from Continuum Analytics to Anaconda was made official on June 28, 2017, per the Anaconda blog post "Continuum Analytics Officially Becomes Anaconda".


Video Introductions


The Origin of Jupyter: IPython (2001)

According to the Berkeley News article "Pioneering data science tool -- Jupyter -- receives top software prize" (May 2018), Fernando Pérez, a Colombian physicist and software developer spent an afternoon writing the original version of IPython, the pre-cursor to Jupyter. while he was a graduate student.

According to the UC Berkeley CDSS article "As Project Jupyter Celebrates 20 Years, Fernando Pérez Reflects On How It Started" (2021), the motivation was partly personal: he had been fired by his first PhD advisor and was struggling to complete his dissertation. His new supervisor allowed him to "productively procrastinate" by building IPython -- a tool he could justify as an aid to finishing the dissertation itself. As Pérez put it in that interview:

"I regained some much needed confidence, I got attracted to building something, and it turned out to be really important."

According to the EarthCube article "Project Jupyter", in what Pérez calls a "thesis procrastination project," he wrote the first IPython over a six-week stretch: a 259-line script, with the note "Interactive execution with automatic history, tries to mimic Mathematica's prompt system."

Brian Granger and the First Interactive Notebook (2001 - 2011)

Brian Granger is an associate professor of physics and data science at Cal Poly State University in San Luis Obispo, and a co-founder of Project Jupyter. According to the O'Reilly JupyterCon speaker page for Brian Granger, he is a leader of the IPython project and an active contributor to open source projects focused on data science in Python.

According to the EarthCube article "Project Jupyter", Pérez and Brian Granger first met as students in 1996. In 2004, Pérez visited Granger in Santa Clara, where they discussed open source, interactive computing, and the idea to build a web-based notebook -- an extension of parallel computing work Granger was doing in Python and Pérez's work on IPython.

Years later, according to that same EarthCube article "Project Jupyter", in 2009 Pérez was visiting Granger and his family at their home in San Luis Obispo, where Granger was then a professor. It was spring break, and the two spent March 21 - 24 completing the first prototype IPython kernel with tab completion, asynchronous output, and support for multiple clients.

According to Fernando Pérez's blog post "The IPython Notebook: A Historical Retrospective" (January 2012), on December 21, 2011, the team released IPython 0.12, the first version to include the browser-based interactive notebook. Pérez described it as:

"an environment that retains all the features of the familiar console-based IPython but provides a cell-based execution workflow and can contain not only code but any element a modern browser can display."


IPython Becomes Project Jupyter (2014 - 2015)

By 2014, the IPython project had grown to include a notebook server, a Qt console, and support for languages beyond Python. The name "IPython" no longer fit.

According to the Berkeley Lab Computing Sciences article "ACM's Software System Award Honors Project Jupyter Team" (May 2018), IPython evolved over the years to meet the needs of various communities and in 2014 the project rebranded itself as "Jupyter" to recognize the fact that it was no longer just for Python.

According to the official IPython "What's New in IPython 4.0" release notes: "IPython 4.0 is the first major release after the Big Split. IPython no longer contains the notebook, qtconsole, etc. which have moved to Jupyter." That release came out in August 2015.

According to the official Jupyter Notebook GitHub repository README: "In 2015, Jupyter notebook was released as a part of The Big Split™ of the IPython codebase. IPython 3 was the last major monolithic release containing both language-agnostic code, such as the IPython notebook, and language specific code, such as the IPython kernel for Python."

Project Jupyter's name is a reference to the three core programming languages supported by Jupyter: Julia, Python, and R. According to the Project Jupyter "About" page, the name and logo are also an homage to Galileo's discovery of the moons of Jupiter -- documented in notebooks attributed to Galileo.


Recognition

According to the Berkeley Lab Computing Sciences article "ACM's Software System Award Honors Project Jupyter Team" (May 2018), the Project Jupyter team received the 2017 ACM Software System Award -- presented at the ACM Awards banquet in San Francisco on June 23, 2018 -- with a prize of $35,000.

In 2015, Pérez and Granger received $6 million from the Leona M. and Harry B. Helmsley Charitable Trust, Alfred P. Sloan Foundation, and Gordon and Betty Moore Foundation to expand the capabilities of the Jupyter Notebook, according to the Berkeley Lab Computing Sciences article "Enhancing Open-Source Software for Scientific Computing and Data Science" (2015).

According to the Nature magazine article "Ten computer codes that transformed science" (2021), Jupyter was named one of ten computing projects that transformed science. The notebook count on GitHub grew from approximately 200,000 in 2015 to 2.5 million in 2018 to nearly 10 million by January 2021.


Do ML and AI Professionals Still Use Jupyter Notebooks in 2026?

Claude, what do you think? "Yes, with survey data and industry coverage going back several years to back that up.

"Kaggle survey data (2017 - 2022)

"Kaggle, a Google subsidiary that fosters an online community of data scientists and machine learning practitioners, has conducted an annual industry-wide survey on the state of data science and machine learning since 2017.

"According to the Nashville Software School blog post "Kaggle Releases First 'State Of Data Science & Machine Learning' Report" (November 2017), which summarized the first Kaggle survey of 16,000+ respondents, 40% of data science practitioners reported using Jupyter notebooks as a tool.

"According to the Visual Studio Magazine article "No. 2 Data Science IDE? VS Code" (February 2021), citing Kaggle's "State of Machine Learning and Data Science 2020" report: "Jupyter-based IDEs continue to be the go-to tool for data scientists, with around three-quarters of Kaggle data scientists using it." That represented a decline from 83% the prior year, with VS Code rising as the second-most-used IDE.

"According to a Medium analysis of the Kaggle 2022 survey "What Changed from 2021 to 2022 in ML and DS?", Jupyter Notebook remained the most-used IDE at 57% of respondents, though that represented a decline of 5.7% from the prior year as VS Code continued to grow.

"The trajectory across Kaggle surveys: 40% (2017) → 83% (2019) → ~75% (2020) → 57% (2022). Usage is declining relative to peak, but Jupyter remains the top-reported IDE in the ML and data science community across that period.

"Industry coverage in 2025 - 2026

"According to the TechTarget article "18 Data Science Tools to Consider Using in 2026", compiled by TechTarget editors based on research and market analysis from Forrester Research and Gartner, Jupyter Notebook and JupyterLab are listed among the top data science tools for 2026, described as enabling interactive collaboration among data scientists, data engineers, mathematicians, and researchers.

"According to the Nebius blog post "What Is Jupyter Notebook? Why It's Essential for AI and Data Science" (September 15, 2025): "Jupyter is the preferred environment for prototyping ML models and debugging pipelines. Within a single notebook, you can handle dataset loading, construct a baseline model, experiment with hyperparameters and track validation metrics." The same article notes that Hugging Face -- the primary hub for sharing pre-trained AI models -- distributes the majority of its tutorials and model cards as Jupyter Notebooks.

"The shift: notebooks as a starting point, not an end point

"The Kaggle usage decline for Jupyter is worth examining carefully. VS Code has added native .ipynb support, meaning a portion of developers who report VS Code as their primary IDE are still running Jupyter Notebooks -- just inside a different shell.

"According to the official VS Code documentation "Jupyter Notebooks in VS Code", VS Code supports working with Jupyter Notebooks natively through the Jupyter extension, published by Microsoft. Once installed, .ipynb files can be opened directly from the VS Code File Explorer, with cells run using Ctrl+Enter or the Run icon. The extension automatically installs companion extensions for keymaps, MIME type renderers (LaTeX, Plotly, Vega), cell tags, and slide show support.

"According to the Microsoft Jupyter extension GitHub repository, the extension also includes a notebook-friendly diff tool for comparing notebooks in version control, and supports running notebooks on remote Jupyter servers, GitHub Codespaces, and Azure Machine Learning -- not just locally.

"According to the VS Code "Data Science in VS Code" tutorial, VS Code with the Python and Jupyter extensions and an Anaconda environment is described as a complete data science setup: "Visual Studio Code and the Python extension provide a great editor for data science scenarios. With native support for Jupyter notebooks combined with Anaconda, it's easy to get started."

"In short: the raw Kaggle Jupyter numbers likely undercount actual notebook usage, because .ipynb files running inside VS Code are still Jupyter Notebooks.

"Using VS Code with Jupyter Notebooks (.ipynb)

Thank you, Claude!

Why This Matters for AI QA

AI evaluation frameworks such as DeepEval and RAGAS deliver their quickstart tutorials and evaluation examples as Jupyter Notebooks. The workflow is: run an evaluation cell, see the output, adjust a prompt or threshold, run again. That iterative loop is what notebooks are designed for.

Next up: I'm going to try to get a notebook running, and see what the environment actually looks like.

Free Courses

Reference




Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTube | LinkedIn | Articles
Posted by at
Labels: , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)