New library streamlit-jupyter, a new way to develop streamlit apps in jupyter notebooks

nbdev
streamlit
jupyter
Author

David Dobrinskiy

Published

February 7, 2023

Intro

Hey there, I created a library streamlit-jupyter to allow for seamless creation of streamlit apps in jupyter notebooks.

The way this works is it detects your environment, and if the script is run from a jupyter notebook, then it wraps existing streamlit methods to replace them with ipywidgets alternatives - that way you can code as you would in streamlit, and then displays your page as-you-code.

Then you export your notebook to a .py file, and run it via streamlit as usual (no change in behaviour there)

Motivation

I have been a Data Analyst at mutliple orgs for the last 6 years, and would always feel the pain of sharing my work to non-technical colleagues.

A typical pipeline would look like this:

  • open jupyter, download some data
  • play around with it, make some tables and visualizations
  • export it to csv
  • upload csv to Google Sheets
  • manually create graphs
  • share the Google Sheet with your colleagues

While this wasn’t too bad, updating anything here filled me with dread:

  • find the code
  • re-run it
  • if the output changes format, good luck updating that Google Doc without anything breaking
  • update all the charts
  • redo everything if the manager has additional questions, or wants to look at other dates/countries/etc

Streamlit partially solves this, it allows you to code a dashboard/app in Python and give the end-user a friendly web-based interface.

And all those dates/cohorts/countries/etc can become GUI sliders, so that your colleagues can fiddle with data without any Python/SQL knowledge themselves.

However, exporting my Notebooks to .py scripts for Streamlit was still a hassle - you need to refactor your code to work with Streamlit, and then it does not quite work in Jupyter anymore.

Now, what if you could have Jupyter notebooks as a single source of truth, and it would play nicely with Streamlit without any extra work? All behind the scenes?

Well, now you can - that’s where streamlit-jupyter comes in.

References

This project builds on the shoulders of giants, and would not be possible without these amazing packages:

More on each:

Jupyter

Most likely jupyter needs no introduction, it is the de-facto tool for anyone doing scientific programming.

Streamlit

Streamlit is a framework to create interactive python apps (mostly focused on data analytics, but can be anything).

It can have dropdowns, tables, maps, charts et cetera:

For example someone built this awesome background remover in Streamlit, see streamlit gallery for more great examples.

I love Streamlit because it is perfect for sharing your work with others, especially in a work-related setting: giving your stakeholders a clickable and responsive app is so much better than a pdf or html version of your notebook!

nbdev

nbdev is a literate programming environment based on jupyter.

This means that your notebooks become a single source of truth for code, tests and documentation.

The main reason that I love it (and chose it for this project), is that a regular Data Science project has a workflow that looks like this:

  • code something up in your jupyter
  • move helper functions to utils.py
  • experiment with different approaches
  • copy-paste your code to app.py
  • don’t forget to update the docs in your .md files (or, god forbid, notion/confluence 💀)
  • now if something doesn’t work, or you want to make some changes - start at the beginning

Nbdev combines all of those steps into a single place - your jupyter notebooks. And exporting your code as a library works seamlessly behind the scenes.

Jeremy Howard from fast.ai explains the benefits more eloquently in his I like notebooks video.

Tying it all together

Now, combining all of those together means that you, as a developer, never have to leave jupyter, and can visually iterate on your work.

And all of that gets exported into Streamlit, so that your project is always shareable and usable to anyone with a browser.

streamlit-jupyter is the missing puzzle-piece here, since it allows you to see how your streamlit app will look like, without having to leave jupyter.

So now my workflow looks like this:

  • write all my code and experiments in Jupyter
  • document it as markdown (Can also be displayed in Streamlit, so your stakeholders know exactly what you did and how to use it)
  • mark the parts relevant to your app for export via nbdev, while keeping your experiments in other cells for future reference
  • parametrize the app with GUI sliders, so your dashboard/app becomes largely self-serving
  • this applies to all the charts as well, no manual work there as well
  • and if you still need to change something in code - no worries, there is a single source of truth now

While this may look harder than a simple script + export to google docs, trust me - down the line, this saved me hours upon hours of debugging, remembering what came from where, and manually iterating the script to answer any follow up questions.

And after Streamlit added multi-page apps this became even easier: now each task can become it’s own page within a single Streamlit App, so you only have to deploy it once, and a single repo contains all your work on multiple projects.

Installing it

pip install streamlit_jupyter

How to use it

Take a look at our example notebook

The main idea is for you experiment and develop in your notebook, visually see all the pieces, and then convert the notebook to .py to be run by streamlit.

It’s as easy as adding these 3 lines to your notebook, ant that’s it: everything just works behind the scenes now!


import streamlit as st

from streamlit_jupyter import StreamlitPatcher, tqdm

StreamlitPatcher().jupyter()  # register streamlit with jupyter-compatible wrappers

Now develop in your notebook as usual, but with the ability to use Streamlit widgets and components.

See some visual examples below, and check out the example

Lifehack: Easy export of your notebook

The best way to export a notebook is NOT jupyter nbconvert, trust me on this.

It’s nbdev, and instead of running nbconvert from your terminal, do these 2 things:

  • add #|export comment to the beginning of cells that you want exported as scripts
    • all the other cells will NOT be exported, so you can use them as playground without having to clean
  • add this cell to the end of your notebook:
from nbdev.export import nb_export
nb_export('app.ipynb', lib_path='.', name='app')

And that’s it, your app.py is now fully syncronized with your notebook.

See a more detailed example in this blog post by Hamel Husain

Demonstration

This is what a running streamlit app looks like: ddobrinskiy-jupyter.streamlit.app

See jupyter vs streamlit below:

Jupyter Streamlit
Markdown and headings alt alt
Interactive data entry alt alt
Pick and choose alt alt
Dataframes, caching and progress bars alt alt
Plots alt alt

Contact me

This is a very early beta and my first try at open source, so please share your thoughts and reach out with any questions/critiques.

I can be reached at:

p.s.

I’m also open to work (open source does not quite pay the bills yet), so if you have a challenge around Data Analytics, let’s connect.