Why I stopped using Jupyter Notebooks

[@IustinDornescu]

It's a good practice to have most of the code in python files - benefiting from the IDE of your choice, and separate reporting to notebooks focused on presentation. The mix of code (functionality) vs reporting (visualisation) depends on the data science project.

[@HmongCrypto]

This video and workflow is great, however you're also right about that. It depends what you're doing. I will be looking forward to trying to adapt to this workflow in the future. Thanks, the both of ya

[@juaneshberger9567]

I used to use jupyter notebooks for everything back in college as well (even stats/math homework). Switched to using neovim and the ipython shell in the terminal, once you learn neovim using the terminal becomes so much faster than jupyter in vscode. Also you can use kitty terminal to display plots in the terminal and termpandas to have scrollable dataframes like in vscode jupyter, all in the terminal. (i3 window manager/tmux also make the workflow so much faster and enjoyable.)

[@KamKamKamKam]

Can it display images directly (using matplotlib or something)? What plugins are needed? I'm also trying to move my jupyter work flow into neovim.

[@juaneshberger9567]

@@KamKamKamKam yes kitty terminal allows for displaying images directly in the terminal its pretty cool (save to png and then use os.system("kitty icat fig.png"), python package termpandas (I created it) allows scrollable data frames in the terminal, and ipython shell allows for the whole REPL experience in python. All inside the terminal. The only thing I have not been able to get to work have been interactive html plots in the terminal like plotly express. However, the best data science experience for this in my opinion is split screen neovim and having a streamlit dashboard that automatically updates on save on the other side of the screen. This allows for moving super fast in neovim on the left side of the screen (also using i3 window manager to move around) and a browser with a streamlit app so I can have fully interactive dataframes, plots, filters, and basically any web technology in the streamlit app. Let me know if you are more interested in my setup, I could probably write a medium blog or something explaining my setup, and listing the list of tools/packages I use.

[@slayga]

This makes so much sense! I am only just beginning my python journey and am loving your videos. Finding information like this so early on for me seems invaluable to build better habits now with regards to best practices and efficient workflows

[@xugefu]

Thanks!

[@daveebbelaar]

Thank you! 🙏🏻

[@TreeLuvBurdpu]

This is very true. I love the idea of Jupiter notebooks, and I've used them for years, but they not composable.

[@tech365ng]

Jupyter notebook can actually work with AI assistant like codeium same way you do in VSCode.

[@zyphtron]

Codeium is buggy in Jupyter notebook

[@WhiteGaming5883]

i started with R, and i'm having a hard time to get used to notebook workflow so this basically helped a lot~! thanks~!

[@erikjohnson9112]

While I am about to get into Marimo, I also found your tips about interactive Python along side code very interesting and something I've been missing out on big time.

[@pawelkubik]

You can also use #%% comments to mark sections of your code as "cells" that would be executed as a whole with the keybinding. I'm not sure if this "run cell" does not conflict with "run line/selection" on some platforms, but you can fix that in the config.

[@sio-salt8979]

I had same problems!😅 This will help me so much!🎉

[@amiralikulov9300]

Cool. Is there anything simillar in PyCharm ?

[@stevehenty6379]

Python Smart Execute. I have no direct experience using it, but it apparently runs code blocks in a console window using a Jupyter cell behind the scenes. Requires the Jupyter plugin to be installed.

[@NotRyan.]

this video is for the people who are already expert in data analysis. letting ya'll new analyst like me before you start watching the video.

[@ruperth.]

Thank you for sharing this is the type of thing a seasoned developer needs to hear.

[@OscarQuintero-td8cy]

Thank you , I'm following your suggestions

[@GuanglinDu]

I couldn't agree with you more, Dave. Thanks for your informative video on how to use my favorite IDE VS Code with Jupyter Notebooks. This combination seems to be the best duet for data scientific research.

[@divyabhatia9916]

really good. thanks so much for teaching this feature I didnt know before. I like this more.

[@bansonsolen5586]

Thanks! For your info, I just dropped pycharm+jupyter, switch to vs now!

[@awakenwithoutcoffee]

Hi Dave, what VsTheme(s) are you using ? looking very clean. Like always, appreciate the sharing of your knowledge.

[@gedtoon6451]

In Jupyter, why can I still not cut and paste text in a cell using the mouse?

[@michaelvarney.]

In matlab you are able to define a code block with a ‘##’ that once you are inside it you can shift-enter and it will run. You can also have a variable explorer window. I wonder if there is such functionality in VSCode?

[@abc_cba]

I completely moved to google colab to save RAM and space being overused by Jupyter, only while complex coding do i use vscode else, even docker images are my option

[@stevehenty6379]

I'm curious if this style of coding creates any lingering limitations or style / organization choices in your "pure" python files due to the holdover of executing "cells" of code during the development cycle. Stated differently, do you find that any production-ready refactoring is necessary after the functionality is ironed out using Jupyter-esque interactive coding?

[@donharrold1375]

It depends on where you are on the learning curve. I personally love Jupyter notebooks.

[@micbab-vg2mu]

Definitely, I prefer your workflow over Jupyter Notebook. I am in the phase of testing Cursor, which is similar to VS Code but has some more AI features.

[@daveebbelaar]

How do you like it so far?

[@Liinyun]

Is there anything same in nvim? I'm looking for a plugin like this

[@axhraf7712]

I'm assuming you don't do any heavy pytorch stuff where you need a GPU? Or do you just run them on your mac?

[@daveebbelaar]

I typically use Azure for that, but I don't often have to run very heavy models.

[@patricior7300]

This methid is like using R. Thanks You! Now I can learn programming in Python sane as in R. Excellent trick ! 👏👏

[@chongdashu]

Great tip, might migrate completely to this workflow. Jupyter Notebooks are great, but as you say, very clunky when approaching as a software engineer.

[@mememaster69-n4x]

If you use VS Code then you need to use your existing GPU/CPU but with collab you can use TPU AND GPU free

[@TreeLuvBurdpu]

Isn't there an .ipy file type or something. There like Python built-in interactive file type, i think. And you can click one segment at a time. I think it was the basis of .ipynb

[@mtin79]

would this work with a deno kernel in a jupyter notebook too?

[@mememaster69-n4x]

i would recommend the google collab but only the runtime problem with free version

[@parisneto]

You can start Jupyter Server in your local machine and connect Colab to a local environment, awesome ! But doesn’t solve the Jupyter notebook challenges, but it’s a much better IDE than VS Code with some many "Microsoft UX” that forces you to adapt in a non efficient way ,plugins and environment challenges…. Modularizing the code and using GIT helps a lot in my experience .

[@realCleanK]

Thank you!

[@filipfloegel3718]

@dave did you try aider with your workflow. I think that could work for us data people working with vs code supported by ai

[@davidmuller3849]

I don't get it 😅

[@ringpolitiet]

Agreed. It's the only way I work.

[@John-xi2im]

I am bounded to use jupyter notebooks because of low level processor (AMD Athlon Silver 3050U) which is present in low priced laptops, as I am searching for a job, I had less money hence I bought this laptop.

[@richardbennett4365]

One can use marimo instead of juoyter, or one can even make Code use a notebook. I think the presenter did the latter, but not marimo.

[@prison9865]

Mate, thanks a lot for this. I never liked Jupiter notebooks and I’m glad I’m not the only one haha. Either way, how would you send your structured codes in different python files to cloud computing if you wanted to use that

[@PerpetualPreponderer]

There’s something I’ve been struggling to understand. Why do so many indie devs, especially those not using windows, continue to use vs code? Unless you’re living under a rock, and even a dev living under a rock would know this, it’s pretty much a given that whatever you type is being shipped off via telemetry, and that then opens up the risk of people with way more resources at their disposal, to simply copy, iterate quickly on whatever you’re building, and bring it to market before you’ve even had a chance to test. What gives??

[@gvm271]

what to use instead ?

[@Cyril...]

That is why there is Jupyter lab, all the issues u are talking about has been addressed in jupyter lab

[@millionsinmemecoins]

great tip

[@oemmah]

i am so switching already

[@wayne7936]

This is amazing.

[@Besinnung]

Thanks for this fidio

[@jeffrey5602]

Jupyter Notebooks are only bad if its the only thing you use. For exploration and prototyping its king. Also VS Code sucks still big time when it comes to NBs. So many things not showing properly or kernels not restarting properly, not releasing memory when training torch models. I use neovim for coding and jupyter lab for exploration. They serve two very different stages of my projects

[@richardt6347]

notebooks are amazing for DE.

[@prylypko]

nbdev is all you need

[@tobymillerFPA]

Rstudio FTW

[@BielAltimiraTarter]

nah

[@Finezzato]

we don't give a fuck.

[@ringpolitiet]

Why are you here, Sir?