Which Python Package Manager Should You Use?

In this episode of AI Adventures, Yufeng discusses some of the options available when it comes to managing your Python environment for machine learning and data science, and helps you make an informed decision based on your needs.
Learn more through our hands-on labs → goo.gle/3gY4Auo
Associated Medium post "Which Python package manager should you use?": goo.gl/uGwXiA
Resources:
Pip: goo.gl/YwDW8
Virtualenv: goo.gl/5ix9f
Anaconda: goo.gl/thHWnF
Pyenv: goo.gl/RU0Swa
Watch more episodes of AI Adventures: goo.gl/UC5usG
Subscribe to be notified of new episodes: goo.gl/S0AS51
#AIAdventures

[@MouldyCheesePie]

Now i finally fkn understand packages and virtual libraries. Spent hours confused about this yesterday... The youtube algorithm is just too good, it knows exactly what I need.

[@gibsosmart]

you never know youtube is reading every bit of your online presence

[@katsuru]

me rn

[@mosheleland5104]

i know Im randomly asking but does anyone know a trick to log back into an instagram account?? I was dumb lost the password. I love any help you can give me.

[@vvzen]

Anaconda/miniconda can actually handle the python version too, it’s kinda the main goal. It’s a bit sad that he missed that point. Also conda is meant as a way to handle lower level libraries too, like c++ dependencies, not just pure python packages.

[@roshunepp]

Should my python update via anaconda take 3 days? I feel stuck. I also feel like "it will finish tonight" every night lol.

[@vvzen]

@@roshunepp what do you mean exactly? what command are you running and on which OS? on which version of anaconda? Have you tried miniconda? 😀

[@meljuncortes4420]

Very good explanation and elaboration. I like this kind of demo where there is a direct elaboration of the topics unlike other video tutorial difficult to understand beside of the accent of language.

[@jorgeastiazaran]

Thanks for your clarification. I was struggling to understand the differences between them, since I was following different tutorials on ML and they use different methods to manage their Venvs.

[@amdenis]

You can, and too often will, get difficult to unravel conflicts when using two alternative pacman environments such as pyenv along with conda. The problem is that there are significant discrepancies between their dependencies/conflicts look-up tables. Also, the complexity of interactions when adding pipelining (e.g. Kubeflow, Airflow, etc), can become even more problematic, since a lot less real life usage testing generally occurs across those two, verses even bare in-environment pip usage (i.e. no assumptions made via a limited number of testers, which would be the case with pyenv competing with Conda/AC).

[@munnabhaimbbf123]

Thank you for the description!

[@bijenderkumar2934]

I mostly work in nodejs and sometime I have to use python which makes me confused about which package manager to choose, your explaination help me clarifying the differences between them. thanks!

[@GregoryYepes]

Fantastic video and super useful. Thanks for sharing!

[@stevequan7306]

Thank you for sharing! Anaconda is kinda "too well packaged/organized", I uninstalled is but only keeping miniconda then combining with venv.

[@autogecko318]

simple and clear explanation. thanks.

[@RenanBenedicto]

Thank you! It was so easy to understand and I could share with my teammates. Could you please post a video explaining maven and how use it to manage and compile Java for DataFlow and Apache Beam?

[@dryogeshbhirud]

awesome presentation with great video editing. which software did u use to edit the video?

[@Aka00710]

I like using the jupyter docker images for data science stuff

[@rhythmsharma4561]

best video on python package manager 👍, i use anaconda and pip

[@amburghermozzarella7506]

thank you for the video! how can I install a python library that I created and use it in anaconda? I need it to create power bi scripts

[@rudeadyet1992]

I use pyenv and pyenv-virtualenv. A bit confusing at the beginning, but once you begin to understand, it’s like a butter.

[@roshunepp]

I've been struggling with my anaconda env. Trying to update my python version to 3.9.6 but it's been running for 3 days dealing with conflicts. It's partially my fault for wanting the latest in conda navigator and not leaving well enough alone with 3.9.4.

[@rahulsriram6295]

Great video, chose to use pipenv :D

[@AndrewStorey]

Great video. What are your thoughts on using Docker for this?

[@matejjukic5776]

Do you know how to resolve conflict between packages? I need 2 different versions of the same package for my code to work but pip deinstalls the older and installs only the newer.

[@realCleanK]

Thank you!

[@marcpanther7924]

Is `virtualenv` still useful given that `venv` is now part of Python 3?

[@e1nste1in]

I use anaconda for separation and pip for package management 😅

[@kwameakuffo9611]

i use pyenv, and virtualenv for the control and security it offers projects. Collaborative dev env requires packages to be synchronised, and secure.

[@rohanjoshi1036]

Hey i am new to python, as you mentioned that you use pyenv sitting over both virtualenv and acnaconda can you plz show us a demo how to setup such environment and basics of using such assembly.

[@hafidhrendyanto2690]

just try it yourself man, it's so easy!

[@DeepakChauhan-wu7ei]

For people who have same question, u can check description or check some tutorial. I myself setup my virtual environment though watching yt tutorial

[@rezamirkhani4747]

I use miniconda but I'll give pyenv a go to see what it can do

[@jhoningsoft]

great and concise video!

[@jitrammaharjan9506]

Great video

[@user-or7ji5hv8y]

Is there a great tutorial for pyenv?

[@NamasenITN]

Why would distinct libraries interfere with each other, when not used (I.e. imported) or even when used? That's a premise that Python's ecosystem might be messy. Please explain me where I am wrong with my reasoning.

[@DatascienceConcepts]

Awesome information! How come I never saw this before :)

[@lakshminarasimmanv]

I’ve been using both. Without knowing both are different.

[@prawnstars3160]

thank you, TechLead

[@eladlevy9494]

Im considering poetry for my next project, what do you guys think?

[@dumidupramith3881]

super introduction

[@AnirudhGargi]

Never knew of pyenv

[@MsKante]

What about pipenv?

[@josedasilva585]

I'm going to study and learn step by step, but I'll get it.

[@amitbhadula7025]

You will! 👍

[@MarcMcRae]

Wish some "teachers" offering their courses at places like YT, Udemy, etc. would explain this stuff properly before diving in and assuming! Cos the Python tooling ecosystem IS painful..!! Assuming we , the "self taught" all know how to NOT mess up our base machine is all too common. After MUCH searching this is one of the simplest (Keep It Simple S*****) explanations to help "production-ise" your base system. Thank you!

[@Shoey]

Pipenv is great for 2018.

[@vedx]

Tell about Auto ML.

[@sunildhaka4528]

awesome great

[@alexanders8341]

It's probably worth mentioning that pyenv does not work on Windows.

[@RC-bl2pm]

what about venv? Where does it fit in?

[@petros-estin-petra-]

pretty much the same as virtualenv, but comes with python preinstalled

[@linsonkurian]

👍

[@ego_sum_liberi]

Docker it's the way to go with me. But, if not miniconda it is........⚙🥳🙋‍♂️😉💡🍻

[@orilio3311]

I still don't get qhy anaconda exists. Is the only reason for it that it comes with pandas and other DS packages built-in? So removing this one command ia the reason: 😅 pip install pandas matplotlib numpy jupyter

[@nardu]

Please recommend a resource that explains how to back track out of a spider's web?

[@VojtechKane]

This is a great video showing why I am not using python ;-)

[@Kathypeer]

Python has the most comprehensive libraries when it comes to data science. Is far from being perfect, but no language gets anywhere close to Python in this area.

[@eaglet3d]

It's too bad pip v10 has broken the installation of many packages. This is only going to create a higher cost of entry for new Python users. You will have less pain by not following pip's recommendation of upgrading to pip v10.

[@dave47663]

lol, good. More jobs for us. Cheers

[@zielin32]

0:35 - I see what you did there

[@sheldonlee7731]

I wil use both

[@sheldonlee7731]

together

[@croninsiglos9855]

Pyenv is unfortunately not a mature project, it doesn't allow environment location customization, and it doesn't work properly in multiuser environments such as HPC clusters, etc when sharing projects on a shared filesystem. Perhaps it's cool for personal projects, but doesn't belong in the scientific computing realm.

[@example.com.]

Miniconda is awesome

[@DURGESH-gy8hb]

what difference to anaconda

[@bunny_the_lifeguard9789]

@@DURGESH-gy8hb Anaconda comes with 3GB of packages preinstalled, Miniconda only with the necessary ones and let's you decide whether/when to install additional packages.

[@ThunderBow98]

why not just use both?

[@triplept1633]

About two years later, people are gravitating towards `venv`.

[@colour_bloo]

Python programmers "Learn Python, it's incredibly intuitive and easy to pick up!" Also Python programmers: "Check out this dependency spaghetti! 🍝 It's SUPER easy, just use 3 SEPARATE TOOLS to manage it all!" On a serious note - I sort of threw in the towel and went all conda. It's not the best solution, but it's (partially) compatible with pip, and I can fake the functionality of pyenv by simply creating new envs with a different version of Python. 👍

[@not_a_human_being]

always spaces..

[@Arville27]

Poetry FTW

[@yournadanvlog]

anaconda ✌

[@Funny9689]

anaconda for da win

[@Funny9689]

anacondas are dope

[@billhab1]

i prefer pipenv ...

[@griesrt]

TABS!!!!!!

[@petros-estin-petra-]

Pretty sure your tabs will be converted to spaces.

[@alice_in_wonderland42]

Poetry with tox

[@deeplearningpartnership]

conda

[@jamilahsakim8817]

M

[@jamilahsakim8817]

G

[@jamilahsakim8817]

H

[@jamilahsakim8817]

J

[@feuerundwasser282]

I use linux. ;-)

[@jamilahsakim8817]

K

[@jamilahsakim8817]

F

[@jamilahsakim8817]

L

[@jamilahsakim8817]

D

[@jamilahsakim8817]

C

[@jamilahsakim8817]

E

[@jamilahsakim8817]

A

[@jamilahsakim8817]

B