Use THIS Language to Speed Up Your Python Code

[@amynagtegaal6941]

Nim is actually also highly configurable, you can virtually shape it to any syntax you want

[@IsaacHarrisHolt]

Interesting! I didn't know that, but honestly I'm not sure if it's a good thing. Allowing language syntax to be changed makes it difficult to maintain long term and scale if the team grows or if something is open source. It sounds like the opposite approach to things like Go and Rust, which standardise everything right down to formatting

[@tom_stephen]

@@IsaacHarrisHolt It is not requirement though, it just offloads the forced syntax additions Nim developers could apply on users - instead, you rather choose if you want to expand your syntax or not. Good example on that is that there's -classes- library basically porting OOP from Python to Nim, using almost the same syntax - which is nice if you find Nim's structs not intuitive. If you want to write your own system using DSL it is also pretty great feature, because you can adjust AST to your needs. In general Nim has really great parsing features to work with string, so it's not only about macros, but all ways around. This topic aside, it makes me very happy to see Nim being showcased. It is by far my favourite language and I can't believe how unknown it is, given it should at least fight with Rust on recognition level. It is everything I wished in Python to have, without any bloat/stupid choices I dislike for example in Go. The only real drawback is immature library system, but that's expected from niche language.. but on the other hand making libraries is the simplest it can get, I already published two libraries while being pretty dumb on anything exceeding beginner level. Hoping that 2.x release will make this language have recognition it deserves and being brought up as "next Python" instead of Mojo, as those are exact opposites in philosophy and I find Nim way more comfy and interesting, and better for everyday use.

[@IsaacHarrisHolt]

I also find Nim really interesting, and there are lots of things I hope get improved over time. I'm still not convinced by being able to define custom syntax, so I might need convincing on that one 😅

[@tom_stephen]

@@IsaacHarrisHolt Well, custom syntax is tricky one, as it has both advantages and downsides - that's why in Nim community people tend to say that "macro is last solution" and things should avoid macroing too much. Most of macro usage is handiness - for example it's really fun to simplifiy string to "str" or "uint64" to "u64" - you can do aliasing very easily. "Classes" library doing Python-like OOP is yet another stellar example, for Rust fans there's also "Results" library that incorporate Result type, and there's also "Questionable" lib that makes options Kotlin-like (with use of "?" as symbol for none values). But if you overuse macros, you will have hard time deciphering your code, so I totally get why this doesn't feel like good idea. It's all about how you play with it that matters.

[@IsaacHarrisHolt]

@@tom_stephen but what about someone coming to read your code for the first time? That person might have a very good understanding of the language, but they'd have to first read your macros to be able to understand the rest of your code, especially where you're changing syntax. Adding `?` in one case might be different to adding `?` in another - that's the problem with things like operator overloading too.

[@anishbhanushali]

Hey Isaac, Is there any ongoing/completed production grade python extension projects written nim such as candle and polars built using Rust ?

[@IsaacHarrisHolt]

Possibly! I didn't come across any major ones, but it doesn't mean there can't be. Let me know if you create anything!

[@anishbhanushali]

@@IsaacHarrisHolt Sure, nim looks tempting

[@blaisepascal3905]

There is Happyx, an asynchronous full-stack web framework that currently have binding for Python and Node.js (it will soon have a binding Java and Kotlin)

[@IsaacHarrisHolt]

Ooh interesting! I'll have a look

[@ViaConDias]

The thing about Nim is that it's so easy (close to writing Python with type-hints) that there's not much need to build part of your project in Python and then only build the process-heavy parts in Nim as you would do with Rust. You just build the whole thing in Nim. Thus, there's not that much incentive to write Python extensions in Nim. That said, there are some and I'm sure more will come just because the community will do it for fun. I love Python3.x and it has been my go-to language for years, so I started out with a similar mindset when I decided to learn Nim. I can just use it for the parts that need a speedup. But after a few weeks, Nim completely took Pythons' place and the only Python I write now is when I'm teaching Python or someone asks me to help them with Python.

[@draxler.a]

hi Isaac . supposed i want to make 1M request to different url with 100 or 200 concurred thread how is more fast python or nim (pure code) ?

[@IsaacHarrisHolt]

If you're making network requests, most of your program's execution time will be sitting around waiting for the response. I don't really know what Nim's threading model is like either, so I'm probably not best placed to answer this. I'd imagine Nim would be faster, but not significantly

[@lolikpof]

What about Mojo? Is it developing? Was all the hype for nothing?

[@IsaacHarrisHolt]

I haven't really kept up. It's open source now, so hopefully it'll be production ready soon

[@AliciaSykes]

All the hype was for nothing (at least for now), it's a hugely limited subset of Python compatibility, and Modular haven't open sourced most of it, which is frustrating.

[@twenty-fifth420]

To add, I think Mojo might be in a weird space because of the AI focus. Which would be great for specialist devs and yeah, it is as fast as C++. But the current state of AI is very closed source with only a few 'alternatives' as exceptions. And that can hurt it. At least you can do other things in Python. IMO, anyone reading this in 2024/2025 should just learn Python and then Nim. Or if you want to be brave, Rust. I don't think Modular will be open sourcing the LLVM stack for Mojo anytime soon due to the AI incentives. Which is a bummer.@@AliciaSykes

[@RagHelen]

The compiled output in the pyd file is binary and not just betacode, right?

[@IsaacHarrisHolt]

PYD and SO files are typically binaries :)

[@RagHelen]

@@IsaacHarrisHolt At 2:15, you say, it makes it really easy to compile Nim to C, C++ or even Javascript, but in our case, we're telling Nimpy that we want to export this function to Python. What does this even mean? Is it Nim code compiled straight to binary with some necessary pyd flourishes or is it, as you imply, translated to Python first and then compiled? That would bring type checking of every instance of a for loop back to the script and all the other stuff which makes Python slow. Wikipedia says: "The Nim compiler emits fast, optimized C code by default." That's why it needs an additional C compiler to get down to binary.

[@IsaacHarrisHolt]

I'm not an expert here, but I believe it will be first translated to C code with Python bindings, and then compiled to the correct format.

[@deathlife2414]

You got skill issues right. I have tried both and setting up the database connector was easy in rust and I failed on nim many times.

[@IsaacHarrisHolt]

Interesting! Just out of curiosity, is this for a Python extension or something else? And if it is a Python extension, I'm curious why you wouldn't just use a Python ORM. I can't imagine the speedups are huge unless you're then doing loads of data processing in memory

[@deathlife2414]

@@IsaacHarrisHolt it was for something else. Python took to much of the processing power when loading csv file and I started looking for ways to make it faster and consume less power. I could have made csv parser in c++ but I had to write it from scratch and didn't like that so looked around and found both nim and rust. I first started with nim initially it was good but when I was trying to connect to sqlite3 database nimible makes alot of dependence crashes. But rust was better for dependence management. But that to have problems for example cargo sometimes have the thing going on in npm. Issues like downloading and compiling to become to large project 1gb space.

[@IsaacHarrisHolt]

Interesting! In the context of a Python extension though, you're unlikely to be experiencing slowdowns as a result of your ORM (and if you are, you probably want to be using a different language anyway), so I'd argue Nim is still a good choice.

[@deathlife2414]

@@IsaacHarrisHolt yeah it easy compared to rust but I still choose rust b/c of its speed

[@MatheusOliveira-er4gq]

Lets see when it gets more mature. Usually we need to speed up small parts of python code

[@IsaacHarrisHolt]

True! It's an interesting language for sure

[@sayorancode]

it is already usable

[@AkeemBousquet]

cool video)

[@IsaacHarrisHolt]

Thank you!

[@GoldenBeholden]

Nim is nice, but a bit obscure for production in my opinion. Zig may be a better option in that regard if you don't mind a more C-like syntax but still think Rust is too overbearing.

[@IsaacHarrisHolt]

Every language is obscure at the start. I think Nim just needs a bit of love. It's an interesting one to play around with

[@tom_stephen]

@@IsaacHarrisHoltExactly! There's no way for language to rise from obscurity if not given enough attention. Library ecosystem alone needs many contributors to be healthy and include all features needed by everyday user. It's not like those libraries are written by ghosts or langdevs.

[@IsaacHarrisHolt]

They'll all be written by ChatGPT soon 👀

[@tom_stephen]

@@IsaacHarrisHoltI know you're joking, but gosh, I damn hope not. Can't imagine how awful code or lib structure would be with GPTs being what they are right now o.o

[@guilherme5094]

👍👍!!!

[@IsaacHarrisHolt]

Glad you found it useful!

[@naranyala_dev]

zig, zig, zig

[@IsaacHarrisHolt]

Looking into it, don't worry 😉

[@goose_clues]

Besides the complexity of syntax, Rust community is the top reason I hesitate to start.

[@IsaacHarrisHolt]

Really? That's interesting. From my, admittedly few, interactions with the Rust community, they all seem lovely. I think it's just the Foundation I disagree with