SurrealDB - Rust Embedded Database - Quick Tutorial

[@SurrealDB]

Great tutorial Jeremy! Thank you! There are a number of features and performance improvements being worked on before a 1.0.0 production-ready release. 1. You can already define indexes for unique record constraints, but indexes are currently not yet used when filtering on a table. As a result, queries which filter on tables with a large number of records will not be performant. This does not affect graph queries, or time-series record ranges. When completed, this will have a major performance impact to query speed on table queries. In the meantime, you can use aggregate views to improve single-table queries. 2. Improvements to how data is managed before being serialised to disk, reducing the amount of data cloning and copying within the database processing runtime. These changes will improve the performance of writing to, and reading from, the underlying storage engine, and will have a significant impact on performance and query speed. 3. Improvements to how data is cloned and copied within the database processing runtime, and within the Rust programming language. Currently when comparing values or filtering on values, certain data is cloned and copied unnecessarily. Improvements to this behaviour will have a significant impact on performance and query speed. 4. Improvements to how data is serialized to disk. Currently when serializing data to disk, the serialization technique is more verbose than necessary, increasing the size of the data residing on disk. Improvements to this behaviour will have a significant impact on disk storage and query performance. We have GraphQL integration and our own key-value storage engine coming in due course too. Thanks once again Jeremy!

[@JeremyChone]

Thank you for the kind words and this great information. This is awesome! Quick question: I plan to do another video about your Rust driver (surrealdb.rs), but I noticed it is not linked from your Rust client Web page (surrealdb.com/docs/integration/libraries/rust). Is the Rust driver ready for public consumption?

[@nickton]

@@JeremyChone I think it's an automatic comment... there is the same message in another video on KZbin

[@kaporos]

@@nickton nah how would they get his name (it would me automatically split channel name,... butt that may be a copy/paste)

[@noblenetdk]

This is the only video I have found that actually explain how to use SurrealDb

[@JeremyChone]

Thanks. Btw, this is when using SurrealDB as an embedded Db. When using SurrealDB as an external DB, SurrealDB has released their Rust driver few weeks ago, and it is a similar but different API. I will do a video about it soonish.

[@flyingsquirrel3271]

Thanks a lot, this is a great introduction to SurrealDB!! One little tipp from me: Your little trick where you set your return type to void first so the compiler doesn't complain is actually not necessary. You can add the correct return type right away and just add "todo!()" at the end of the function. The todo!-Macro panics if it's executed, so the compiler knows that the function never returns - that's why it stops complaining about the return type, until you remove the todo! Edit: Sorry, in this case it doesn't work because of the impl Trait return type xD

[@JeremyChone]

Yes, I usually use todo! when I can except when returning impl.

[@ThePromptTailor]

I wish I found this video four hours ago, before I lost my mind. Very good job explaining everything!

[@dragonmax2000]

Incredibly excellent tutorial and very complete. Thank you!

[@cthutu]

Fantastic video, especially providing an iterator over the SurrealDB data structure, which is a bit cumbersome to use. It helped me a lot.

[@thorjelly]

Using Diesel + Postgres right now and I like that it deserializes results of queries into well-typed structs, with fully static type checking because diesel knows the database's schema. Any way to do that with schemafull tables in surrealdb?

[@ehfoss]

Great content. And Option::transpose.. wow! This was super helpful!

[@jl4zz]

Yeah, I agree. You do a great job of explaining your thought process as you go. I learned a few ways to more cleanly write my rust in this one video alone.

[@geekofia]

Thanks to the GOD and YT algo for pointing me to the best rust channel! I said best for the great production grade code following best practices and architectures. Incredible stuff! Love it ❤

[@JeremyChone]

Thanks. Btw, this video uses a pretty old API from SurrealDB. They have a new API now. Make sure to use the latest. Happy coding!

[@RobertoOrtis]

Amazing! Please post more videos about surrealdb and rust!

[@alexandersemionov5790]

this is superb! What is the resulting app size in release mode?

[@JeremyChone]

Sorry for the late answer. This is a very good question. With the 1.0.0 beta 9 (this video was beta 8), it's 10MB (--release).

[@jensereal511]

Awesome video as alway! One thing i wonder is: how do you have folding like in 8:55 on line 26. When i fold my code i always get 2 lines of code with the second line being the closing bracket :(

[@JeremyChone]

Thanks for the kind words. I did not do anything special. I get the two lines as well when the function signature takes multi line.

[@0xZeeS]

This is probably 1 of the best resources for getting started with Surreal. A small improvement: Timestamps (specifically for the CRUD operations) would be awesome :)

[@JeremyChone]

Thanks for the kind words, and good point about the timestamps. I will add them.

[@fractal-tess]

Very informative and throught tutorial. I cannot wait to see what surreal has in `store` for us in the future. On a sidenote: What is the application that allows you to draw on the screen ?

[@JeremyChone]

Screenbrush, in the Mac App Store.

[@David-iq1kd]

The official SurrealDB site still has no documentation on embedding SurrealDB. Maybe you could contribute some step by step instructions??

[@noblenetdk]

I didn't get the map - why do we need this?

[@JeremyChone]

It's what SurrealDB API takes to update the record (i.e. "row") values behind the scene.

[@subramanianjegannathan3299]

hello, is there a way in tauri to expose the db operations for actix actors?

[@JeremyChone]

I assume you could write your bridge from the Tauri commands. Also, not sure if you mean Actix Web, as running a local host. I would recommend to use Tauri commands instead. (Always better to avoid local host when possible)

[@subramanianjegannathan3299]

am trying to use actix actor repo alone in a project and need surrealdb to be accessed async by the actors. actix web naturally has a manage function to manage the db as state. but what am trying here is without the actix-web framework, manually expose db operations for the actors also implemented manually.

[@JeremyChone]

I have not seen anything Actix/Actor specific in Tauri or SurrealDB.

[@bouguerrahamza7902]

can you make some GUI applications ? with rust using rust linked database and do some CRUD ?

[@nickton]

Are there any rust ORM for surrealdb available now? I think it is not a good idea to use execute command without a database mapping

[@michah3956]

Is it possible for SurrealDB to store binary files?

[@JeremyChone]

I'm uncertain if they offer a blob type. If they do, it's likely best to use it with caution.

[@ranafge]

I am getting "no function or associated item named `for_db` found for struct Session in the current scope

[@JeremyChone]

This code use a pretty old version on surrealDB. Things might have changed. I would need to do another one. They now have a doc about embedded surrealDB.

[@ericrodriguez3580]

Jeremy, thank you for the AWESOME content, you have been immensely helpful in my journey to learn rust! I've been trying to use the embedded DB file in Tauri apps, but I keep encountering the following error: error: failed to run custom build command for `librocksdb-sys v0.8.0+7.4.4`, I'm curious if you experienced the same issue with awesome-app or found a solution? Thanks again for the great videos, very cool to see someone using the rust driver for SurrealDB.

[@JeremyChone]

I have not seen this specific one. For the filestore, SurrealDB uses RocksDB which introduces C++ dependencies. Perhaps you do not have the necessary C++ environment. I am not sure. Try to update your Xcode or windows C++ environments. Also, you can go to the SurrealDB discord and ask, they are very helpful. Feel free to let me/us know the solution here. I am curious.

[@ericrodriguez3580]

Thanks for the response! I've had no issue with the filestore outside of Tauri apps, which is what made me curious about what exactly is going on for me there. I'll continue to dig and post in case anyone else is experiencing the same issue. For context, I'm on a mac M1, which I suspect might be causing an issue with the default Tauri dependencies?

[@JeremyChone]

I have a m1 too. I did encounter some weird issue with Tauri at some point, but not this one specifically. What helped, if I can remember, is to do a cd src-tauri and do a first cargo build there.

[@ericrodriguez3580]

That actually got it working for me! Ran cargo build in src-tauri first, then npm run tauri dev from the root directory, thanks again!

[@JeremyChone]

@@ericrodriguez3580 Cool!! I had a similar weird case as well, but it seems to stop at the "surf" dependencies which is also C++ I think. This is why, "awesome-app dev" command line does a cargo build first from src-tauri. Happy all is working for you! Happy coding!

[@maxali6591]

Hi Jeremy, great tutorial again, I plan to use surreal for production for an app I am coding. Is it OK for prod?

[@JeremyChone]

For embedded databases, I believe it is likely an excellent solution. However, for a main cloud production database, it hasn't been as extensively battle-tested as Postgres. Therefore, for my primary cloud database, I will stick with Postgres, using AWS RDS for production. I may consider using SurrealDB for cloud edge/pod transient databases if and when necessary.

[@maxali6591]

OK, the reason why I asked this is because I saw how to use it over a tidb cluster. I succeeded last night by the way and it has great performance. I'll make connectors for postgres and surreal db. I want a 100% rust application 😊

[@JeremyChone]

@@maxali6591 This sounds cool. But TiDB is a C++ dependencies, right? (I am not familiar with TiDB)

[@maxali6591]

In fact, tidb isn't in rust. I wish surreal would be with the same functionality (why it shouldn't be after all 😊). It's effectively not 100% rust for the moment. But I'm confident surreal will be distributed 😊

[@maxali6591]

It is in golang, nobody's perfect 😂

[@hitmanpete1]

Thanks for the video! Im only just getting into rust so forgive me for the question here but if you want to take a whole result of task (being title and priority) and put it back into a rust struct how would you go about that? As when using into_iter_objects method you are able to get a single property of the object, would you have to use .get.map for each property of the object manually here?

[@JeremyChone]

Sorry for the late answer. Here is the way I would do this (pseudo code): 1. impl TryFrom for Task { ... } 2. into_iter_objects(..).map(|o| Task::try_from(o)).collect()... Little trick let tasks: Result = into_iter_objects(..).map(|o| Task::try_from(o)).collect(); This way, the Rus compiler knows you want to collect to a Result

[@revdznet]

Hi Jeremy , i kinda did what you did before watching your video , but i'm still a little bit confused because of the execute method return type . the result part has an array of array which has an iterator of an object of an object , and even when you first printed the object , you got some weird long type wraping the id . can you please explain to me how to wrap my head around that? thanks for the video is was educational .

[@JeremyChone]

Sorry for the delay. Good question(s). Here are my answers. 1) long id thingy - SurrealDB ids are special composite ids with the table_name:unique_id (assuming you do not provide the id). I think their design is to be "distributed ready" by default, and adding the table name part of the id allows to encode the FK into the value (which has its pros/cons) 2) Rust arrays of arrays of arrays of arrays - Yes, this can be cumbersome at first, but nothing a nice little crate API cannot clean up. I think their design was to have a flexible generic return type for eventual batch queries. Personally, I am ok having core APIs focus on completeness over convenience, as the convenience can be resolved at the application/library level. This is especially true with Rust, as we can add implementation to external types with crate/lib traits. Let me know if I missed something in your quetions. P.S. I deleted the duplicated question you had on this video.

[@peternierop3241]

I love to learn, could you share why at 3:57 line 10 is not let ses:&Session = &Session::for_db("my_ns","my_db");

[@JeremyChone]

Yes, I could have done a: let ses = &Session::for_db("my_ns", "my_db"); But I did the DB Type alias more to pass only one arg to the create_task and not just to remove the & on the "ses." I did not explain that part well. But yes, your way works too.

[@saefulrahman3478]

Hi, great tutorial, do you have any examples using embedded surrealDB with rockDB

[@JeremyChone]

I updated the code to 1.0.0-beta.9, I will add comments to show how to connect to localfile/db.

[@saefulrahman3478]

@@JeremyChone wow thanks,

[@raphaeljaggerd3585]

What exaclty do you mean by embedded? Does this mean it can be built with my app into a single executable like PocketBase?

[@JeremyChone]

Yes, that the database is part of the binary, so, no external process to manage the database files. Like sqlite.

[@raphaeljaggerd3585]

Brilliant. Thank you very much.@@JeremyChone

[@David-iq1kd]

Did something change where the "Datastore" type was removed from SurrealDB? It doesn't appear as an option. 5 | use surrealdb::{Datastore, Response, Session}; | ^^^^^^^^^ ^^^^^^^ no `Session` in the root | | | no `Datastore` in the root | = help: consider importing this struct instead: surrealdb::kvs::Datastore = help: consider importing this struct instead: surrealdb::dbs::Session

[@JeremyChone]

Yes, it moves module in the latest surrealdb beta. The GitHub repo linked in the description has been updated. Here the lathe commit for the update: github.com/jeremychone-channel/rust-surrealdb-intro/commit/40bc45f34b7bae10b9016039a7685b24533e7554

[@ishi92]

What do you think about sled?

[@JeremyChone]

Disclaimer: I have only done some quick experiments with Sled. Sled seems to be a great technology and should be considered if a simpler key/value store suffices. It's a lower-level library, but the benefit is that it is pure Rust and should be smaller. My 2 cents is that if a simple key/value store is enough, Sled might be good, but if a fully model store is needed, with relationships and all, then SurrealDB might be a better fit. One little note is that Sled's last update was a year ago. I am not saying it is a stopper, but something to be watched. Interestingly SurrealDB uses RocksDB as the persistent key/value store implemented with C++. I am sure they looked at Sled as well, and I would imagine they had some reasons for not going pure Rust.

[@ishi92]

@@JeremyChone Rocks is a pretty large dependency but it seems to have a much more complete feature set while Sled is smaller and faster with just basic kV store. I hadn't noticed that Sled's lack of update and hope it is because it is stable. Thanks for the content!

[@David-iq1kd]

no such command `watch` = requires another crate.

[@JeremyChone]

Ho, yes, requires “cargo install cargo-watch” I will add this popup to next videos.

[@GlobalYoung7]

thank you 👍😄🦀

[@HagbardCelineOnRust]

ty for the vid. will start my rust year in 2023, lf to surealDB

[@RobertoOrtis]

Good stuff!! How can I do something like this: "SELECT $fields FROM $th";

[@cthutu]

Same way he did in the video but you add two entries instead: one for "fields" and one for "th"

[@noblenetdk]

otherwise awesome explanation!

[@nocturnalcook]

#![allow(unused)] use anyhow::{anyhow, Result}; use std::collections::BTreeMap; use surrealdb::dbs::{Response, Session}; use surrealdb::kvs::Datastore; use surrealdb::sql::{thing, Datetime, Object, Thing, Value}; #[tokio::main] async fn main() -> Result { let ds = Datastore::new("memory").await?; let ses = Session::for_db("my_ns", "my_db"); // create let sql = "CREATE task:1 SET title = 'Task 01', priority = 10"; Ok(()) } Error: There was a problem with the underlying datastore: Unable to load the specified datastore

[@JeremyChone]

The latest surrealdb moved the Datastore to another package. Look at the latest main branch.

[@rakyatsemesta1945]

hey, i encounter with this problem too , i just need to add { features = "kv-mem" } in cargo.toml file

[@rakyatsemesta1945]

[dependencies] surrealdb = {version = "1.0.0-beta.9+20230402", default-features=false, features = ['kv-mem','kv-rocksdb']}

[@rakyatsemesta1945]

this is the script i add in cargo file

[@max_ishere]

Oh god is SQL nooooo

[@cthutu]

It's not SQL, it's SurrealQL - different :)