How do NoSQL databases work? Simply Explained!

NoSQL databases power some of the biggest sites. They're fast and super scalable but how do they work?
Behind-the-scenes, they use a keyspace to distribute your data across multiple servers or partitions. This allows them to scale horizontally across many thousand servers.
NoSQL databases can operate in multiple modes: as key-value store, document store or wide column store.
You can run your own NoSQL database with software like Cassandra, CouchDB, MongoDB or Scylla. You can also use a cloud version like AWS DynamoDB, Google Cloud BigTable or Azure CosmosDB.
💌 Sign up for Simply Explained Newsletter:
newsletter.sim...
Monthly newsletter with cool stuff I found on the internet (related to science, technology, biology, and other nerdy things)! No spam. Ever. Promise!
🌍 Follow me
Twitter: / savjee
Instagram: / simplyexplained_com
TikTok: / simplyexplained_com
Website: simplyexplaine...
❤️ Become a Simply Explained member: / @simplyexplained
📚 Sources used to make this video:
savjee.be/vide...
#database #aws #amazon #dynamodb #simplyexplained

[@fa6805]

In SQL, you can also partition databases and it's called sharding. You can also duplicate your databases and it's master and slave databases

[@jaeken]

Thats racist

[@asjadazeez]

Although your comment is a very good one its not liked by the author because your comments nullified the purpose of nosql

[@ArneChristianRosenfeldt]

@@googleJay Numbers? Also why is this racist? I am all for switching from blacklist to deny-list, because what has color to do with this. Ancient Rome had Master and slave. Maybe we should express that the "Master" does all the work here. So I would call them "active" database and "follower". With interrupt controllers and IDE drives it would be high priority, and low priority controller / drive or really primary and secondary because Interrupt Numbers and Drive Numbers count upwards.

[@MrStealthWarrior]

@@jaeken "Master" and a "Slave" has nothing to do with race to begin with. People in the past often enslaved people of the same race. And if someone had no bad feelings towards other races, but still enslaved someone - that person wasn't a racist.

[@currymagc]

Sharding is essential for large relational databases, but usually it's a pretty and expensive problem to solve that involves making strategic decisions based on your data. NoSQL dbs give that to you out of the box.

[@markrosenthal9108]

This video incorrectly conflates query language with storage architecture. These concepts are independent of each other. Columnar, distributed storage provides most of the horizontal scaling discussed in this video. In today's leading distributed/columnar databases, the data can be accessed with either SQL or NoSQL depending on the access requirements of the application problem you need to solve while realizing the performance benefits of horizontal scaling.

[@therealantiarchitect]

Cosmos DB even offers two storage modes. The default option is a typical KVS described here, while the analytic mode is columnar.

[@mmmMindo]

But does it scale?? /p laughts in old meme language

[@paulodpereira]

Relational databases can be way more efficient at acessing well structured data, as it is placed in predictable memory locations, and the relation keeping processing needed isn't comparable with the overhead a noSQL DB has at acessing data. The choice of SQL vs noSQL really depends on the specific application, one isn't better than the other.

[@crateer]

Yes, he mentioned that. You just repeated what he said lol

[@malikrahman8649]

Exactly. It's like when people ask what's the best programming language.

[@dragonhunter8807]

its a bit late after your post, but what are the resources the server needs to maintain the relationships? That should only be the case IF you make the relations clear to the SQL-Server right?

[@TheViperZed]

Actually no, this heavily depends on how you access the data in the database and how many branch and cache misses you have. If you regularly access a significant amount of the data that is arranged so that the other data to be accessed can be loaded with it into the cache then a normal SQL is going to be outperformed by orders of magnitude by the alternatives. There is actually an error in the basic explanation though, as the unique key in the first "wide scaling" example is formed by a hash function there is a relation between the unique key and the stored data, so it doesn't do away with relationships it just leans really heavily into a single one. What should really be explained here is how NoSQL databases break "Normal Forms" and atomicity of operations to improve performance.

[@2112jonr]

Absolutely. But you'll still get the religious nuts here claiming that database ''x' is the cure to all of the world's ills and a golden bullet for all of those hated platforms by Oracle, Microsoft and IBM that the majority of the world still run on. NoSQL is good for some things and crap at others. Just like relational databases. But the fanboiz will never have this.

[@skullteria]

2:50 is such a surprising random indian accent moment

[@khalafnasirov]

Twist: he watched an indian tutorial before making the video

[@nickpelov]

You can store key value in relational database. And you can partition relational database to multiple servers based on primary key or something else. Relational db can do everything nosql can, but not the other way around

[@ydvisual5530]

WRONG. Relational DB cannot be distributed across hundreds of servers.

[@garethhanby]

@@ydvisual5530 Yes, they can. You can hash and partition(shard) on a relational database.

[@nickpelov]

Also I gorgot to mention one thing. Let's say that you have billions and billions of records on 1000s of servers. What do you do with them? Pick one record by key? How often do you know the key of a record. Most often you pick the key from a search result list. When you go to online shop do you say "I want to buy products with skus 5, 7 and 25"? Not really. You filter by category, you search by keyword and you sort by price and thrn you pick products - none of these are possible with no-sql. no-sql could be useful in few specific cases, but its usage is very limited.

[@Meleeman011]

@@ydvisual5530 lol look up what a galera cluster is.

[@ydvisual5530]

Oops sorry guys, I was really drunk last night and was just trolling with that comment. haha sorry. I looked at this morning and laughed. sorry about that. I dont know much about what I am talking about anyways haha. I have recently been working with oracles OBIEE. Its quite powerful. anyways, sorry about that !

[@powerpc6037]

In MySQL, you don't HAVE to use the relational structuring of your data. you can simply have multiple tables and treat them as truely separate tables where you run multiple queries to get all your data, each query runs on one table for example. Then you can kame the relations in your program if you need it.

[@srinivasreddyt9555]

Well, that is the point of the relational model. relationships enforcement is optional. tables are just tuples of data.

[@delturge]

What almost everyone seems to be leaving out is that arrangement of data and retrieval is not the whole story. Normalizing data makes it quite logical when you want to analyze it later. SQL is useful that way.

[@Me__Myself__and__I]

@@delturge Very true. Doing RDBMS well means actually thinking about and planning the data structure. This leads to better architecture and designs and makes using that data for other purposes, including adding new features, easier. NoSQL tends to be a lot messier - kinda like spheghetti code. Sure, I'm sure some devs do a great job with NoSQL. But MOST just see it as an excuse so they don't have to bother learning or thinking about databases.

[@guytech7310]

@@Me__Myself__and__I Issue is performance & very large tables that have billions of records. SQL coding is far, far easier than NoSQL since noSQL code has to handle any relational lookups or decoding the value (ie when it contains a structure with multiple fields).

[@jackpatteeuw9244]

"EVERYTHING OLD IS NEW AGAIN !" What you are describing is also called an Indexed Sequential Access Method (ISAM) database/file. I believe it was invented by IBM in the 60s or 70s. This was long before relational database existed. Other types of databases existed but they were very complex and hard to program and maintain. In those days "management" want reports about what was n these data files. IBM invented software to "query" these file called Report Program Generator (RPG). It was easy to use and could be setup to do various queries quickly. It was supperseeded by RPG-II that lived on for many, many years. Digital Equipment Corporation (DEC) had their own version that came out in the 70s on PDP-11 and VAX computer systems. It had its own querey/report generator language cal Datarieve. Datatieve was a bit query as it was both an interactive as well "fixed" report generator.

[@jackpatteeuw9244]

I am more familiar with the DEC implementation of ISAM. First, unlike many modern implementations, the "partitions" lived within the same file. Accessing any file (other than direct maping) was through a layer called Record Management Services (RMS). This was mostly invisible to the programmer/end user, but it did allow simple sequential access to ISAM files for any utility/program without having to understand how the data was actually organized on disk. The "key partition" was "hidden" when using sequential read/writes. While a unique primary was required, you could have multiple secondary keys and these did not have to be unique. Accessing a record via the primary or secondary key was fast ! The key "definition" (meta-data) was actually stored within the file. It could be accessed using a utility called the File Definition Language (FDL). FDL was also used to define/create a new file with different keys. If you wanted to add a key to an exiting ISAM it was really quite easy using Datatrieve. You would define the new file using FDL, use Datarieve to read the old file and populate the new file. Large ISAM file did require some maintenance for optimal results. Tools were supplied to do this.

[@robertboudrie2234]

Although, curiously, the default storage engine for MySQL is "MyISAM", though InnoDB does a much better job of implementing SQL principles like ACID.

[@JimAllen-Persona]

When I think of ISAM, I think of spinning tape. But it's not indexed, the PK is hashed. If you that, go back to IDMS... that would make me very happy. All calls will be to a Calc. I was an IDMS DBA for a long time and it was a pretty simple system to manage and navigate if you undetstood pointers. No.. we wants RDBMS and huge cartesian products.

[@jackpatteeuw9244]

@@JimAllen-Persona - Datatrieve was DECs answer to RPG. It was very much a "Swiss Army knife" ! It could do interactive queries or be programmed to generate "reports". It also had a callable interface. Last it could be extended by writing your own "functions" and link them into the product. One of the coolest features is it could "search" flat (plain, sequential) files as if they were ISAM. Slow, but it worked ! Last, using the DEC Convert utility and its companion File Definition Language you could quickly load an ISAM file OR add another key. I could also be used with DECs relational data base (Rdb) and hierarchical data base by only changing the file description. Sadly, very few people purchased the product and even fewer people used ISAM from any of the multitude of high level languages.

[@mark9118]

Teradata (since the 1980's) and IBM's DB2 Parallel Edition (since the mid-1990's) are examples of SQL databases that use sharding to scale horizontally in a linear fashion. Although both of these typically have more than just a key and a single value, they could be used with only one value (or one value in XML format) just like NoSQL databases, but they can also have multiple values. The problem with Teradata and DB2 Parallel Edition is that the cost of license is significantly higher than NoSQL databases, which are typically open-source software with only support fee costs.

[@wpyoga]

Keep in mind that the most popular NoSQL database, which is MongoDB, is not Open Source. It is free to use, which does help with its popularity somewhat.

[@venugopal-mr4nr]

Yes, structured data can be stored and maintained with XML format like restricted NoSQL value column. But to get the XML data into structured data takes lot of memory doesn't it? I think that's why it is effective. Moreover, If we want to just access the data, NoSQL is faster, cheaper and needs less maintanence

[@alexandermelchers1497]

Thanks for this great simple and short explanation of data storage in NoSQL-databases. Exactly what I was looking for 🙂

[@WayneAtkinsonEntrepreneur]

I have been doing this for over 30 years with relational db's. We have never used fixed foreign keys (db managed ones), but always done this ourselves inside the code. This means our relational db's have been stand alone tables (with indexing) since the start and all queries we do just link the keys to get the data. Still scaling is an issue, but we can split chop, remove data into archives/servers at will and the db doesnt break.

[@disgruntledtoons]

Foreign keys are one of those things that SQL offers to maintain data integrity, but they have a performance cost. They are best used in development and test so that when the code violates data integrity, there is a noisy failure; but once the code is working properly, it should be left out of the production environment.

[@merlinjim]

@3:30 a hash is used rather than just the primary key's range directly because the hash will always partition evenly (i.e. the magnitude of the computed hash is randomly spread throughout the range)

[@jimmiller5891]

If it is a good hash.

[@mikeofallon]

True - This worked very well in the mid-1970's when computer resources were extremely limited / expensive. I designed a simple hash within a custom access method for storing & retrieving all medical & dental claims history, as part of a software package for our client, for Equitable Ins. The file size was 16X IBM's max at that time, even after data compression & spanning multiple hard drives! So part of the hash was which of the 16 physical files the data was in. In testing we achieved an average of 1.07 physical (system level) read requests per logical application request -- before a cache was added! Code was also tight -- about 50 Assembly Language instructions to calc physical record # (or block), and about 70 AL / machine instructions within the IBM access method (BDAM) to convert our record # to CHR (Cylinder / Head / Record). This was still in use in the 1990's.

[@robertmainville4881]

Ouf... There are so many debatable stuff in there, I don't know where to start. Let me just say that NoSQL databases have lot more disadvantages than what is explained here. The main one being that they are very slow in input (write) operations, making them unusable for most real life use - no supermarket chain or telecomm company would use a NoSQL database to capture real time transactions. Their market acceptance, which is still very low, testifies that NoSQL databases are a niche product.

[@livelaurent]

Yes, I agree with that. This video is actually very misleading. And people upvoting it should really spend more time understanding the difference between the two models. This video basically shows "problems" with SQL database versus "advantages" of No_SQL database... Like Robert said, there are WAY more disadvantages to the NO-SQL database and you will face a lot of different issues with different problems that SQL database can solve.

[@deedoodeedoo6382]

@@livelaurent Same feeling. I watched it to the end and was like "well? what are the drawbacks of NoSQL? Everyone would have used them if there were none"

[@ArneChristianRosenfeldt]

Apache Kafka

[@robertmainville4881]

@@ArneChristianRosenfeldt It's not a database; Kafka is essentially a pub / sub framework.

[@glowiever]

THIS. the hesitance of adopting NoSql is not without a reason.

[@LandNfan]

There has been a style of database around since the late 1960’s that offers the best of both, plus many features that both SQL and NoSQL lack. It’s Pick and it’s many variants and descendants. Hashed, variable length, delimited, and it’s native programming language is a high-octane version of that old favorite, BASIC. It may not scale to Amazonian proportions, but it is great for most of the real world. The company I worked for started using it in 1996. By the time I retired in 2009, they had grown to over $100 million in annual sales and are still thriving on Pick clone Unidata.

[@ellisz5972]

I feel like NoSQL is what you get when a coder creates a database that never learned to clean up his room or organize anything.

[@oliveryt7168]

It's basically a stack made of everything XD

[@madman4043]

I started watching, thinking "what the hell is NoSQL?". The further I went, the more I was just thinking "holy shit why"

[@Me__Myself__and__I]

That is EXACTLY true. NoSQL is ideal for certain limited limited use cases, but it is very popular because lazy programmers don't need to bother learning SQL or good database design. And since NoSQL is so "flexible" these same lazy developers often end up storing JSON that is inconsistent over time so incredibly hard to work with and maintain long-term.

[@evergreen-]

Me myself I, so you wanna say that all tech giants like Apple, Amazon, Facebook, etc use noSQL just cuz their engineers “are lazy and never bothered to learn SQL” xD. You are so delusional, it’s actually scary

[@Me__Myself__and__I]

​@@evergreen- No, I'm the rare developer with extensive experience in both dev & databases/DBA. Plus, apparently you can't read. I said there are some non-typical cases where nosql is a good choice. But it is a bad choice for the majority of typical business systems. Also, anyone who confuses mega-scale projects at huge companies like Amazon, Google, Facebook, etc. with the work that 95% of devs do is delusional.

[@matthewbaynham6286]

So relational databases are for developers who design the system they are working on, and NoSQL databases are for developers who just bang away at their keyboard without any thinking.

[@gavintantleff]

I think you’re oversimplifying the problem. Unless you didn’t finish the video, then you should’ve seen that there were clear advantages and weaknesses of both relational and nosql databases. NoSQL databases are great at sharding and horizontally scaling, which relational databases tend to suffer with. Depending on your product, key value pairs might make more sense to represent your data as well. It’s all about tradeoffs and we shouldn’t oversimplify things like this. There are good and bad reasons to use both.

[@gerar10s]

Me: *watch this video then go to google* "What database does Facebook Use?". Google: "MySQL". Me: "Yep I'm good with relational databases then."

[@AlbertusVanSchalkwyk]

2 & a half minutes, and I know what NoSql is. Thanks video creator.

[@Me__Myself__and__I]

The comments on this video make me very happy. Lots of people calling out the inaccuracies and bullshit in this video. Shows that there are still many people out there who actually understand databases and tradeoffs.

[@tomahawk5357]

The video is actually very good in explaining NoSQL databases. I like SQL databases and after watching this video I still prefer SQL databases and don't see many advantages using NoSQL but many drawbacks.

[@sguillory6]

Hate to disappoint you, but most of the bullshit in this comment thread is coming from the RDBMS proponents. It's pretty clear that none of they folks have actually worked with a NoSQL database in recent times have any idea why they exist.

[@Me__Myself__and__I]

@@sguillory6 So enlighten us, why do they exist? I mean, other than letting programmers be lazy so they don't have to bother learning things like proper database design or bother thinking about things like performance.

[@eduardovalentim1386]

Was looking for an explanation regarding NoSQL and this was the best one I've found. Straight to the point and complete. Congrats! Subscribed and looking for more videos in your channel :)

[@mauricenpa9292]

It would be helpful to see example queries (and the syntax) made against a NoSQL database. I'm finding it hard to wrap my head around how that would work in such a design.

[@jeschinstad]

In CouchDB, you simply write a JavaScript function and tell it what to return. Couch will then run your documents through that function and generate an index. Each time you write a document, it is passed to that function and the view index is updated with the new values. It makes writing fairly slow, but reading extremely efficient. It also means that if you have a very large dataset, then adding a view can take a long time. But on the other hand, you don't need to know in advance what kind of combinations of data you may wish to retrieve in the future; you'll just write another function.

[@max-tp1xv]

@@jeschinstad thank you

[@AJDF24]

NoSQL Databases don’t have a specific syntax that’s the point!

[@hil449]

@@AJDF24 Oof. Isn't that bad though? One of the best things of SQL is that it's a very old language that still works for multiple different dbms

[@AJDF24]

@@hil449 no, why should this be bad?

[@RogerValor]

just as nosql servers might have a sql layer, so have sql servers like postgres already the ability to partition or have dynamic data in json blobs (including a subset of sql to query it). not sure how powerful that is however compared. also, your queries only double if you keep it in the pk range; otherwise they dont as each partition has to perform your query, i would suppose

[@bazoo513]

About NoSQL consistency: the problem is not "eventual consistency" of mirrors or replicas - relational databases also have mirrors, replicas, stand-by copies and whatnot. The problem is that _logical_ consistency between different pieces of data is now completely applications' responsibility, because there are no referential constraints, chack constraints and other consistency checks performed by the database server itself. Several more questions worth exploring: - How do NoSQL databases handle atomicity of complex transactions, various isolation levels etc? Is the answer "they don't, it is too resource-intensive", or something else? - How do NoSQL databases (I would prefer to call them data stores) compare to ancient index-sequential files? - How do schema-less databases handle update-intensive loads? - How do they compare to schema-constrained, but very fast "index plus physical address" database systems such as ancient CODASYL? I have an impression we are reinventing the wheel, yet again.

[@Oxalis11]

Have been going through KZbin, watching everything to try to understand the topic. You are the first one to succeed. Brilliant explanation. Thank you!

[@AjitV]

Precise . Illustrations at every point and even someone without DB experience can understand easily. As the name suggests ... Simply Explained.

[@inventoresanonimos]

This video is so good that I have watched a few times and also referenced it during my paper in college.

[@dandumitrescuLXXXVII]

Concise and interesting, thanks!

[@ArneChristianRosenfeldt]

So I am at 1:37 and "not only SQL" was not yet mentioned. This picture of the buildings reminds me of the " New Testament ". How am I supposes to follow this metaphor? I mean, network hardware was insanely overpriced for some time, but today? Mainframe CPU runs at 5 GHz, commodity CPU runs at 3 GHz. There is no "vertical" anymore. So your database uses a RAID -- uh that is horizontal right there.

[@CoentraDZ]

A lot of hate to NoSQL in comments 😂 I was looking for an efficient way to implement a shipping system, the problem was in storing some data and I found that using NoSQL is the easiest, fastest and efficient method to do that. SQL and NoSQL are both a "powerful tools" to achieve goals, stoo comparing them and enjoy them both.

[@antonioanerao]

Now I got it. I took me a while to understand how does the nosql works. I've been working with SQL Server and MySQL for a few years so the nosql idea had confusing my brain a little bit lol. Thanks

[@SimpleLivingHigherThinking]

Thank you for such a simple,clear and quick explanation of NoSQL😀

[@Dino7759]

You would need to hash to a pointer not the actual data itself, otherwise it would be too inefficient. Foreign Keys would need to be hashed that way as well, but you lose the neat splitting across partitions. Or else then you would have to duplicate data.... On top of all that you would need some conflict resolution on the hashing function for keys that produce the same hash result. So it is not quite so simple. All in all it is all swings and roundabouts. You need to look at various combinations of warehousing options to suit your needs. As would be the case for all applications, you need to look at the best alternatives (or combination of alternatives) to achieve the scale and performance to produce the desired results.

[@walterwhite6499]

a hash key collision sounds like a super rare occasion.

[@cigmorfil4101]

Pick works quite well with hashing to a group which then stores the whole of the data item, with multiple items in a group just stored one after the other in the group, linking extra frames from the overflow spare frames of disk space. A file with 23 groups is guaranteed to have hash collision with 24 items stored in it...

[@melihhopa2684]

Best video about noSQL vs SQL

[@rafaelmarcos9733]

That's what I'm looking for, I want to store non consistent entities in my database, I'd have to make many relationships between tables and many queries to construct the object of an entity, not to mention the complexity. Thaks a lot, now I know how to make the things easier.

[@downburst3236]

You can easily design software to scale SQL over multiple servers. Have customers in one DB, products on another, and orders on another. The advantage is easy searching.

[@chunkieliest]

the moment I realized this is animated in ms powerpoint completely threw me off haha. great explanation!

[@BharathRamMS]

Thank you very much. I was struggling to understand NoSQL, and this video not only helped me understand it but also answered several questions I had.

[@kasuha]

Looks like another example of a question with a simple and easily understandable wrong answer.

[@RadenVijaya]

Ive been working with this NoSQL since 2007 and completely happy about it. However I actually used it on an MySQL database engine, because back then NoSQL DB is not yet existed. Kinda crazy to think of it, eh ...

[@synthwave7]

Very nicely explained - I learned a lot as IT admin.

[@oleksovd]

Mongodb can filter by field and also by nested fields, has aggregation pipelines. Has fast writes due to LSM tree.

[@carlfranz6805]

Well... that may have been the most clear, if not most complete, explanation I've heard.

[@aroonsubway2079]

Is there anything important that he didn't cover in this video? I was planning to prepare for some interviews based on this one : )

[@henryseldon6077]

The MUMPS database provides all of this and has since 1966. Easily, the most flexible database I've ever worked with.

[@cigmorfil4101]

The easiest I've used (and still do for my home stuff) is Pick of the same era.

[@swapode]

The best comment I've ever heard in regards to MUMPS is that it should've been called after a worse disease instead. Think of any bad decision in software development, MUMPS probably has its own version that's somehow even worse. And that's before developers, deranged enough to voluntarily touch it, got their hands on it and applied it to the real world. Luckily it's just used for unimportant data like medical records. ;-)

[@henryseldon6077]

@@swapode I'm talking about the mumps database, not the mumps language.

[@stevenpaul4394]

Putting our time and effort in activities and investments that will yield a profitable return in the future is what we should be aiming for. Success depends on the actions or steps you take to achieve it.

[@adamnathan6744]

My advice is always to keep the majority in ETFs that cover multiple sectors. Then have a couple stocks/cryptocurrency, as a small percentage

[@ashleykimberly4285]

18

[@ashleykimberly4285]

36

[@raysunqi]

Great explanations, hope you can provide some examples of when to use Sql and NoSql.

[@urilou777]

i think nosql is only for gafams really, perhaps scientist with huge databases trying to train IAs also, things like that but other people I'm pretty sure don't deal with that much data to justify it

[@tiofies]

So simply explained that I understood it! Many thanks ;)

[@coolhanmant]

Oracle db can scale both horizontally and vertically

[@gisdevops7665]

really like your videos, I have a request that make a video on OOP or Polymorphysum specially

[@jackpatteeuw9244]

I thing that was "glossed over" was the initial size of the "key partition" which of course is related to the algorithm used to create the key based on the data field. If the key was too small, a new record could create a "collision" and the key would have to be split, effectively making it a sequential search. If the key was too large, there would be a lot of wasted space in the key partition.

[@awesh1986]

Nice explanation can you share some practical videos where you are demonstrating the use of these NO SQL tools

[@Patrick_McFadin]

One small correction. Apache Cassandra is a full schema database with tables, rows and typed columns. Nice overview otherwise.

[@simplyexplained]

True, Cassandra is a weird mix, but it's still considered to be a NoSQL database. It partitions data across many servers and it can't run queries against any column (must include key or you need indexes. This is also the case for other NoSQL databases). On top of that: the database schema is still very flexible. One row can have more columns that the rest of the items in the table.

[@sriramakrishnabattu3408]

Simple and straight forward explanation of NoSql. Looking forward to more videos...

[@jackt6112]

A few questions: - "Finding orders by ID is no problem." What do you mean by ID? Do you mean primary key value, a relational database's primary key, or an ID field in the NoSQL database? - If you mean by an ID field in the NoSQL database, I would think you would need to query all of the servers with the same query unless NoSQL indexes the fields you tell it. - I would think ACID compliance would be a challenge.

[@zaheerbeg4810]

Precise, Simple Explanation, Adorable. Keep Posting

[@ericcheung8790]

Mostly talking NoSQL advantage is its horizontal scaling for the big data. If I need storage schema less data, but not a large data. Is NoSQL a prefer choice?

[@alberteinsteinthejew]

SQL cant be expanded horizontally, does it mean it cant be partitioned or cant be mirrored to multiple servers?

[@scottpowell9985]

I’m struggling to get how actual data would work in a NoSLQ database. For instance what would the orders example (order, customer, product) look like in a NoSQL database? And if NoSQL doesn’t guarantee that all items look alike aka have the same fields, how could you even run a query like “show all orders where price is over $100”? You don’t know that every order record even has a price value? And output columns wouldn’t line up because each “order” could have a completely different set of columns from the next one?

[@thebatprem9271]

In MongoDB, a document (aka a row in SQL) will be right {"order": 111, "customer": "xxx", "product": "yyy"}. As you mention, NoSQL doesn't guarantee the fields must be the same, there are two options for MongoDB developers. First, just remain them the same, when you query them with "price greater than $100", documents whose price doesn't exist will be ignored. However, if you sort documents by price ascendingly without any filter, the null price documents will be shown first, but you can fix it by adding filter {"price": {"exists": True}}. Second, in the case you really want MongoDB to guarantee the fields must the same, you may use the feature called "validation". It allows developers to control which fields are must-have and strict data types (number, string, etc.) for each field or which fields are free to insert.

[@pawemarsza9515]

@@thebatprem9271 or you can just use relational database

[@charleschang9520]

Instead of joining 3 tables with 1 where statement. It would 4 where statements and no joins. If you are not precise enough on your query you will get non relevant data.

[@zyansheep]

Nice video! Can you do a video about Kademlia distributed hash tables?

[@ravindranaths513]

Could you please make a video on Wide column vs column family vs columnar vs column oriented DBs with some examples?

[@raphi72]

The main issue with eventual consistency is the lack of record locking, which is very important if you handle value assets.

[@simplyexplained]

Maybe not. Strong consistency is supported by many NoSQL databases (but it comes at a small cost in terms of performance). Record locking could be implemented in the application layer, and I believe that some have built-in support for it (definitely DynamoDB).

[@raphi72]

@@simplyexplained It is impossible to have eventual consistent replicating nodes (a.k.a. "horizontal scaling") and record locking at the same time. NoSQL still is useful for many applications (unless you need this level of consistency).

[@stevo728822]

High value trading systems use dual transactional locking.

[@sguillory6]

@@raphi72 Nice try, thanks for playing. Dynamodb offers strong consistency at a cost. Every partition has a master node associated with is that receives the write before it is replicated. If you want to pay extra for a strongly consistent read, your query will be routed to this node. In fact, DynamoDB writes to two of the three nodes before returning to success, so even with an eventually consistent read, there is a 2/3 chance your data will actually be consistent.

[@raphi72]

@@sguillory6 I don't know Dynamodb. But you cannot take chances, if your consistent read is an asset amount.

[@netsurfers9357]

Azure can also use Kusto Query Language (KQL) and translate directly into Power M-Query into Applications like Power BI for Analysis. COSMOS is still used, but considered the old way of doing things.

[@BytebroUK]

In any 'sensible language', that would be a 'hash table'. Quote (from whom I cannot remember) but "If I had to live on a desert island and could take *one* data structure with me, it would be a hash table." Totally agree.

[@yani1307]

I have a question, when tha data stored in the data storage becomes large and humongous in the multiple servers, will there be any issues when the users try to collect data maybe from the data in the early servers.and when users update data, will it cause any problems?

[@muess]

Would adding a new server require redistributing the existing records based on their existing hash values? That's what it looks like from the video but it seems like it would be quite compute intensive. Extending the range of hash values and adding new records to the new server would seem to be more efficient - is that how it actually works?

[@JOHNDATGOONV2]

I've been using NoSql databases for a while but could never understand what the partition key was or how to use it, but your video has finally explained it, thanks!

[@brookeguthrie3417]

Thank You for providing Clarity.

[@aaryab7370]

At 6:01 the author says "in some time the new entry of data will be mirrored across the servers"; he is mentioning the "Eventually be Consistent" property of the BASE theorem.

[@sguillory6]

Note that some NoSQL engines, i.e. DynamoDB, actually write your data to two nodes before reporting success and then propogates the write to the third node.

[@himanshusaini5316]

That's literally the best ever video on explaining "NoSQL", so simply. Thanks! 💯

[@adaslesniak]

Correct me if I'm wrong but graph databases have no sql syntax, so they are not not-only-sql, but they are all about relations, and they belong to category of NoSql, so your last minute seems inaccurate. Other than that thanks for sharing. I wasn't aware about advantage of how easy it is to scale horizontal data base... However this still can be done with Sql databases... just not so easy.

[@alexandriashrake7548]

What a wonderfully simple take on the subject. Thanks for sharing and making!

[@willemachternaam690]

Not only simple, sometimes dead wrong and inaccurate. See comments

[@eduardoligeiro2817]

excellent, clear, highlighting and unveiling key examples, in a powerful and simple fashion

[@mac19999x]

Thanks bro, you've made it easy to understand.

[@GoluKidsLearning174]

It's a very nice video to better understand the NoSQL database concept.

[@jimmiller5891]

Nice video but left open the question where do you get the primary key and how do you retrieve for example the product by its id (like in the example). I get the point of NoSQL and don't belittle its value but its is a bit like Porsche 911 GT3, "we left out all the creature comforts to make it fast". If you buy a car without aircon you need to figure out some other way to stay cool and comfortable in the summer. Same with databases. There is no free lunch.

[@cigmorfil4101]

In Pick the primary (only) key is the Item ID (denoted as attribute (or field) 0). The item ID can be anything you like. When an item is stored in a file a hash function is performed on this item ID which is then used to determine into which group (partition) of the file to place the item. When accessing the data in a file, an attribute of the file can contain the item ID (primary key) of an item in another file. You can define "dictionary" items which can pull data from another file based on the item ID found in the given attribute of an item in the current file. Each Pick file is like a SQL table, except that no structure is imposed on each table (upto the programmer to ensure data integrity) and each attribute can be multivalue - eg an attribute of a customer can contain a list of their orders and when listing the customer file those orders can all be accessed from the single customer entry, pulling across the attributes as required (similar to, but not quite the same as a sql join).

[@jimmiller5891]

@@cigmorfil4101 Thanks, especially on behalf of those who did not know. My point was that this could have been explained in the video and that leaving out those SQL features trades functionality for scalability. Nothing wrong with that but it is a trade-off-

[@scottgaines2677]

Thank you for a fantastic overview! Appreciate it, well done!

[@lazolabucwa8821]

And just like that you gained a new subscriber. Thanks for the excellent video.

[@Khatarnak.Bakchodi]

Great explanation, got the gist in few minutes!

[@cheaterman49]

I smelled BS when they said relational can't scale out, only up. Glad comments confirmed before I wasted further time.

[@Coffee_is_ready]

Why BS. Relational database are indeed architected in a way that scaling out is difficult to do and not widely supported. As opposed to a distributed database where operations are designed to work in a distributed environment. The common RDBMS is not. I think the video is BS because it talks about NoSQL database as if there is one big category of them. Which is not. The only thing NoSQL database have in common is that they do not implement the relational model. Not all NoSQL scale out, not all NoSQL database don't use SQL, not all NoSQL databases work with semi-or unstructured data.

[@Davide73]

Totally right... But you know, nosql is cool, for smarter iGuy playing during working hour... But they are lucky their payroll is managed using an old-school database

[@henklieftink3357]

Thanks for this simple explanation.

[@creaturerohit]

Thanks for explaining .Simple and to the point .Subscribed !

[@Foba_Bett]

Very intuitive explanation - thank you!

[@DeCamJ]

Simple explanation, perfect.

[@nitikaraj4648]

precise & short - the perfect explaination

[@EiRA-hiE-x]

There's a thing call sharding in SQL servers lol , the NoSQL is still pretty nice to use

[@youvanced6593]

So how does rhe db onow whether it needs to teturn the Macbook or the hand sanitizer if they both have the same hash?

[@stevo728822]

NoSQL means the interpretation of the data has to be performed on the client computer. With SQL it can be performed on the server.

[@abhijithvm3166]

I think sql is the one of the best place for manage manipulate relational data base but Nosql database is one of the best place because compare to sql we can store store large set of data in easily also handle.

[@afzaalawan]

excellent understanding

[@thiemokellner1893]

Been working for 22 years + as DWH engineer with SQL databases. I have virtually no experience with NoSQL databases, but some suspicions it is a big hype. Usually, we do not have any foreign constraint implemented. Partitioning is old in SQL databases. Having a JSON document as value in a key - value store is not removing complexity at all. It is just hiding it. In your example at 2:05, if you want to get the price of the MacBook, your NoSQL DB would have to search all the JSON documents for the correct name to extract the price. Does that really scale if the documents grow in size and/or complexity? How much faster can you get than to have indexed data? No problem in SQL databases. Maybe your explanation of partitioning is just beyond me, but what is the use of partitioning by an artificial key? It will be seldomly queried. Sure, you can partition by a natural key, take name in the mentioned example, but then there come along queries that want to get a list of prices for laptops. Do NoSQL DBs query each and every node for it? Sure, nice parallelism if 75000 servers are queried. Does that mean, 75000 bare metal devices? Is schemalessness a real advantage, as described at 4:25? To me, it is just a procrastination of the complexity and effort. First, you just dump everything into some data swamp and then the user is to know how to fish for the information at computational costs, to me, look immense in comparison to a relational model, especially because the bringing data in to a relational model is done once when loading. The NoSQL approach, to me at least, seems to require the "modelling" with each query on the fly. I am still not convinced at all.

[@drew21t]

Typically the document isn't growing, you just have a lot of documents and you would use views to index them how you want, you can even create views arbitrarily depending on the NoSQL db you use. You have to think of them differently than you would a relational db. You're not looking for data, you're looking for documents, thats why they are often referred to as document stores rather than data stores. And because of that, when you run a query, you don't get back a piece of data, you get a whole document back. I'll give you an example. In my application, I use NoSQL (CouchDB), when a user wants to log in, they enter their info, either a UName and Pass or Email and Pass. In my "Accounts" database/document store, I hold user account documents that I've predesigned as a json doc, it holds all the information you would expect. username password name company info etc etc. When that user sends me a login request I query a view in the db that indexes all documents in the "Accounts" store by the most comin means of looking someone up, usually email, userid, and username. When that request hits, I go and get the entire document from the "Account" store and run validation in the background from that document. At that point I already have everything associated with that users account, from JWTs to arbitrary identifying information, so I can do a full suite of validation techniques without having to query more information from the db because I already have it all. If I want to get that users activity, I have that stored in another db called "Activity" which is another doc store that stores docs by the userID, and when I want their activity, I get that doc and it just has an array on it that holds other JSON objects representing their various activity. I have it all from two uncomplicated queries. What he doesn't tell you is this, you get the whole document, or none of it, and that means when you update you have to update the whole document, not part of it. This, to me, is where the real power of the NoSQL db comes in. Every update completely replaces the previous version of the document, but the db keeps track of this, to an extent. Meaning you have automatic version history for every change made to a document. Again to an extent, most dbs allow you to change these settings to keep more or less docs for longer or shorter periods of time. And this is a natural feature of these dbs. I know for sure mongo and Couch work this way, others may vary but as far as I recall they all have some form of this version history. So on each document you get a "Revision" number which is what you use when you actually go to post an update to a document. So you get the doc you want, it comes with all the data you designed in plus a Rev number, you make your changes and you post the doc back to the db with the Rev number it was on and it makes a new one. Depending on your settings it will track this for a few revisions and then start dumping out the older ones. This helps with replication, which is why I love NoSQL, spinning up clones to handle more requests is a breeze because of the automatic control of versioning. To answer your question about querying each node. The answer is no. When you request data, you have to know something about the data you want, usually the ID, it will then hash that and find the specific db its stored on and only make a request to that db. So no sequential searches which would be horribly slow. To answer the question about a list of prices, you would use a view for that, which is essentially a function you write that tells the db what you're looking for and it will create an index based on that, and when you run a query it will reference that index. So you would write a view that indexes things by price and query that view rather than querying the Product db directly and running a loop or something, which means you'd have to pull all the products to do it. That would be bad, but at least in the case of Mongo and CouchDB you don't have to do that. To your last point. When I first started learning about them, I thought the same thing, how do you even sift data, but, at least in the case of couchDB, they in the documentation tell you to think of the stores each as their own "Database". I have an "Accounts" db and a "Activity" db and each only holds those types of documents which I've designed in my application layer to be of a certain type and makeup. I didn't have to write out a schema for the documents that the db holds I just denoted that this db is where I'll store this type of document and then I just put them there. Then when I wanted to run more advanced queries on those documents, I wrote out some views for getting at the data. I could have had my application layer write those views out, but I opted just to write them into the CouchDB instance itself and I simply run the query with the key I'm looking for, which looks something like this. (I normally work in JS but working on a python application and just happened to have this view right here but the principle is the same) record = list(user_db.query('findUser/findUserByPhone', key = number)) The findUser part is a design document which holds views and the findUserByPhone is the view I wrote into CouchDB that creates an index of users by their phone number and the key is the number I'm looking for. It returns a list even if theres one hit, but its a list of json objects. Now with this query I'll get back something that looks like this. [{'id': '6f1e8976-7a7e-4592-9389-c4d28785098a', 'key': '5555550109', 'value': True}] So now I have the ID of the user, and I can then go and query the Account db for that users document and pull the whole thing. {'_id': '6f1e8976-7a7e-4592-9389-c4d28785098a', '_rev': '1-66091d150834f9b494c2f3d2a9e44760', 'created_date': '1711650186090408700', 'updated_date': '1711650186090408700', 'phone_number': '5555550109', 'firstname': 'Tron', 'lastname': 'Tastic', 'company_info': '', 'onboarding_complete': True, 'timezone': ''} Which looks like this on return. Just a JSON object. Now I get that is two queries, and in SQL you can use something arbitrary like a phone number to get a whole user row, but in NoSQL you don't want to index then entire document, the index only indexes the important portions of the document that you need, I use it to just get the ID of the document I need to go and get, but you can index the entire document, but if you have thousands or millions of users that will add up right quick. And even though this instance was two queries, it will still be way faster than an SQL query. Thats just a simple breakdown on why I love them, you may still prefer SQL and I can respect that, they are still powerful for a lot of use cases. But I don't think people should be sleeping on NoSQL, they have a lot to add. I think combining the two is probably the best use, but I wanted to prove that I could build any traditional application without SQL and now I've just come to prefer them over SQL. Regards

[@manilavaswani154]

Such a great explaination thank you so much..and its a request sir kindly make a vedio on what is data warehouse and what is data mining in detail.

[@oliveryt7168]

I'm happy to have learned that both concepts of DBs have their legitimacy.

[@MrJatan90]

Great explanation, thank you so much! After watching so many videos and getting thoroughly confused, you helped clear it up. Thanks again.

[@danm8642]

Here's what I don't understand. Worked with relational databases for years so I'm thinking I need to open my mind more. How do you retrieve data? Typically don't know a primary key to pull back the data. However, if you use a filter of something in the value field then it's slow.... Someone please enlighten me.

[@sguillory6]

Simple. You better know the primary key. It is a required argument to the Query API. Your alternative is to do a table scan which will be expensive on a large table.

[@nxtfenox1592]

This video is brutal, thanks brother!

[@RanjuRao]

Worth every minute watching this video :)

[@klam77]

Mercifully clear explanation! Thanks!

[@circonflexechess]

Thanks for explanation But : how a nosql DB is any different of a filesystem ?

[@suabiansouthern1764]

Wonderful & easy-to-understand video which really shows that you've mastered this area of technology.

[@venusumanthpotru5256]

Thank You for the awesome video, Very clear explanation regarding NoSQL.

[@arbobmehmood5831]

Most people probably don't know this, but Google Search, Google Maps and many other Google services runs on a NoSQL database. That's how scalable NoSQL is.

[@harishlodha8794]

Sir kya Google Play Store se video recorder app se video record karte hai kya vo video app developer ke pass bhi save hota hai kya Please reply.....

[@mattmcewan2550]

SQL Server has partitioning - enabling horizontal scaling.