How Wagtail CMS StreamFields are Saved in the Database (behind the scenes)

[@23TrevorE]

What a great video in a great series. That little bit more understanding of what's happening under the hood makes all the difference. Thank you for taking the time to educate us with these superb videos.

[@CodingForEverybody]

I'm happy to hear you're enjoying this series Trevor! Don't forget we have an awesome slack channel too at wagtail.io/slack - Kalob

[@code-snippets]

This course is amazing 💯

[@Lumpiness9149]

Thank you for your work

[@sakyaris2458]

Thank you very much! I think Wagtail has a huge potential.

[@CodingForEverybody]

I agree, HUGE potential! Big names like Google, Apple, NHS, Mozilla and NASA are already using it.

[@unperrier5998]

Interesting, thanks. How does that work with searching? Normally you can specify which fields are searchable and which aren't. Doest that mean that if you search using say "type" or "value", you'll match all StreamFields even if none of the actual content fields contain any of those 2 words? It would be nice if it used a JSON column type (where available) but that's probably an ORM limitation. The standard way of implementing this would be to create multiple tables, one for each type of field (richtext, cta, cards...) and we'd have multiple rows (one for each field) and even a bridging table like we havein many-to-many relationships, but that means a whole lot more dB accesses, so the performance trade-off seems good to me: only one dB access then interpret some JSON and voilà. The only issue I see is that of searching.

[@CodingForEverybody]

This is a good point. And likely the reason that services like elastic search exist. If you used something like elastic you wouldnt have to worry about querying the JSONified streamfields. For the "Type" or "Value" query, it won't perform a fulltext search on the JSON as a string, it'll likely look through the JSON as a dictionary. An example would be: learnwagtail.com/search/?q=value - only 2 results (as of writing this). But every page on this site has a streamfield.

[@unperrier5998]

@@CodingForEverybody on a side note I wanted to use wagtail but moved to django-cms in the end because of streamfields: if you modify the model, the json can't be migrated. But with django-cms everything is sound relational. So that makemigrations will always apply the change.

[@CodingForEverybody]

@@unperrier5998 Sorry to hear that. Migrations with StreamFields are a bit tricky to navigate I completely agree with that. But at the end of the day it's just a text column with formatted json on it. You can write django data migrations to be applied before and after a migration file is run. To be honest, Ive almost never had to migrate streamfield data though - can I ask, what were you trying to do?

[@unperrier5998]

@@CodingForEverybody well during development you add and remove fields. to StreamFields, which makes the content in the dB is out of sync. Either you write some migration code like you said or you trash the dB. During the dev phase that's fine to restart from zero, but if I were to add a field in production I prefer to rely on tested migration automation (django, south) using a CMS like django-cms designed to natively support such migration rather than write some migration code. Also if it's really treated as a TEXT column then the only search available is textual and we lose the power of relational dB.

[@unperrier5998]

Another reason why I want to use django-cms (sorry for meandering, but I think you asked why) is because I want the non tech savvy editors to easily edit fields and I find django -cms awesome in that regard. Especially the personalisation: you can very easily set visitor or authenticated and then instantly view the content and more importantly edit it with a double click. And this can happen without any coding on my part :)

[@AntonZap]

thank you

[@IllevensKO]

it would've been extra helpful to mention how to access {{ blocks.first }} - for cases like displaying truncated richtext streamfield description of a post on a listing page. Adding a separate field on top of the streamfields would be confusing for admin.