3 Strategy Killers for Data Teams
6:10
What is Data Mesh?
4:28
Жыл бұрын
3 Ways to Deploy Data Projects
3:29
Пікірлер
@pranayreddy7685
@pranayreddy7685 17 сағат бұрын
can we reference the example sql model in snowflake_sample_data_store_sale.sql
@shanks-rf2xr
@shanks-rf2xr Күн бұрын
How to fix this issue? When I run dbt run, it stuck. Here is debug status log when it stuck. 20:20:34 Opening a new connection, currently in state init
@WingYeeLee-b2x
@WingYeeLee-b2x 3 күн бұрын
Very well explained, thank you. Often when I read the official documentation, even they are probably clearly written already, it doesn't sink in until someone explains it slightly differently. Cheers!
@ksaha6387
@ksaha6387 4 күн бұрын
How come Spark not mentioned ?
@OZTutoh
@OZTutoh 6 күн бұрын
I came here hoping to understand what this $300 of free credit means. Looks like the search continues.
@camus83489
@camus83489 7 күн бұрын
perfect explanation, thanks so much
@bubu60112
@bubu60112 9 күн бұрын
what a fantastic video. well put together.
@daqa290885
@daqa290885 9 күн бұрын
Thanks for the explantion, I know the tutorial is from three years ago, but it's very usefull, only the actual version for fivetran is little different, but is easy to understand and create transformation using dbt. Thanks and great video
@lukasjerabek2504
@lukasjerabek2504 10 күн бұрын
Outdated :( prefect changed a lot, no deployment build or deployment apply commands any more...
@cs-gl2dm
@cs-gl2dm 10 күн бұрын
HI, I searched many places to find the automation script documentation for release. I couldn't find. Can you please share that?
@harishprajapat4217
@harishprajapat4217 13 күн бұрын
Hey bro , i followed your every steps carefully but at the end my vs code isn't got updated on GitHub there is showing index.html file but not code inside it , can u pls help me out for this
@foodie_nextdoor0
@foodie_nextdoor0 13 күн бұрын
Great video ,but can we configure it in such a way that the approver get an email notification that a change has been made and make comments during the review for the user to action?
@cs-gl2dm
@cs-gl2dm 14 күн бұрын
Good work
@davidporter575
@davidporter575 15 күн бұрын
I think ETL is the way to go you need to know what you or need before adding to a permanent source.
@technicaltalk8833
@technicaltalk8833 16 күн бұрын
Hi ,I am currently working in Technical Operations support role which I am doing since 1.5 years mostly use MySQL and linux on daily basis, now wanted to transition to Data Engineer role, but most org are not having entry level job roles for this, so should I first try for data analyst role and then being working as data analyst post that should try for data engineer?? What would u suggest
@stevenshrii
@stevenshrii 17 күн бұрын
Can you have nested environments?
@MaBuSt
@MaBuSt 22 күн бұрын
I saw data mesh fully implemented at a company and the variance in best practices on the ETL pipelines was enormous. Politics got super weird as some teams were known to set up better pipelines than others. People were asked to cross-train but politics and other incentives got in the way from that really working… and I think that those are going to happen basically everywhere (especially the divergence in best practices).
@cs-gl2dm
@cs-gl2dm 23 күн бұрын
Thanks a lot
@cs-gl2dm
@cs-gl2dm 23 күн бұрын
Awesome
@quarteratom
@quarteratom 25 күн бұрын
It is difficult to overstate how counterintuitive YAML’s decision to rely on spaces for indentation is, especially when considering the longstanding benefits of tabs in programming and data formats. While YAML was designed to be a human-readable data serialization format, the choice to use spaces for indentation introduces significant practical drawbacks that undermine its usability and efficiency. First, the use of spaces is inherently error-prone. In YAML, the hierarchical structure of the data is defined by indentation, and a single misplaced space can completely alter the way the document is interpreted. This creates an environment where developers must meticulously count spaces to ensure proper alignment, increasing the cognitive load and the likelihood of mistakes. Even small discrepancies, such as an extra space or an inconsistent number of spaces between lines, can break the entire document, resulting in cryptic error messages that are often difficult to debug. In contrast, tabs are a single, fixed-width character, which eliminates these risks. One tab is always one tab, regardless of editor settings or environment, making it far easier to ensure consistent indentation without the risk of accidental misalignment. Moreover, the use of spaces introduces additional inefficiencies in both file size and readability. Unlike tabs, which are a single character representing an indentation level, spaces require multiple characters to achieve the same result. This becomes particularly problematic in complex YAML files with deep nesting, where the number of spaces required per indentation level rapidly increases. As a result, the file size grows unnecessarily, leading to increased memory usage and slower processing times for large files. From a readability standpoint, spaces also contribute to visual clutter. A deeply nested YAML file with multiple indentation levels quickly becomes difficult to parse visually, especially when working with large datasets. The excessive use of spaces makes the document harder to scan, as the indentation can appear inconsistent depending on the text editor’s configuration, further exacerbating the issue. Tabs, on the other hand, are compact, visually distinct, and consistent across all environments, making them far more efficient in both terms of space utilization and human readability. Additionally, the reliance on spaces for indentation creates inconsistencies across development environments and teams. Different text editors and IDEs have varying configurations for handling spaces, and many automatically convert tabs to spaces, further complicating the situation. In collaborative environments, this can lead to situations where developers are unknowingly working with different indentation conventions-one developer might use two spaces, another might use four, and yet another might use eight. These differences often go unnoticed until parsing errors occur, at which point the developer is left to diagnose the issue, wasting valuable time and effort. Tabs, by contrast, are universally consistent. They do not require any configuration and are universally understood as a single unit of indentation. This consistency eliminates the risk of human error and makes it far easier to maintain uniformity across a team or a project, reducing the likelihood of misalignment and improving overall productivity. Furthermore, YAML’s insistence on spaces ignores the broader context of programming language design, where tabs have long been the standard for indentation. In languages like Python, Go, and JavaScript, where indentation plays a critical role in defining program structure, tabs have proven to be far superior to spaces in terms of clarity, efficiency, and developer ergonomics. These languages have adopted tabs not as a matter of preference but as a deliberate design decision aimed at promoting consistency, readability, and productivity. Python, for example, mandates consistent indentation for defining code blocks, and the tab character serves as an efficient and unambiguous way to delineate these blocks. The widespread use of tabs in programming languages highlights the fact that this is not merely a question of style, but rather a fundamental decision that impacts the quality of code and the developer experience. YAML, in its departure from this convention, introduces an unnecessary and inefficient step backwards, making it more difficult for developers to work with data in a way that is consistent with modern practices. Finally, it is important to consider the mental overhead involved in using spaces for indentation in YAML. Developers working with YAML must not only keep track of indentation levels but also ensure that they are using the correct number of spaces and that those spaces are consistent throughout the document. This adds unnecessary cognitive load, especially in large, complex files with deeply nested structures. By contrast, using tabs would streamline the process. A single tab keypress would automatically enforce consistent indentation, reducing the risk of error and allowing developers to focus on the content of the document rather than on formatting. This is particularly important in larger teams or open-source projects, where consistency across contributors is essential. Adopting tabs for indentation would greatly reduce the complexity of working with YAML files and improve overall efficiency. In conclusion, while YAML has many strengths as a data serialization format, its reliance on spaces for indentation is a significant flaw that introduces unnecessary complexity, inefficiencies, and errors. The use of spaces for indentation forces developers to manually track alignment and increases the risk of human error, especially in large and complex files. Furthermore, spaces lead to inconsistent formatting across different text editors and environments, and they create visual clutter that diminishes readability. By contrast, tabs provide a consistent, compact, and unambiguous way to define indentation, offering significant benefits in terms of file size, clarity, and overall developer experience. Given the advantages of tabs and their widespread adoption in other programming languages, it is clear that YAML would benefit from reconsidering its choice of indentation and embracing tabs as the default. This change would align YAML with best practices in the broader software development ecosystem, improving both usability and productivity for developers working with this format.
@tonylim_nvd
@tonylim_nvd 26 күн бұрын
outdated, there is no more `build` command
@JohnFPederson
@JohnFPederson 26 күн бұрын
Bro how do I install a desktop? I'm so annoyed nobody shows that.
@rouakessaba7825
@rouakessaba7825 26 күн бұрын
is it possible to show us how to create a sharePoint source? I ve been at it for mor than a week now and it is not working.
@s_m_exist
@s_m_exist 28 күн бұрын
Hello, I have a question here on using dbt Cloud. If fivetran is injecting data to only one env, say Dev, Is there a way to maintain single codebase for dev, staging and prod by pointing source at dev raw datasets while staging raw and prod raw are views created on top of it? Each env transformation is based on their respective raw layers (but raw of staging and prod are just views on top of dev raw)
@hoangle-r5b8d
@hoangle-r5b8d Ай бұрын
thanks 👍it's very help me to know use git
@Banned-I3
@Banned-I3 Ай бұрын
I’m having trouble running HTML and css together
@vamshiyadav7259
@vamshiyadav7259 Ай бұрын
How to handle deletion in incremental model
@ripperx444
@ripperx444 Ай бұрын
It makes sense when you explain it. Problem is knowing all the syntax and nonesense with this GHA in complex flows.
@pennycasey1166
@pennycasey1166 Ай бұрын
I have a paginated report that dynamically selects the groupings based on the user's selections. There are four levels of groups each with their own parameters - although the parameters are identical and based on a query. The parameter has a friendly name as a label and a field as the value, so each of the four groups will be selected in order according to the selected label/field. I know how to make this work in SSRS but don't know how to make it coordinate with parameters in Power BI. Any suggestions?
@nileshdeshmukh7386
@nileshdeshmukh7386 Ай бұрын
It looks like this solution is no more valid. Can you please confirm.
@srikantaghosh2386
@srikantaghosh2386 Ай бұрын
Any best practices for handling late arriving data? How do handle this along with having an incremental model in place?
@AnupallaviBalaboina
@AnupallaviBalaboina Ай бұрын
hi canyou please make a playlist on gcp
@_pralhadbehera6341
@_pralhadbehera6341 Ай бұрын
Thanks its working now
@Mahsa_Faraji
@Mahsa_Faraji Ай бұрын
Thanks for your video. In your code, you've used flatten function. Is it possible to do like this in dbt with postgresql (I know that flatten doesn't exist in postgresql)? or it's better to use other method to flatten json?
@robertcomer6832
@robertcomer6832 Ай бұрын
I was taking you very seriously until you put the 76ers as a future NBA champs!
@clouddataconsulting
@clouddataconsulting Ай бұрын
We have developed a Snowflake Configuration Jumpstart, where we set up RAW databases for each ingestion tool, Fivetran, Matillion, Streaming, Custom, etc. We create service account users for each database so that Fivetran can only create and see objects in the Fivetran RAW database and Matillion in the Matillion Raw database. Each Raw database has a read-only role granted to the "application" database owner role, aka EDW or ODS. We can customize/adapt the framework/jumpstart for each client's needs by building the whole thing with Terraform. Ask me how we use future-based grants to seldom have to think about issuing grants.
@DhrubajyotiSinha-b6h
@DhrubajyotiSinha-b6h Ай бұрын
1 Insufficient memory. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod. how to fix?
@Priyakitchen-vlogs
@Priyakitchen-vlogs Ай бұрын
Hi Kahan its great vedio for new fivetran and dbt learner's i need one small help how we validate fivetran data is correctly loading to snowflake how we have to validate data using dbt models
@krishnatapse1994
@krishnatapse1994 2 ай бұрын
Bro UI is changed please re-create the video, Thanks in advance
@michaelKong-i7b
@michaelKong-i7b 2 ай бұрын
thank you. just solved my problem right away.
@altaneryilmaz3356
@altaneryilmaz3356 2 ай бұрын
Pfff anything new?
@ProXxXify
@ProXxXify 2 ай бұрын
Thank you for that video! Really! As a beginner in data engineering, I'm overwhelmed by the number of tools that are out there. I've learned the basics of SQL, but I've put it behind so I can learn all the other stuff too. And yes, I've learned some of the most used tools, but I'm constantly looking for the next new tool. Now I know what my next steps should be - to take a step back and spend more time to master SQL. 🚀
@juanete69
@juanete69 2 ай бұрын
Does {{ref('example_teams')}} refer to the original csv file or to the imported table?
@juanete69
@juanete69 2 ай бұрын
Thank you for the tutorial. But the image is very blurry.
@mhmdhosny25
@mhmdhosny25 2 ай бұрын
Thanks a lot
@CarbonsHDTuts
@CarbonsHDTuts 2 ай бұрын
At around 11:30 you say dimensions shouldn’t join other dimensions, but when creating the dimensions just before that moment, you did join for example customers and employees together ? Same for city and state or something like that. Confused on the contradictory advice, could someone please help clarify?
@ignatiusreilly8280
@ignatiusreilly8280 2 ай бұрын
An ETL pancackgce??! Wow!
@fgreve
@fgreve 2 ай бұрын
What if rather than creating a new table, I want to add new rows to an existing table in Snowflake with a csv that I locally have in my computer. And I want to do this with python in jupyter notebook?
@kaelandchatman4780
@kaelandchatman4780 2 ай бұрын
That was really helpful. Thank you!
@ShreyasSureshDhamore
@ShreyasSureshDhamore 2 ай бұрын
time stamp 7.38 in this vide I have see when you do dbt init then you can see which db you need to be configured instead of doing manually.?