Is the gold layer in Databricks really necessary when using AtScale? What I have read, the gold layer is mostly aggragations and that is what you dont need and should not do when using AtScale right?
@harigabor9102 жыл бұрын
Based on my experience with the semantic/self service-style lakehouse, when the data quality is bad enough you kind of need the Gold Zone for flexibility because you are free to partition and filter it however you want, while you want to keep the number of Silver tables to a minimum and possibly partition them on a date field for MERGE deduplication.