Low-code Data Transformations in Azure Synapse using Data Flows

[@gloverjpuk]

Great video, i’ve already shared it with our internal team who still think the only ETA tool on the planet is SSIS 😂 azure synapse is definitely the future

[@AzureSynapse]

Thank you, James! Well, we also have Azure-SSIS IR in Azure Synapse and Azure Data Factory, so SSIS fans can still work with their beloved technology in the cloud. But we share your opinion, that the Azure Data Integration tools (ADF, Synapse Pipelines) are the future :-)

[@sanjibsrkr1]

Can you explain this to my company as well? They think there data will be stolen if they move to the cloud :P

[@nitinpandey4857]

How come your cluster start time is 2s. Mine takes 3 minutes to spin up a cluster in DF pipeline.

[@AzureSynapse]

Hi Nitin, good point. Nothing secret here. The cluster spin up took pretty much the same for us as in your case. Notice the "Data flow debug" option was enabled right after we created a new data flow. This was because the IR was still running after the last data preview (read - we worked on another data flow before we started the one which we recorded). The debug time to live option allows keeping the cluster active for some time (of course, for cost optimization purposes we recommend shutting the debug session when you're done with your data flow).

[@sadaiyappangopi2775]

First of all thanks for this initiative for the KZbin channel on Synapse. It's really interesting to learn from the experts who built the product. I am facing an issue in one of my data flow. Here it is,I am tried to extract a table content from the Dedicated pool to pipe Delimited file using the data flow - conditional split transformation based on a attribute value Y Or N... I am getting one extra blank record in one of the file with only pipes, though I don't have any blank record in the source table. Any idea to find the root cause?