Рет қаралды 4,253
Azure Databricks Learning: Spark Reader: Skip First N Records While Reading CSV File
=================================================================================
Processing CSV files in Spark and Databricks is one of the very frequently seen scenario. While reading CSV data, we come across requirement of skipping first few records in certain usecases. I have explained that requirement in this video
To get more understanding, watch this video
#SparkCSVReader, #SparkCSVSkipRows, #DatabricksCSVSkipRows,#CSVDataframe,#PySparkCSVOptions,#SparkDevelopment,#DatabricksDevelopment, #DatabricksPyspark,#PysparkTips, #DatabricksTutorial, #AzureDatabricks, #Databricks, #Databricksforbeginners,#datascientists, #datasciencecommunity,#bigdataengineers,#machinelearningengineers
Id,Year,First Name,County,Gender,Count
1,2007,ZOEY,KINGS,F,11
2,2007,ZOEY,SUFFOLK,F,6
3,2007,ZOEY,MONROE,F,6
4,2007,ZOEY,ERIE,F,9
5,2007,ZOE,ULSTER,F,5
6,2007,ZOE,WESTCHESTER,F,24
7,2007,ZOE,BRONX,F,13
8,2007,ZOE,NEW YORK,F,55
9,2007,ZOE,NASSAU,F,15
10,2007,ZOE,ERIE,F,6
11,2007,ZOE,SUFFOLK,F,14
12,2007,ZOE,KINGS,F,34
13,2007,ZOE,MONROE,F,9
14,2007,ZOE,QUEENS,F,26
15,2007,ZOE,ALBANY,F,5
16,2007,ZISSY,ROCKLAND,F,5
17,2007,ZISSY,KINGS,F,27
18,2007,ZION,KINGS,M,15
19,2007,ZION,BRONX,M,14
20,2007,ZEV,ROCKLAND,M,6
21,2007,ZEV,KINGS,M,30
22,2007,ZARA,QUEENS,F,10
23,2007,ZAIRE,KINGS,M,14
24,2007,ZACKARY,SUFFOLK,M,6
25,2007,ZACKARY,ERIE,M,5