Beginners Guide to Read and Write data in Azure Databricks

1. Provision the Resources Required

2. Upload the Sample file to Databricks (DBFS)

3. Read and Write the Data

# File location and type
file_location = “/FileStore/tables/Country_Sales_Records.csv”
file_type = “csv”

# CSV options
infer_schema = “false”
first_row_is_header = “false”
delimiter = “,”

# The applied options are for CSV files. For other file types, these will be ignored.
df = spark.read.format(file_type) \
.option(“inferSchema”, infer_schema) \
.option(“header”, first_row_is_header) \
.option(“sep”, delimiter) \
.load(file_location)

display(df)

# CSV options
infer_schema = “true”
first_row_is_header = “true”

Create a table and query the data using SQL

# Create a view or table
tblCountrtySales = “Country_Sales”
df.createOrReplaceTempView(tblCountrtySales)

%sql
select * from `Country_Sales`

tbl_name = “tbl_Country_Sales”
# df.write.format(“parquet”).saveAsTable(tbl_name)

ref

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store