Beginners Guide to Read and Write data in Azure Databricks

This article is a continuation of my earlier article (Dumb Down Azure Databricks Delta Lake Architecture) and the primary goal of this article is to demonstrate how we can read and write the data in Azure databricks and delta lake.

Below are the steps we are going to cover in this article

1. Provision of the required resources
2. Create the Cluster and Upload the sample file
3. Create a notebook and read the file
4. Create a data frame and a delta-lake table
5. Query the table using SQL script

1. Provision the Resources Required

Azure Databricks Workspace:-

→From the Azure portal, click Create a resource → Analytics → Databricks.
Enter the required details and Click Review+Create

Create a Spark Cluster:-

Open the Azure Databricks Workspace and click on the New Cluster

Give a meaningful name to Cluster and select the Runtime version and Worker Type based…