Synapse Analytics and .NET for Apache Spark Example 2 - ROW_NUMBER()

The next example is how to do a ROW_NUMBER(), my favourite window function.

T-SQL

SELECT 
	*, ROW_NUMBER() OVER(ORDER BY dateTime)  as RowNumber
FROM 
    chicago.safety_data

Spark SQL

SELECT 
	*, ROW_NUMBER() OVER(ORDER BY dateTime)  as RowNumber
FROM
    chicago.safety_data

DataFrame API (C#)

var dataFrame = spark.Read().Table("chicago.safety_data");

var window = Microsoft.Spark.Sql.Expressions.Window.OrderBy("dateTime");
dataFrame = dataFrame.WithColumn("RowNumber", Functions.RowNumber().Over(window));
dataFrame.Show();

To see this in action, please feel free to deploy this repo to your Synapse Analytics repo: https://github.com/GoEddie/SynapseSparkExamples