Spark

Data, Devops, Pipelines, Testing, Fun in the Agile SQL Club

2024

Spark Connect Dotnet November 2024 Where are we?

Nov 16

Spark Connect Dotnet Variant Data Type

Oct 12

Delta Lake from .NET in a Spark Connect gRPC world

Feb 17

Implementing functions and more fun in Spark Connect using gRPC and .NET

Feb 10

Apache Spark from PHP - it is not just a .NET thing

Feb 10

Moving towards the DataFrame API using the Spark Connect gRPC API in .NET

Feb 5

Exploring the Spark Connect gRPC API more

Feb 1

Exploring the Spark Connect gRPC API

Jan 26

Using Spark Connect from .NET to run Spark jobs on Databricks

Jan 23

Using Spark Connect from .NET

Jan 17

Spark Connect Dotnet First Major Milestone

Jan 17

2021

Synapse Analytics and .NET for Apache Spark Example 4 - JOINS

Jan 25

Synapse Analytics and .NET for Apache Spark Example 3 - CTE()

Jan 19

Synapse Analytics and .NET for Apache Spark Example 2 - ROW_NUMBER()

Jan 18

Synapse Analytics and .NET for Apache Spark Example 1 - Group By

Jan 18

How to get data in a DataFrame via .NET for Apache Spark

Jan 17

2020

TF-IDF in .NET for Apache Spark Using Spark ML v2

Dec 19

Blog .NET for Apache Spark UDFs Missing Shared State

Oct 29

TF-IDF in .NET for Apache Spark Using Spark ML

Jul 7

Approaches to running Databricks ETL code from Azure ADF

Jun 3

Spark Delta Lake, Updates, Deletes and Time Travel

Jan 20

2019

How to test ETL Processes in production

Oct 2

How to connect spark to ms sql server without '[Error] [JvmBridge] java.sql.SQLException: No suitable driver'?

Oct 1

Help! My Apache Spark configuration on windows isn't working (I think)

Aug 29

How do you debug your spark-dotnet app in visual studio?

Aug 20

spark-dotnet how to manually break a file into rows and columns

Aug 6

java.lang.ClassNotFoundException: org.apache.spark.deploy.DotnetRunner with 0.4.0 of spark-dotnet

Aug 2

How do we test ETL pipelines? Part one unit tests

Jul 25

spark-dotnet examples - reading and writing csv files

Jul 15

spark-dotnet how does user .net code run spark code in a java vm

Jul 14

Spark and dotnet in a single docker container

Jul 11