set up for linux laptop

scripts and tools for grassroots political campaigns

Utilities that automate tedious mathematical calculations like finding eigenvalues, applying quadratic formula, etc.

text and source code for articles i wrote

solutions-to-various-discrete-math-number-theory-problems-while-i-learn-up-on-python

Demo of how when we deploy on AWS/EMR the structured streaming metrics tab shows no batches processed and no graphs

Gradle plug-in that sets up testNG as test runner, and adds a new 'integrationTest' task

Code examples for article here: http://datalackey.com/2020/09/22/unit-integration-testing-kafka-and-spark/

A home for various Scala examples that I want to retain for posterity.

generate features for machine learning

spark sql, dataset/dataframe and structured streaming examples used to study for databricks certification exam

toy project illustrating how to process a mock stream of sensor data via Spark structured streaming and the (older) dstreams API

code samples for this article: http://datalackey.com/2019/06/21/exploring-event-time-and-processing-time-in-spark-structured-streaming/

short demo -- in scala -- of how to validate arbitrary JSON instance documents against a JSON schema

complex event processing code