Commits

History for

Commits on Jul 1, 2026

[SPARK-57741][PYTHON] Add timestamp_nanos to PySpark public API

jubins
authored and
MaxGekk
committed

Commits on Jun 24, 2026

[SPARK-57579][PYTHON] Add PySpark support for unix_nanos function

jubins
authored and
MaxGekk
committed

Commits on Jun 17, 2026

[SPARK-57417][SQL] Add `variant_delete` expression

bojana-db
authored and
cloud-fan
committed
[SPARK-57452][DOC] Add missing 4.2 behavior changes to migration guides
cloud-fan
committed

Commits on Jun 16, 2026

[SPARK-57021][CONNECT][PYTHON] Add SQLContext wrapper for Spark Connect

dbtsai
authored and
Yicong-Huang
committed

Commits on Jun 9, 2026

[SPARK-57253][SQL] Add `jaro_winkler_similarity` built-in function

sarutak
authored and
cloud-fan
committed

Commits on Jun 6, 2026

[SPARK-57284][PYTHON][SQL] Add Scala/Python bindings for vector functions
sarutak
committed

Commits on May 31, 2026

[SPARK-55886][SQL][PYTHON] Add DataFrame.zip for merging column-projected DataFrames

zhengruifeng
and
cloud-fan
committed

Commits on May 27, 2026

Commits on May 19, 2026

[SPARK-56928][PYTHON] Increase minimum numpy version to 1.23.2

gaogaotiantian
authored and
dongjoon-hyun
committed

Commits on May 16, 2026

[SPARK-56889][PYTHON][INFRA] Drop Python 3.10 Support
dongjoon-hyun
committed

Commits on May 15, 2026

[SPARK-56820][SQL] Add counter_diff window function for converting cumulative counters to delta format

pnikic-db
authored and
cloud-fan
committed

Commits on May 14, 2026

[SPARK-56854][PYTHON] Filter None values in DataFrame[Stream]Reader/Writer .option(s)

lavanv11
authored and
HyukjinKwon
committed
[SPARK-56395][CONNECT][PYTHON] Add NEAREST BY DataFrame API

dilipbiswal
authored and
cloud-fan
committed

Commits on May 10, 2026

[SPARK-40336][PS] Implement DataFrameGroupBy.cov

devin-petersohn
authored and
HyukjinKwon
committed
[SPARK-40373][PS] Implement ps.show_versions

devin-petersohn
authored and
HyukjinKwon
committed

Commits on May 6, 2026

Commits on May 5, 2026

[SPARK-52857] Add is_valid_variant expression

chenhao-db
authored and
cloud-fan
committed

Commits on May 1, 2026

[SPARK-56594][SQL] Add time_bucket scalar function

vranes
authored and
cloud-fan
committed

Commits on Apr 17, 2026

[SPARK-56256][PYTHON][DOCS][FOLLOWUP] Document SparkSession.emptyDataFrame in API reference

zhengruifeng
authored and
HyukjinKwon
committed
[SPARK-56518][PYTHON] Add current_path to PySpark functions

srielau
authored and
HyukjinKwon
committed

Commits on Apr 15, 2026

[SPARK-56351][PYTHON][DOCS] Add dedicated documentation page for Arrow Python UDFs

Yicong-Huang
authored and
zhengruifeng
committed

Commits on Apr 8, 2026

[SPARK-56221][SQL][PYTHON][FOLLOWUP] Rename TablePartition and remove SHOW CACHED TABLES / listCachedTables
cloud-fan
committed

Commits on Apr 7, 2026

[SPARK-55450][SS][PYTHON][DOCS] Document admission control in PySpark streaming data sources

jiteshsoni
authored and
HeartSaVioR
committed

Commits on Apr 3, 2026

[SPARK-56341][PYTHON][DOCS] Fix outdated PyArrow minimum version in arrow_pandas.rst

Yicong-Huang
authored and
zhengruifeng
committed

Commits on Mar 26, 2026

[SPARK-56221][SQL][PYTHON] Feature parity between spark.catalog.* vs DDL commands
HyukjinKwon
committed

Commits on Mar 23, 2026

[SPARK-56135][DOCS] Add option to use venv in PySpark docs

Yicong-Huang
authored and
zhengruifeng
committed

Commits on Mar 21, 2026

[SPARK-55806][DOCS] add section for running `pyspark` with `uv` inline dependencies

markjm
authored and
HyukjinKwon
committed

Commits on Feb 23, 2026

[SPARK-55558][SQL] Add support for Tuple/Theta set operations

authored and
dtenedor
committed

Commits on Feb 19, 2026

[SPARK-55540][DOCS] Fix capitalization inconsistency in PySpark Overview "Live Notebook" link

knkarthik01
authored and
HyukjinKwon
committed

Commits on Feb 17, 2026

[SPARK-55279][SQL][FOLLOW-UP] Add sketch_funcs and Datasketch grouping for SQL functions

authored and
dtenedor
committed

Commits on Feb 13, 2026

[SPARK-55416][SS][PYTHON] Streaming Python Data Source memory leak when end-offset is not updated

vinodkc
authored and
HeartSaVioR
committed

Commits on Feb 11, 2026

[SPARK-54179][SQL][FOLLOW-UP] Add Dataframe API support for Tuple sketches

authored and
dtenedor
committed