pyspark.sql.DataFrame.zip#

DataFrame.zip(other)[source]#

Combines the columns of this DataFrame with another DataFrame side-by-side, preserving row alignment between the two inputs.

Both DataFrames must produce the same canonicalized plan after stripping outer Project chains. In practice this means they derive from a common source through chains of projection-only operations (select(), withColumn(), withColumnRenamed(), etc.); the chains may differ between the two sides, but anything below them, including any filter(), orderBy(), join(), or aggregation, must be identical on both sides so the two sides stay row-aligned. Non-scalar Python UDFs (e.g., GROUPED_MAP) are not allowed on either side. An AnalysisException is thrown when the two DataFrames cannot be aligned.

Added in version 4.3.0.

Parameters:

otherDataFrame

The DataFrame to combine with, which must derive from the same source as this DataFrame.

Returns:

DataFrame

A new DataFrame containing the columns of this DataFrame followed by the columns of other.

Examples

>>> df = spark.createDataFrame([(1, 2, 3), (4, 5, 6)], ["a", "b", "c"])
>>> left = df.select("a")
>>> right = df.select("b")
>>> left.zip(right).show()
+---+---+
|  a|  b|
+---+---+
|  1|  2|
|  4|  5|
+---+---+