pyspark.sql.DataFrameWriterV2

class pyspark.sql.DataFrameWriterV2(df: DataFrame, table: str)[source]

Interface used to write a class:pyspark.sql.dataframe.DataFrame to external storage using the v2 API.

New in version 3.1.0.

Changed in version 3.4.0: Supports Spark Connect.

Methods

append()

Append the contents of the data frame to the output table.

create()

Create a new table from the contents of the data frame.

createOrReplace()

Create a new table or replace an existing table with the contents of the data frame.

option(key, value)

Add a write option.

options(**options)

Add write options.

overwrite(condition)

Overwrite rows matching the given filter condition with the contents of the data frame in the output table.

overwritePartitions()

Overwrite all partition for which the data frame contains at least one row with the contents of the data frame in the output table.

partitionedBy(col, *cols)

Partition the output table created by create, createOrReplace, or replace using the given columns or transforms.

replace()

Replace an existing table with the contents of the data frame.

tableProperty(property, value)

Add table property.

using(provider)

Specifies a provider for the underlying output data source.
pyspark.sql.DataFrameWriter pyspark.sql.UDFRegistration