pyspark.sql.functions.desc_nulls_last#

pyspark.sql.functions.desc_nulls_last(col)[source]#

Sort Function: Returns a sort expression based on the descending order of the given column name, and null values appear after non-null values.

New in version 2.4.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: target column to sort by in the descending order.

Returns

Column: the column specifying the order.

Examples

Example 1: Sorting a DataFrame with null values in descending order

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(0, None), (1, "Bob"), (2, "Alice")], ["age", "name"])
>>> df.sort(sf.desc_nulls_last(df.name)).show()
+---+-----+
|age| name|
+---+-----+
|  1|  Bob|
|  2|Alice|
|  0| NULL|
+---+-----+

Example 2: Sorting a DataFrame with multiple columns, null values in descending order

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame(
...   [(0, None, "Z"), (1, "Bob", None), (2, "Alice", "Y")], ["age", "name", "grade"])
>>> df.sort(sf.desc_nulls_last(df.name), sf.desc_nulls_last(df.grade)).show()
+---+-----+-----+
|age| name|grade|
+---+-----+-----+
|  1|  Bob| NULL|
|  2|Alice|    Y|
|  0| NULL|    Z|
+---+-----+-----+

Example 3: Sorting a DataFrame with null values in descending order using column name string

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(0, None), (1, "Bob"), (2, "Alice")], ["age", "name"])
>>> df.sort(sf.desc_nulls_last("name")).show()
+---+-----+
|age| name|
+---+-----+
|  1|  Bob|
|  2|Alice|
|  0| NULL|
+---+-----+