Built-In Functions

• PySpark - when()

• PySpark - expr()

• PySpark - lit()

  PySpark SQL functions lit() and typedLit() are used to add a new column to DataFrame by assigning a literal or constant value. Both these functions return Column type as return type.

• PySpark - split()

• PySpark - concat_ws()

• Pyspark - substring()

• PySpark - translate()

• PySpark - regexp_replace()

• PySpark - overlay()

• PySpark - to_timestamp()

• PySpark - to_date()

• PySpark - date_format()

• PySpark - datediff()

• PySpark - months_between()

• PySpark - explode()

• PySpark - array_contains()

• PySpark - array()

• PySpark - collect_list()

• PySpark - collect_set()

• PySpark - create_map()

• PySpark - map_keys()

• PySpark - map_values()

• PySpark - struct()

• PySpark - countDistinct()

• PySpark - sum(), avg()

• PySpark - row_number()

• PySpark - rank()

• PySpark - dense_rank()

• PySpark - percent_rank()

• PySpark - typedLit()

• PySpark - from_json()

• PySpark - to_json()

• PySpark - json_tuple()

• PySpark - get_json_object()

• PySpark - schema_of_json()