PySpark Aggregations

Not this time. In PySpark, group-based aggregations are performed using the groupBy method to specify the grouping column(s), followed by applying aggregation functions like sum, avg, or count

Not this time. describe() function in PySpark computes aggregate statistics like count, mean, standard deviation, minimum, and maximum for numerical columns in a DataFrame

Not this time. df.groupBy("columnA").count() operation groups the DataFrame by the unique values in columnA and returns a DataFrame with each unique value and the number of rows corresponding to it

Not this time. Pass several aggregate expressions to agg(), for example df.groupBy("country").agg(F.sum("amount"), F.avg("amount"), F.max("amount")). A dictionary such as {"amount": "sum"} maps each column to only one function, so it can't compute two aggregations of the same column.

Not this time. count("*") counts rows and count("column") counts the column's non-null values, while countDistinct() counts the number of different non-null values.

Not this time. groupBy() function groups data based on exact column values, while rollup() creates hierarchical subtotals for columns, making it useful for multi-level aggregations

Not this time. pivot() function, used after groupBy(), reshapes the DataFrame by converting unique values of a specified column into new columns, enabling aggregations for each value

Not this time. rollup() function is used for hierarchical aggregations, allowing you to compute subtotals and grand totals for multiple levels of grouping

Not this time. agg() function allows you to perform multiple aggregations in a single call, making your code more concise, readable, and efficient compared to chaining multiple aggregation methods

Not this time. Aggregate functions skip nulls: the average of 10, null and 20 is 15, and count("amount") counts only non-null values. If a null should count as a value (for example as 0), replace it first with fillna() or coalesce().