Package pyspark

source code

PySpark is the Python API for Spark.

Public classes:

• SparkContext Main entry point for Spark functionality.
• RDD A Resilient Distributed Dataset (RDD), the basic abstraction in Spark.
• Broadcast A broadcast variable that gets reused across tasks.
• Accumulator An "add-only" shared variable that tasks can only add values to.
• SparkFiles Access files shipped with jobs.
• StorageLevel Finer-grained cache persistence levels.