pyspark.sql.datasource.DataSourceWriter.commit#
- DataSourceWriter.commit(messages)[source]#
Commits this writing job with a list of commit messages.
This method is invoked on the driver when all tasks run successfully. The commit messages are collected from the
DataSourceWriter.write()method call from each task, and are passed to this method. The implementation should use the commit messages to commit the writing job to the data source.- Parameters
- messageslist of
WriterCommitMessages A list of commit messages.
- messageslist of