PySpark custom data sources are created using the Python (PySpark) DataSource API, which enables reading from custom data sources and writing to custom data sinks in Apache Spark using Python. You can ...
The following release notes provide information about Databricks Runtime 18.2. This version incorporates all features, improvements, and bug fixes from all previous Databricks Runtime releases.