pyspark.sql.datasource.DataSourceStreamReader.latestOffset

DataSourceStreamReader.latestOffset()

Returns the most recent offset available.

Returns

dict

A dict or recursive dict whose key and value are primitive types, which includes Integer, String and Boolean.

Examples

>>> def latestOffset(self):
...     return {"parititon-1": {"index": 3, "closed": True}, "partition-2": {"index": 5}}