pyspark.streaming.StreamingContext.queueStream#
- StreamingContext.queueStream(rdds, oneAtATime=True, default=None)[source]#
Create an input stream from a queue of RDDs or list. In each batch, it will process either one or all of the RDDs returned by the queue.
- Parameters
- rddslist
Queue of RDDs
- oneAtATimebool, optional
pick one rdd each time or pick all of them once.
- default
pyspark.RDD, optional The default rdd if no more in rdds
Notes
Changes to the queue after the stream is created will not be recognized.