Interface ReaderGroup
-
- All Superinterfaces:
java.lang.AutoCloseable,ReaderGroupNotificationListener
public interface ReaderGroup extends ReaderGroupNotificationListener, java.lang.AutoCloseable
A reader group is a collection of readers that collectively read all the events in the stream. The events are distributed among the readers in the group such that each event goes to only one reader. The readers in the group may change over time. Readers are added to the group by callingEventStreamClientFactory.createReader(String, String, Serializer, ReaderConfig)and are removed by callingreaderOffline(String, Position)
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description voidclose()Closes the reader group, freeing any resources associated with it.java.util.concurrent.CompletableFuture<java.util.Map<Stream,StreamCut>>generateStreamCuts(java.util.concurrent.ScheduledExecutorService backgroundExecutor)Generates aStreamCutafter co-ordinating with all the readers usingStateSynchronizer.java.lang.StringgetGroupName()Returns the name of the group.ReaderGroupMetricsgetMetrics()Returns metrics for this reader group.java.util.Set<java.lang.String>getOnlineReaders()Returns a set of readerIds for the readers that are considered to be online by the group.ReaderSegmentDistributiongetReaderSegmentDistribution()Returns current distribution of number of segments assigned to each reader in the reader group.java.lang.StringgetScope()Returns the scope of the stream which the group is associated with.java.util.Map<Stream,StreamCut>getStreamCuts()Returns aStreamCutfor each stream that this reader group is reading from.java.util.Set<java.lang.String>getStreamNames()Returns the set of scoped stream names which was used to configure this group.java.util.concurrent.CompletableFuture<Checkpoint>initiateCheckpoint(java.lang.String checkpointName, java.util.concurrent.ScheduledExecutorService backgroundExecutor)Initiate a checkpoint.voidreaderOffline(java.lang.String readerId, Position lastPosition)Invoked when a reader that was added to the group is no longer consuming events.voidresetReaderGroup(ReaderGroupConfig config)Reset a reader group with the providedReaderGroupConfig.voidupdateRetentionStreamCut(java.util.Map<Stream,StreamCut> streamCuts)Update Retention Stream-Cut for Streams in this Reader Group.-
Methods inherited from interface io.pravega.client.stream.notifications.ReaderGroupNotificationListener
getEndOfDataNotifier, getSegmentNotifier
-
-
-
-
Method Detail
-
getMetrics
ReaderGroupMetrics getMetrics()
Returns metrics for this reader group.- Returns:
- a ReaderGroupMetrics object for this reader group.
-
getScope
java.lang.String getScope()
Returns the scope of the stream which the group is associated with.- Returns:
- A scope string
-
getGroupName
java.lang.String getGroupName()
Returns the name of the group.- Returns:
- Reader group name
-
initiateCheckpoint
java.util.concurrent.CompletableFuture<Checkpoint> initiateCheckpoint(java.lang.String checkpointName, java.util.concurrent.ScheduledExecutorService backgroundExecutor)
Initiate a checkpoint. This causes all readers in the group to receive a specialEventReadthat contains the provided checkpoint name. This can be used to provide an indication to them that they should persist their state. Once all of the readers have received the notification and resumed reading the future will return aCheckpointobject which contains the StreamCut of the reader group at the time they received the checkpoint. This can be used to reset the group to this point in the stream by callingresetReaderGroup(ReaderGroupConfig)if the checkpoint fails or the result cannot be obtained an exception will be set on the future. This method can be called and a new checkpoint can be initiated while another is still in progress if they have different names. If this method is is called again before the checkpoint has completed with the same name the future returned to the second caller will refer to the same checkpoint object as the first.- Parameters:
checkpointName- The name of the checkpoint (For identification purposes)backgroundExecutor- A threadPool that can be used to poll for the completion of the checkpoint.- Returns:
- A future Checkpoint object that can be used to restore the reader group to this position.
-
resetReaderGroup
void resetReaderGroup(ReaderGroupConfig config)
Reset a reader group with the providedReaderGroupConfig.- The stream(s) that are part of the reader group can be specified using
ReaderGroupConfig.ReaderGroupConfigBuilder.stream(String),ReaderGroupConfig.ReaderGroupConfigBuilder.stream(String, StreamCut)andReaderGroupConfig.ReaderGroupConfigBuilder.stream(String, StreamCut, StreamCut).- To reset a reader group to a given checkpoint use
ReaderGroupConfig.ReaderGroupConfigBuilder.startFromCheckpoint(Checkpoint)api.- To reset a reader group to a given StreamCut use
All existing readers will have to callReaderGroupConfig.ReaderGroupConfigBuilder.startFromStreamCuts(Map).EventStreamClientFactory.createReader(String, String, Serializer, ReaderConfig). If they continue to read events they will eventually encounter anReinitializationRequiredException.- Parameters:
config- The new configuration for the ReaderGroup.
-
readerOffline
void readerOffline(java.lang.String readerId, Position lastPosition)Invoked when a reader that was added to the group is no longer consuming events. This will cause the events that were going to that reader to be redistributed among the other readers. Events after the lastPosition provided will be (re)read by other readers in theReaderGroup. Note that this method is automatically invoked byEventStreamReader.close()- Parameters:
readerId- The id of the reader that is offline.lastPosition- The position of the last event that was successfully processed by the reader.
-
getOnlineReaders
java.util.Set<java.lang.String> getOnlineReaders()
Returns a set of readerIds for the readers that are considered to be online by the group. i.e.EventStreamClientFactory.createReader(String, String, Serializer, ReaderConfig)was called butreaderOffline(String, Position)was not called subsequently.- Returns:
- Set of active reader IDs of the group
-
getStreamNames
java.util.Set<java.lang.String> getStreamNames()
Returns the set of scoped stream names which was used to configure this group.- Returns:
- Set of streams for this group.
-
getStreamCuts
java.util.Map<Stream,StreamCut> getStreamCuts()
Returns aStreamCutfor each stream that this reader group is reading from. The stream cut corresponds to the last checkpointed read offsets of the readers, and it can be used by the application as reference to such a position. A more preciseStreamCut, with the latest read offsets can be obtained usinggenerateStreamCuts(ScheduledExecutorService)API.- Returns:
- Map of streams that this group is reading from to the corresponding cuts.
-
generateStreamCuts
@Beta java.util.concurrent.CompletableFuture<java.util.Map<Stream,StreamCut>> generateStreamCuts(java.util.concurrent.ScheduledExecutorService backgroundExecutor)
Generates aStreamCutafter co-ordinating with all the readers usingStateSynchronizer. AStreamCutis generated by using the latest segment read offsets returned by the readers along with unassigned segments (if any). The configurationReaderGroupConfig.groupRefreshTimeMillisdecides the maximum delay by which the readers return the latest read offsets of their assigned segments.The
StreamCutgenerated by this API can be used by the application as a reference to a position in the stream. This is guaranteed to be greater than or equal to the position of the readers at the point of invocation of the API. TheStreamCuts generated can be used to perform bounded processing of the Stream by configuring aReaderGroupwith aReaderGroupConfigwhere theStreamCuts are specified as the lower bound and/or upper bounds using the apisReaderGroupConfig.ReaderGroupConfigBuilder.stream(Stream, StreamCut, StreamCut)orReaderGroupConfig.ReaderGroupConfigBuilder.stream(Stream, StreamCut)orReaderGroupConfig.ReaderGroupConfigBuilder.startFromStreamCuts(Map).Note: Generating a precise
StreamCut, for example aStreamCutpointing to end of Q1 across all segments, is difficult as it depends on the configurationReaderGroupConfig.groupRefreshTimeMilliswhich decides the duration by which all the readers running on different machines/ processes respond with their latest read offsets. Hence, theStreamCutwould point to a position in theStreamwhich might include events from Q2. The application thus would need to filter out such additional events.- Parameters:
backgroundExecutor- A thread pool that will be used to poll if the positions from all the readers have been fetched.- Returns:
- A future to a Map of Streams (that this group is reading from) to its corresponding cuts.
-
updateRetentionStreamCut
void updateRetentionStreamCut(java.util.Map<Stream,StreamCut> streamCuts)
Update Retention Stream-Cut for Streams in this Reader Group. SeeReaderGroupConfig.StreamDataRetention.MANUAL_RELEASE_AT_USER_STREAMCUT- Parameters:
streamCuts- A Map with a Stream-Cut for each Stream. StreamCut indicates position in the Stream till which data has been consumed and can be deleted.
-
getReaderSegmentDistribution
ReaderSegmentDistribution getReaderSegmentDistribution()
Returns current distribution of number of segments assigned to each reader in the reader group.- Returns:
- an instance of ReaderSegmentDistribution which describes the distribution of segments to readers including unassigned segments.
-
close
void close()
Closes the reader group, freeing any resources associated with it.- Specified by:
closein interfacejava.lang.AutoCloseable
-
-