Metrics
Pravega metrics are collected and exposed via Flink metrics framework when using FlinkPravegaReader or FlinkPravegaWriter.
Reader Metrics
The following metrics are exposed for FlinkPravegaReader related operations:
| Name | Description |
|---|---|
readerGroupName |
The name of the Reader Group. |
scope |
The scope name of the Reader Group. |
streams |
The fully qualified name (i.e., scope/stream) of the streams that are part of the Reader Group. |
onlineReaders |
The readers that are currently online/available. |
segmentPositions |
The StreamCut information that indicates where the readers have read so far. |
unreadBytes |
The total number of bytes that have not been read yet. |
Writer Metrics
For FlinkPravegaWriter related operations, only the stream name is exposed:
| Name | Description |
|---|---|
streams |
The fully qualified name of the stream i.e., scope/stream |
Querying Metrics
The metrics can be viewed either from Flink UI or using the Flink REST API (like below):
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.readerGroupName
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.scope
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.streams
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.onlineReaders
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.stream.test.segmentPositions
curl -i -s -f /jobs/<JOB-ID>/vertices/<SOURCE-TASK-ID>/metrics?get=0.Source__<SOURCE-OPERATOR-NAME>.PravegaReader.readerGroup.unreadBytes