Monitor state & per node

Alginte shows a stream’s state at the depth it can see it, and that depends on where the stream runs. Find your case first; the sections that follow describe what each view shows.

​

Which view you get

• Deployed in Alginte. The console hosts the Kafka Streams client and reads it directly, no agent and no scraping: restore progress, store and changelog lag, thread state, rebalances and events, plus the cluster’s own facts beside them. This is the State tab and the Stream Flow badges, everything below.
• Running elsewhere, on a cluster this console reads. The console never touches the application; it reads what the cluster alone says about it, with read-only rights: group state and size, committed offsets and lag, the changelog topics and how far they reach. Thread state, restore progress and store sizes are not visible at this depth and are shown as absent. This is the Observed on the cluster table and the observed page.
• Running elsewhere, reporting its own state. Coming. An application you run outside Alginte can push the same state a hosted stream shows, thread state, restore progress and store sizes included, through a small open-source reporter library added to the application, one call before start(). The console side, the observations endpoint under a bearer token, is in 0.11.0 and documented on the configuration page; the library follows. Until it does, such an application is shown at the cluster-only depth above.

Both views that exist today use the same State tab, and its Observer card names which case you are looking at, so a screenshot carries its own depth.

​

The State tab

Open a stream’s detail page and switch to State. The tab is one snapshot of the stream as its observer sees it, plus the events that led there. A restore banner appears at the top only while a store partition is replaying its changelog: how many partitions, how many records of how many, and a progress bar. It disappears the moment the last partition finishes. This is the view for the question that usually sends people to the logs: “the stream says REBALANCING, is it stuck or is it restoring?” Lifecycle shows the state the console derives, not only the one Kafka reports, because the raw state lies in two known ways:

• Kafka says REBALANCING forever while every stream thread is dead. The console reads that as ERROR.
• Kafka says RUNNING while threads died and were replaced faster than the client noticed, so some tasks are assigned but nobody serves them. The console reads that as DEGRADED, and Tasks served shows the shortfall in red, 3 of 6.

Kafka’s own state is written beside the verdict in both cases, so nothing is hidden. Since tells you how long the current state has lasted, which is “rebalancing for a day” made visible without reading an event log. Rebalances counts them since the observer started and shows when the last one began, ended, and how long it took. Cluster is the corroboration: what the cluster says about the same stream, independent of the client. Group state and member count, the changelog topics that exist for it, and for a streams-protocol group (KIP-1071) the tasks it has assigned. Observer names who is looking. The Tier tag says how much the observer can see: an in-process observer sees everything, an inferred one only what the cluster says. Observed at is when the snapshot was taken. Stores lists every store partition the stream holds, active or standby, with the one figure that matters most read three ways: while restoring it is records left to replay, on a standby it is how far behind the active it runs, and on an active partition at steady state it is the changelog lag, colored as a warning only in that last case. Beside it, the restore phase and progress, the live entry count, and the RocksDB size on disk. Source partitions are the cluster’s view of where the stream is reading: committed offset, end offset and lag per input partition. Threads shows each stream thread’s state and the tasks it runs. Events is the timeline: state transitions, rebalances starting and ending with their duration, restores starting and ending with their record counts, thread deaths with the decision taken, and standby updates.

​

Streams you did not deploy here

The Streams page ends with Observed on the cluster: every Kafka Streams application the cluster knows about, whether or not this Alginte instance runs it. The console finds them by the signs Kafka Streams leaves behind: a streams-protocol group, a consumer group whose members carry the StreamThread client id, or an internal -changelog or -repartition topic, which is how a stopped stateful application is still listed. Each row shows the group state, the member count, the protocol (classic for a consumer group, streams for a KIP-1071 group) and how the console observes it: hosted here links to the stream’s detail page, inferred from the cluster opens the same State tab alone, under a banner that says what this view can and cannot see. What the cluster alone can say is honest but sparse: the group’s state and size, committed offsets and lag, the changelog topics and how far each reaches. What it cannot say is shown as absent: there is no lifecycle, because a group reads Stable while threads die and the console refuses to invent one; no restore progress, no store sizes, no threads. Two events are synthesised from successive reads, since the cluster offers no listener: the group’s state changing, and for a streams-protocol group its epoch rising, which counts the rebalances in between.

​

Per-node badges on the canvas

Switch to Stream Flow. Each node carries a badge with the measurements Kafka actually records for it:

• N entries is the live size of a stateful node’s store (aggregates, counts, reducers). Always on.
• N msg/s / N total is throughput. Exact on source nodes and on grouping nodes (records crossing the repartition boundary); values prefixed with ~ are attributed from the surrounding subtopology, exact for linear chains and an upper bound around branches.
• The canvas frame is colored by the stream’s state, green running and red error, so a screenshot of the tab is a health check in itself.

Hover a badge to see the node’s runtime name, the same name you find in logs, JMX, and the internal topic names on your cluster.

​

The DEBUG metrics toggle

Kafka Streams records throughput only at its DEBUG metrics level, which is off by default because it has runtime cost. The Live throughput (DEBUG metrics) switch above the canvas turns it on for this stream only:

1. Flip the switch. Alginte recycles the stream’s runtime client to apply the level, waiting for the previous client to release its state directory first. Offsets and state continue; processing resumes within seconds, longer for a large store.
2. Throughput badges light up with real numbers as records flow.
3. Flip it back off when you are done watching. The store-size badges and state coloring keep working either way.

The toggle is an operational control, not part of the stream’s definition: a restart brings the stream back at its deployed metrics level.

​

Where the numbers come from

Everything on the State tab is one call, GET /api/kafka-streams/streams/{applicationId}/observation: the snapshot and the recent events. Observed streams answer at GET /api/kafka-streams/observed and GET /api/kafka-streams/observed/{applicationId}/observation. The canvas badges are served by GET /api/kafka-streams/streams/{applicationId}/node-metrics, a projection of the same snapshot grouped per canvas node, and the raw runtime metrics remain on the Metrics tab. Refresh follows the page’s auto-refresh control, so every view updates on whatever cadence you set.

​

From the blog

• Monitoring Kafka Streams applications: the state a cluster view cannot show: what the process knows and the cluster does not (a store restoring, a standby lagging, a thread lost), and why the State tab reads it from the application.