> ## Documentation Index > Fetch the complete documentation index at: https://docs.alginte.com/llms.txt > Use this file to discover all available pages before exploring further. # SpEL reference & cookbook > What every expression field can see, operator by operator; what each serde hands you for key and value, including the missing-field behaviour; windowed keys; and a cookbook of recipes, each one a row in the fidelity corpus that checks the editor against the deployed runtime. This page is the scan-first companion to [SpEL expressions](/streams/spel), which explains the ideas. Everything here is pinned: every scope rule and every recipe is a row in the fidelity corpus, a test suite that runs the same expression through the editor's validator, the sample preview and a deployed topology and fails when they disagree. The scope rules below have always been what the runtime does. Since Alginte 0.12.0 the editor enforces two of them it used to let through: an aggregate **initializer** has no record in scope, and **Map Values** sees only the value. ## What each expression can see `key` and `value` are the record at the operator's input. `#`-variables are extra context some roles carry. A name that is not in scope is rejected by the editor with a hint, and would fail deployed. | Operator, field | `key` | `value` | Variables | Result | | ---------------------------------------------------- | :----------: | :--------------------: | ---------------------------------- | ---------------------------------------------------------------------------------------------- | | **Filter** / **Filter Not**, Inclusion Predicate | yes | yes | | `true` keeps the record | | **Split**, each branch predicate | yes | yes | | first `true` branch wins; unmatched records take the default branch | | **Map**, Map Key Expression and Map Value Expression | yes | yes | | the new key and the new value | | **Map Values** | | yes | | the new value; the key is not in scope | | **FlatMap**, key and value expressions | yes | yes | | lists, zipped into records | | **FlatMap Values** | yes | yes | | a list, one record per element, same key | | **Select Key** | yes | yes | | the new key | | **Group By** | yes | yes | | the grouping key | | **Group Table By**, key and value expressions | yes | yes | | the grouping key and value | | **Foreach** / **Peek**, Action Expression | yes | yes | | discarded; a throw is logged and the record flows on | | **Aggregate** / **Cogroup**, Initializer | | | | the empty accumulator: a constant, a literal such as `{'total': 0}`, `T(...)` or a `#function` | | **Aggregate** / **Cogroup**, Adder and Subtractor | grouping key | the new record | `#aggValue`, the accumulator | the new accumulator | | **Reduce**, Adder and Subtractor | | the new record | `#aggValue`, the accumulated value | the new accumulated value | | **Join** (every kind), Value Joiner | | | `#leftValue`, `#rightValue` | the joined value | | **Join** with a global table, Key Value Mapper | yes | yes | | the key to look up | | **Foreign-Key Join**, Foreign Key Extractor | | the left table's value | | the referenced key | Two of these bite people: * **Map Values has no `key`.** The runtime evaluates it against the value alone. To read the key while changing the value, use **Map** with the key expression set to `key`. * **The initializer has no record.** Kafka calls it once per new key with no arguments, so `value.get('quantity')` there has nothing to read. Put the first record's contribution in the adder, which runs for it like for every other record. ## What each serde hands you The deserializer's own object reaches your expression, for `key` and `value` alike. There is no conversion layer, so what the preview shows is what runs. | Serde | `key` / `value` is | Reach into it with | A field that is not there | | ----------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------- | | String | `String` | String methods: `value.toUpperCase()`, `value.contains('Ket')` | `value.get('x')` fails: a String has no fields | | Long, Integer, Short, Double, Float | the boxed number | arithmetic and comparisons | | | Bytes | Kafka `Bytes` | `T(...)` on `Bytes` | | | UUID | `UUID` | `.toString()` | | | Void | `null` | | | | JSON, JSON Schema | a `Map` | `get('field')`, `['field']`, `containsKey(...)`, `keySet()` | **`null`** | | Avro | a `GenericRecord` | `get('field')`; nested records chain `get(...).get(...)`; arrays are `List`s | **throws** `Not a valid schema field` | | Protobuf | a `DynamicMessage` | `get('field')`; nested messages chain; repeated fields are `List`s; unset proto3 scalars give their defaults | **`null`** | **Strings inside Avro** are `org.apache.avro.util.Utf8`, not `String`: `== 'Kettle'` compares content and works, `.contains('ttl')` does not exist on it and fails, `.toString()` first makes every String method available. Protobuf and JSON Schema hand you real `String`s. An Avro enum symbol compares through `.toString()`; a `timestamp-millis` field is a `long` and does arithmetic; a `decimal` logical type is bytes to the expression and does not compare as a number. **A missing field** is loud on Avro and silent on the other two. On Avro and Protobuf an unknown name is always a typo or schema drift; on JSON Schema an absent key may be an optional property, which is why the quiet `null` is the honest answer there. The practical consequence: a filter on a misspelt field fails visibly on an Avro topic and quietly drops every record on a Protobuf or JSON Schema one. Whether to make the three agree is an open decision; until it is taken, this table is the behaviour. ## Windowed keys After a **Tumbling**, **Hopping**, **Sliding** or **Session Window** and its aggregate, `key` is Kafka's `Windowed` object all the way through **To Stream** and beyond, not the plain key. It has to be unwrapped before a sink, because no key serde here can serialize a `Windowed`: deployed as it is, the stream would die on the first record. Alginte refuses that graph, twice: the sink node grows an error badge while you draw, and the deploy is rejected with the same message, both naming the fix. Read the key with `key.key()`, `key.window().start()` and `key.window().end()` in any expression that has the key in scope: ```text theme={null} // Select Key: the plain key and the window start, as one String key key.key() + '@' + key.window().start() // Map: the plain key back as the key, the window into the value key.key() // Map Key Expression key.window().start() + '-' + key.window().end() + '=' + value // Map Value Expression ``` Map Values cannot do this, since it does not see the key. The [tumbling-window example](/streams/examples#3-—-tumbling-window-count-windowed) uses the Select Key form. ## Cookbook Each recipe shows the expression, the record before and the record after, as its corpus row runs it. `orders` is an Avro topic (`orderId`, `customerId`, `item`, `quantity`, `priceEur`); `shipments` is JSON Schema (`carrier`, `weightKg`); `payments` is Protobuf (`method`). ### Filter on a field Filter, Inclusion Predicate, on `orders`: ```text theme={null} value.get('quantity') > 1 ``` `{quantity: 3, ...}` passes; `{quantity: 1, ...}` is dropped. The same predicate is what a **Split** branch evaluates. ### Reach a nested field Map Values, on an Avro record whose `customer` is itself a record: ```text theme={null} {'cid': value.get('customer').get('id')} ``` `{customer: {id: 'c-1', ...}, ...}` becomes `{"cid": "c-1"}` at a JSON sink. A nested record placed into the output map, `{'c': value.get('customer')}`, lands as a JSON object. ### Rekey by a field Select Key, then a downstream filter on the new key: ```text theme={null} value.get('customerId').toString() // Select Key key == 'c-6650' // a Filter downstream ``` The `.toString()` turns Avro's `Utf8` into a `String` key; without it the key still matches `'c-6650'` and still reaches the sink, but as a `Utf8`. ### Map to a smaller record Map Values, on `orders`, keeping three fields and computing one: ```text theme={null} {'customer': value.get('customerId'), 'product': value.get('item'), 'total': value.get('quantity') * value.get('priceEur')} ``` `{orderId: 'o-1', customerId: 'c-6650', item: 'Kettle', quantity: 3, priceEur: 17.0}` becomes `{"customer": "c-6650", "product": "Kettle", "total": 51.0}`. Downstream, `value.get('total') > 50` passes it and `value.get('total') > 100` drops it; the editor knows the computed shape and completes `get('total')` for you, marked *(inferred)*. ### Several records from one FlatMap Values, an inline list: ```text theme={null} {value.get('item'), value.get('orderId')} ``` One order becomes two records, `"Kettle"` and `"o-1"`, with the same key. The word-count example's `value.toLowerCase().split('\\W+')` is the same recipe over a String value. ### Count or sum per key Aggregate over `orders`, grouped by key. A numeric accumulator: ```text theme={null} 0 // Initializer #aggValue + value.get('quantity') // Adder ``` Records with quantities 3 and 4 leave `7` in the store. A map-shaped accumulator, which is what you want when the aggregate carries more than one figure: ```text theme={null} {'total': 0} // Initializer {'total': #aggValue.get('total') + value.get('quantity')} // Adder ``` The initializer builds the shape; the adder reads it back with `get`. A String method on the accumulator, `#aggValue.substring(0, 3)`, fails deployed on a map-shaped aggregate, and the editor says so because it evaluates the adder against what your initializer really built. ### Concatenate with reduce Reduce over a String topic: ```text theme={null} value + '|' + #aggValue ``` `a`, `b`, `c` for one key reduce to `c|b|a`. Reduce keeps the value's type: it cannot change a String into a map, which is what Aggregate is for. ### Join two records Value Joiner, both sides Avro: ```text theme={null} #leftValue.get('item') + '/' + #rightValue.get('quantity') ``` gives `Kettle/4`. Mixed serdes read each side its own way: `#leftValue.get('item') + ':' + #rightValue` with a String right side gives `Kettle:warehouse-7`, and `#rightValue.get('item')` on that String right side fails, deployed and in the editor alike, once both join lanes are wired. Arithmetic between two String sides, `#leftValue - #rightValue`, fails. ### Foreign-key join Foreign Key Extractor, on the left table's value, then the joiner: ```text theme={null} value.substring(value.indexOf(':') + 1) // extractor on a String value like "3:Kettle" value.get('item').toString() // extractor on an Avro left table #leftValue + '@' + #rightValue // Value Joiner ``` `3:Kettle` joined to the right table's `Kettle` row gives `3:Kettle@warehouse-7`. The extractor sees only `value`: `key` there fails. ### Unwrap a windowed key Select Key after a windowed count, as the tumbling-window example does: ```text theme={null} key.key() + '@' + key.window().start() ``` Seven records of key `k1` over seven seconds in 5-second windows produce counts keyed `k1@0` and `k1@5000`, with the `Long` count as the value. ### Call your own Java Any expression, once the function is registered ([Custom functions](/streams/custom-functions)): ```text theme={null} {'masked': #maskEmail(value.get('customerId') + '@example.com')} ``` gives `{"masked": "c***@example.com"}`. A function that throws fails the expression deployed and in the editor; an unregistered name fails everywhere too. ### Peek without risk Peek, Action Expression: the result is discarded, and a throw is logged and swallowed, so a peek can never stop the stream. That makes it a place to call a registered function for its side effect, `#audit(key, value)`, and not a place for a guard: an expression that fails in a peek does not hold the record back. ## The forms that do not work Pinned as failures on every surface, so the editor reports them before you deploy: * `value['item']` on Avro or Protobuf: indexing is a `Map` operation and those are not maps. It works on JSON and JSON Schema. * `value.get('item')` on a String topic: a String has no fields. * `value.get('priceEur').asDouble()` on Avro: the field is already a number; there is no `asDouble()`. * `value.get('amount') > 10` on an Avro `decimal`: bytes do not compare as a number. * `#newValue` in an adder: the new record is `value`; `#newValue` is not bound. * `key` in a Map Values, `value` in an initializer, `key` in a foreign-key extractor.