Berry ZCN system manual

Berry ZCN template

This document describes the Berry side of the ZCN converter system: writing device-specific Zigbee Hub converters as Berry scripts, without rebuilding the firmware.

The concept of ZCN primarily refers to native C++ converters, but we decided to keep this name for Berry converters for simplicity. Actually Berry converters are not native, they are a scripting interface.

1. What Berry ZCN is and when to use it

A Berry ZCN converter is the runtime-scriptable twin of a native zcn_converter_t: a description of how a specific Zigbee device deviates from standard ZCL handling — extra/proprietary attributes, remapped clusters, custom value normalization, custom MQTT/Home Assistant discovery — plus Berry callback functions that the hub invokes at the same points where native converters invoke C function pointers.

Use Berry ZCN when you want to:

• Prototype a converter for a new device without a firmware rebuild — iterate directly in the on-device script editor, then port the result to a native converter for shipping.
• Support a device locally that the stock firmware does not have a converter for.
• Experiment with normalization/serialization/discovery logic against a live device.

Everything here requires the device to run in Zigbee Hub mode.

2. Architecture and lifecycle

2.1 Slots

Berry converters live in a fixed global slot array (converter_list.cpp):

There are ZCN_BE_COUNT (currently 2) slots for Berry converters. Each registered converter occupies one slot (id 0..ZCN_BE_COUNT-1). Registration fails with "no free slots for ZCN" when all slots are taken.

Each slot stores a zcn_converter_be_t — the C mirror of the native zcn_converter_t, but with bvalue Berry callbacks instead of C function pointers, heap-allocated strings, and a back-pointer to the owning Berry VM (converter_defines.h:194).

2.2 VM binding and garbage collection

• A converter belongs to the VM that registered it. ZCN._new() records the VM and marks the script as waiting (be_events_mark_waiting) so it stays resident to serve callbacks.
• All callback functions passed into the structure are GC-pinned (be_gc_fix_set) while the converter exists, and unpinned on delete — you can use closures/anonymous def blocks safely.
• When a script's VM stops, all its converters are automatically deleted. Restarting the script re-registers them.

2.3 Attachment is NOT persisted — attach on every boot

This is the most important lifecycle difference from native ZCN:

• The native converter index (dev->data.zcn) is saved in the device database (/zb/db/{IEEE}.json, key "zcn") and restored on boot.
• The Berry converter index (dev->data.zcn_be) is runtime-only. It resets to 0xff (unattached) every boot and is set only by:
  1. Interview matching — during a device interview, intw_zcn checks Berry converters and sets zcn_be when one matches;
  2. Explicit ZCN.attach(slot, ieee) from your script.

Since interviews only run at pairing (or forced re-interview), a converter script must call ZCN.attach() for its devices every time it starts. The standard pattern:

Optional, run the script with autostart (#META {"start":1}) so the converter is registered and attached right after boot.

2.4 Priority over native converters

Berry converters take priority everywhere:

• Interview matching (intw_zcn, standalone.cpp:3638): all Berry slots are checked first; if one matches, zcn_be is set and native matching is skipped. (If several Berry converters match, the highest slot id wins.)
• Runtime dispatch: every hook site checks ZCN_CHECK_BE(dev) before the native ZCN_CHECK(dev) — data handling, on_cmd, MQTT write/cmd, HA discovery, trigger discovery. A device can technically have both zcn and zcn_be set; the Berry one is consulted first wherever it provides an override.

2.5 Matching logic

Same rules as native (zcn_device_signature_t):

• If the converter has a matcher function — it is called as def (dev) with a ZigbeeDevice instance and must return true/false. model/manuf strings are ignored.
• Otherwise — exact string compare of both manuf (Basic attr 0x0004) and model (Basic attr 0x0005) against the interviewed values.

3. Quick start

The easiest way to produce a converter is the ZCN Builder page in the web UI (Zigbee Hub → ZCN Builder): assemble the structure in the form, pick callback presets, and copy the generated Berry code. What follows explains what that generated code actually does.

Minimal hand-written converter:

ZCN.register() is implemented in Berry and solidified into the firmware: it validates the map, finds a free slot, and feeds the structure into the native slot via the low-level ZCN._* functions. It returns the slot id (needed for ZCN.attach) and raises a descriptive error on failure (the partially built slot is deleted automatically).

4. Converter map reference

The converter is a plain Berry map. Missing keys get defaults: int → 0, string → "", bool → false, function → nil (exceptions noted below). Extra/unknown keys are ignored.

4.1 Top level

4.2 signature

The signature tells SLZB-OS whether this converter is suitable for the device for which the interview is currently being performed.

4.3 Endpoint override (element of overrides)

4.4 Cluster override (element of clusters)

4.5 cmd_config

Tells SLZB-OS what actions(triggers) this cluster provides. Typically used for Zigbee buttons and scene remotes.

Note: the command path is only taken when the cluster has on_cmd and at least one of exposes / exposesOverride.

4.6 Attribute override (element of attrs)

mqttClass values:

5. Callback reference

All callbacks run inside your script's VM, scheduled through the Berry event system. The hub task waits up to ~20 ms for the VM to become available; if the VM is busy longer (e.g. a blocked loop in your script), the event is dropped — keep both the callbacks and the rest of the script non-blocking.

dev arguments are ZigbeeDevice instances, record arguments are ZclAttrRecord instances (§6). doc/data are JSON proxy objects supporting obj["key"] = value style access.

5.1 on_annonce — def (dev)

Called when an attached device sends a device announce (typically power-on or rejoin). Return value ignored.

5.2 on_intw_stage — def (dev, tag) -> int

Called before interview stages, identified by string tag. Return one of:

Stage tags: req_ep, discover_ep, get_power_source, autobind, conf_reporting, ias_enroll, get_ias_type, send_tuya_magic.

Caveat (same as native): on the first interview the converter is matched at the intw_zcn stage, so tags for stages ordered before it (req_ep, discover_ep) only fire on re-interview — unless the device was pre-attached with ZCN.attach().

5.3 cmd_config.on_cmd — def (dev, record) -> string | nil

Called when a ZCL command arrives on the cluster (record.isCmd() is true; record.getAttr() holds the command id). Return the action string to publish (must be one of the exposes values for HA to recognize it). Non-string return = not handled.
Important for Tuya commands: record.asInt() contains the ID of the Tuya command and record.getAttr() its data.

5.4 on_normalize — def (record, dev)

Called right after the raw ZCL value is parsed into the record, before it is stored and serialized. Mutate the record in place:

You can also redirect a record to another cluster/attr (record.setCl(), record.setAttr(), record.setEp()) — e.g. unpacking a proprietary struct into standard records. The redirect target attr must exist (declare it with ram: true).

5.5 on_serialization — def (record, dev, data)

Called when the stored record is converted to JSON for MQTT / web dashboard. Write into data:

Keys follow the native zcl_json convention: "type_sm" (usually "report") and "val". If absent, standard serialization for the data type applies.

5.6 on_discovery — def (ep, cl, attr, zType, dev, doc, zhbBase) -> bool

Called when generating discovery payload for this attribute. doc is the discovery JSON (base payload already filled from mqttClass/mqttSubClass/name/unit); zhbBase is the MQTT base topic. Add or override keys, e.g. doc["state_class"] = "measurement". In most cases, you only use doc if you need to add something (like min, max and step for a slider).

Return true to publish, false to veto discovery of this attribute. Note: with no callback set the attribute is still discovered when mqttClass != 0; but if you do set a callback, you must return true for it to be published.

5.7 on_mqtt_write — def (ep, cl, attr, dType, topic, data, dev) -> bool

Called for MQTT messages on the {base}/write/... topic targeting this attribute. data is the raw payload string. Convert and send to the device (dev.sendCmd, dev.sendTuyaData, etc.). Return true = handled (standard ZCL write skipped).

5.8 on_mqtt_cmd (cluster level) — def (ep, cl, cmd, topic, data, dev) -> bool

Same idea for the {base}/cmd/... topic — cluster commands sent from MQTT (e.g. switch toggles). Return true = handled.

6. Objects available in callbacks

6.1 ZigbeeDevice (class be_class_ZigbeeDevice)

6.2 ZclAttrRecord (class be_class_ZclAttrRecord)

Getters: getEp(), getCl(), getAttr(), getStatus(), getZtype() (raw ZCL type), getSMtype() (internal storage type, compare against ZclAttrRecord.TYPE_* constants: TYPE_NONE/U32/I32/FLOAT/BUF/STR/BOOL/CMD/CMD_PAYLOAD).

Predicates: isCmd(), isCmdPayload(), isTuya() (cluster == 0xEF00), hasPayload() (type is not TYPE_NONE), matcher(cl, attr [, ep]) -> bool.

Value access: asInt(), asFloat(), asStr(), asBytes(), asLumiStruct([start]) — parses the Lumi/Aqara proprietary struct (Basic 0xFF01/0xFF02 buffer) into a {tag_id: int} map; getMSB16() / getLSB16() — high/low 16-bit halves of the raw 32-bit value (e.g. packed color X/Y).

Mutation: setInt(v), setUInt(v), setFloat(v), setBool(v), setStr(v), setBuf(bytes) (replaces the payload with a copy of a Berry bytes() buffer, 1–255 bytes), setCmd(cmdId [, tuyaCluster]), setMSB16(v) / setLSB16(v), setEp(v), setCl(v), setAttr(v).

7. ZCNP — native callback presets

The ZCNP module exposes the preset functions used by built-in converters, so a Berry converter can reference them directly instead of re-implementing the logic (and automatically picks up native fixes):

8. ZCN module API

Public functions:

The ZCN._* functions (_new, _add_signature, _add_ep_ov_info, _add_ep, _add_cl, _add_cl_cmd_config, _add_cl_attr_1/2/3, _is_zcn_ex, _get_slot_count, _suspend_zhb) are the low-level building blocks used by ZCN.register(). Use the map + ZCN.register() instead of calling them directly — they must be called in the exact allocation order and with the ZHB task suspended.

9. Worked example — Aqara water leak sensor

lumi.sensor_wleak.aq1 reports leak state via IAS Zone and battery/telemetry via the proprietary Lumi struct in Basic attr 0xFF01:

A fully custom (no-preset) variant of the struct unpacking would use record.asLumiStruct() in on_normalize and redirect values with record.setCl()/record.setAttr()/record.setFloat() — the ZCN Builder page contains this and five more complete examples ported from native converters.

10. Gotchas

1. Counts are explicit. overrides_count, clusterCount, attrCount are read from the map, not derived from list sizes. A count smaller than the list silently drops entries; a count larger than the list reads will crash device.
2. Attach every boot. zcn_be is not saved to the device DB. No ZCN.attach() after reboot (and no fresh interview) = converter silently inactive.
3. signature is mandatory — registration raises without it. matcher must be a function if present.
4. Callbacks must be fast. The hub waits ≤ ~20 ms for your VM; a busy script drops the event (a lost report/command, not an error).
5. on_cmd needs exposes. Without exposes or exposesOverride on the same cluster, the command path is never taken.
6. on_discovery must return true for the attribute to be discovered — a forgotten return (nil) vetoes it.
7. ram: true for redirect targets and dashboard-only values — otherwise the record does not exist until the device first reports it (or ever, for synthetic attrs).
8. Slot exhaustion. Only ZCN_BE_COUNT Berry converters can exist at once, across all scripts. A crashed-then-restarted script frees and re-takes its slots automatically, but a script that registers in a loop will run out.
9. ZCN.attach requires a running hub and a paired device — it raises "enable zHub first" / "wrong device ieee" otherwise. If your script autostarts before the hub is up, wait via ZHB.waitForStart() first.
10. Multiple matching converters: Not recommended. Converters are meant to be unique.
11. Berry converters are much slower than native ones! Berry ZCN intended for rapid prototyping, not for continuous use as it will slow down the system.