MQTT Payload Design for IIoT: JSON vs Sparkplug B

An MQTT topic answers where a message is routed; its payload must answer what the bytes mean, when the measurement applied, and whether a consumer may use it. MQTT 5.0 deliberately leaves that application contract open. It provides PUBLISH metadata such as Payload Format Indicator, Content Type, and User Properties, but it does not define an IIoT JSON object, a unit system, or a quality model [C-A-S-003] [C-A-P-002]. A durable payload design therefore starts with semantics before serialization.

This article owns the payload boundary: envelopes, scalar and structured values, timestamps, quality, units, validation, and change control. It compares JSON with a pinned Sparkplug/Protobuf vector without claiming a universal byte, throughput, security, or interoperability result. Namespace, birth/death, retained lifecycle state, and spBv1.0 message rules belong to the companion Sparkplug lifecycle article; use it when the system needs that application protocol rather than merely a well-defined JSON contract.

Start with an envelope a receiver can interpret

An envelope separates transport context from the measurement itself. It should let a receiver identify the schema, identify the producing source, determine the measurement time, and decide whether the value is usable. The smallest useful shape depends on the system, but the fields need unambiguous definitions. For example:

{
  "schema": "com.example.iiot.telemetry/1",
  "source": "plant-a/line-2/press-07",
  "observedAt": "2023-11-14T22:13:20.000Z",
  "sequence": 7,
  "metrics": {
    "temperature": { "value": 24.5, "type": "number", "unit": "Cel", "quality": "good" }
  }
}

schema is an application identifier, not an MQTT feature. source identifies the producing contract and must not be assumed trustworthy merely because it appears in the body; deployment authorization should bind publisher identity and permitted topic/source combinations. observedAt is the time the source observed the value, whereas broker receipt time is a separate operational fact. sequence can help detect gaps or duplicates within one producer epoch, but it is not a globally ordered event ID.

Keep fields orthogonal. Do not overload timestamp to mean both sampled time and upload time, or status to mean communication health, engineering validity, and alarm severity. If a gateway needs both source and gateway timing, publish named fields such as observedAt and ingestedAt, each with a documented clock and precision. ISO 8601 UTC strings are readable in JSON; an integer epoch can be compact and avoids timezone parsing, but its unit and range must be explicit. Either choice needs a clock-synchronization and out-of-order policy.

Preserve type, unit, and quality as data

A scalar telemetry value is one measurement with one value: temperature, valve position, or a counter. A structured value represents a coherent object: a vibration spectrum, three-axis acceleration, a batch result, or a device configuration snapshot. Do not flatten structured telemetry into invented names such as axis_1_42 when the shape matters to consumers. Conversely, do not wrap every scalar in a large object if a documented scalar contract is enough.

JSON has only number, string, boolean, null, array, and object types. A JSON number does not distinguish a 32-bit integer from a 64-bit counter, a decimal quantity from a floating-point approximation, or a code that only looks numeric. A JavaScript consumer can lose integer precision above its safe-integer range; another decoder may retain a wider integer. For identifiers, sequence values with a defined range, money-like values, and values requiring exact decimals, state the representation explicitly and test the actual producer and consumer libraries. A quoted decimal may be safer for exact transfer, but then consumers need validation and conversion rules.

The type field in the example makes the intended meaning visible, but a schema can also make it implicit. Choose one approach and document it. A numeric engineering value needs a unit such as Cel, kPa, rpm, or a locally governed code. A bare key named temperature is not enough: different components can legitimately use Celsius, Fahrenheit, or a raw sensor count. Adopt a naming convention that describes the quantity rather than the display label, publish the unit beside the value or in an immutable schema, and define conversion ownership. Do not silently convert at multiple hops.

Quality answers a different question from type. good, uncertain, and bad are useful only when their meanings and transitions are documented. A quality value can convey sensor failure, substituted data, a communication timeout, or an out-of-range check, but those are not interchangeable. Include an optional bounded reason code where operations need diagnosis; do not expose stack traces, credentials, personal data, or unrestricted device errors in a telemetry payload. Consumers should make a conscious decision about whether a non-good value updates a display, historian, calculation, or control workflow.

Make absent, null, and stale distinct states

Missing, null, and stale values are three different signals. A missing metric can mean “this producer version does not support it,” “this update does not change it,” or “the producer omitted it incorrectly.” null can mean “known but unavailable,” but only if the schema explicitly reserves that meaning. Stale usually means the last value was valid when observed but is now older than the receiving application’s freshness budget. It should be derived from observedAt, receipt time, and a documented timeout rather than inferred from an arbitrary zero or empty string.

For a partial-update topic, omission may correctly mean “unchanged.” For a snapshot topic, omission may mean “not present in this snapshot.” Those contracts cannot share one decoder rule. Name the message mode or schema accordingly, and keep retained operational state separate from an event stream. A retained snapshot may help a late subscriber learn the last published representation; it is not evidence that the measurement is still fresh. An event records something that happened once and should carry an event identifier, occurrence time, and retention policy appropriate to that fact. A state message represents the latest known condition and needs expiry or freshness handling.

This distinction prevents a common failure: treating a broker-delivered retained JSON document as live process truth. A receiver must apply its own freshness rule even when QoS delivered the message successfully. MQTT QoS governs a client-to-broker delivery interaction; it does not certify sensor validity or create exactly-once business processing. Design duplicate handling and stale-data behavior at the application boundary.

Evolve schemas without making old consumers guess

Version the contract from the first message. A major version in schema, such as com.example.iiot.telemetry/1, gives consumers a clear compatibility boundary. Additive optional fields can be compatible only if old consumers ignore unknown fields and new consumers have defaults for absent fields. Renaming a key, changing its unit, changing scalar to object, changing a number to string, or assigning new meaning to null is a semantic change even when the JSON still parses. Publish a new major schema or topic and support a deliberate migration window.

Canonicalization is equally important when a payload is signed, hashed, deduplicated, or compared in tests. JSON object member order is not semantic, whitespace is not a reliable identity, and number formatting can vary between encoders. Define the exact canonicalization algorithm before using a JSON byte sequence as an identifier. Otherwise hash a stable application representation rather than pretending arbitrary JSON bytes are equivalent. The deterministic receipt below deliberately uses sorted keys and compact separators for this one fixture; it does not establish a production-wide canonicalization standard.

Keep a schema registry or versioned repository where producers and consumers can inspect required fields, allowed types, ranges, unit rules, nullability, examples, and deprecation dates. A human-readable schema document is useful, but a machine-enforced validator at the trust boundary is what rejects malformed input before it enters a historian, dashboard, or control-adjacent workflow.

Validate in stages. First reject a message whose MQTT topic, authenticated publisher, Content Type, or declared schema is not permitted for that intake. Then apply cheap byte and depth limits before JSON parsing, validate the parsed structure against the selected schema, and finally apply semantic checks such as permitted units, plausible ranges, monotonic counters, and source-specific freshness. Validation failures should produce a bounded reason code and metric, not an exception dump containing the payload. A rejected message must not overwrite the last known good state; whether it is quarantined, dropped, or retried is an explicit operational policy.

Measure one vector; do not generalize it

JSON repeats names and punctuation, while Protobuf encodes fields by numeric tags and typed wire values. That difference can matter on constrained links, but the result depends on field names, values, optional metadata, batching, compression, and the actual encoder. The following self-contained receipt compares exactly one compact JSON object with one hand-constructed vector based on the pinned Eclipse Tahu schema. It is intentionally an offline byte calculation, not a broker or device test [C-A-P-001] [C-A-P-003].

Environment: Python 3.14.6; date: 2026-07-22; receipt: V-A-P-001 [V-A-P-001].

python3 - <<'PY'
import hashlib,json,struct
def v(n):
 b=bytearray()
 while n>127:b.append((n&127)|128);n>>=7
 b.append(n);return bytes(b)
def out(n,b,h=False):print(f'{n} bytes={len(b)}'+(f' hex={b.hex()}' if h else '')+f' sha256={hashlib.sha256(b).hexdigest()}')
j=json.dumps({'metrics':[{'name':'temp_c','type':'Double','value':24.5}],'seq':7,'timestamp':1700000000000},sort_keys=True,separators=(',',':')).encode()
m=b'\x0a'+v(6)+b'temp_c'+b'\x20'+v(10)+b'\x69'+struct.pack('<d',24.5)
s=b'\x08'+v(1700000000000)+b'\x12'+v(len(m))+m+b'\x18'+v(7)
out('json',j);out('sparkplug',s,True)
PY

Expected and observed output, on two identical runs:

json bytes=94 sha256=b8e0e19bcc6ba98c5eb6c7b1a2b20b50f101e824f6fffedbd9caf0409a24462b
sparkplug bytes=30 hex=0880d095ffbc3112130a0674656d705f63200a6900000000008038401807 sha256=c06d68e146bab6ab68b7fc2bd2d7669f7b0cc20e4ab1d33f32496858ef97f036

The SHA-256 of this two-line stdout without its trailing newline is fe1690d6f555caf6b6c7ad886bfb97ea7fb2ef236943c06be94930d88074246f. The receipt demonstrates only its stated input and encoder logic. Adding unit, quality, IDs, repeated metrics, a different JSON spelling, or a generated Sparkplug library changes the bytes. Compression may reduce repeated data in some transport paths, but it has CPU, latency, observability, and compatibility tradeoffs; measure the negotiated system rather than layering compression by default.

Batch deliberately and preserve message boundaries

Batching several measurements can amortize MQTT topic and framing overhead, but it increases the loss and retry unit, peak memory, decoding latency, and ambiguity when one record is invalid. A batch should state whether every item has its own observedAt, whether ordering is meaningful, how duplicates are identified, and whether the consumer may process a valid subset. Limit item count, decoded bytes, nesting depth, string length, and processing time before parsing untrusted data. Reject or quarantine oversized and malformed input with bounded diagnostics; do not log entire payloads if they can contain sensitive operational or personal information. Privacy review should classify sources and fields before publication, then set retention and access rules for diagnostics as well as telemetry storage.

Compression is a separate decision from serialization. It can help when the payload is large and repetitive enough to justify it, but small messages may not benefit after headers and implementation cost. It also makes inspection harder and can expose resource-exhaustion risks if a receiver decompresses unbounded input. Enforce compressed and decompressed size ceilings, authenticate and authorize publishers, use TLS for broker connections, and review topic permissions. Payload validation does not replace broker authorization; broker authorization does not replace payload validation.

Choose a protocol, then keep the boundary observable

JSON is often a good choice when teams need inspectable messages, broadly available tooling, and a contract they can validate clearly. A structured binary protocol can be appropriate when its schema, generated implementations, and lifecycle semantics match the participating system. Sparkplug adds its own agreed namespace and lifecycle model; this article does not repeat those rules. The useful choice is not “JSON versus binary” in the abstract, but a documented contract whose consumers can reject safely, evolve predictably, and diagnose in production.

At each receiver, record bounded metadata: schema version, source identity, payload size, validation result, freshness decision, and a correlation or sequence value where permitted. Avoid treating raw payload logs as a default diagnostic store. Create migration adapters that read the old schema, validate it, emit the new schema, and expose a count of rejected or ambiguous records. Run both paths against representative fixtures before retiring the old one; do not relabel a topic or mutate a unit in place.

References and verification

• [C-A-S-003] OASIS, MQTT Version 5.0, published 2019-03-07.
• [C-A-P-001] Eclipse Tahu, pinned sparkplug_b.proto, commit 5736e404889d4b95910613040a99ba79589ffb13.
• [C-A-P-002] MQTT payload-envelope boundary derived from the MQTT 5.0 PUBLISH contract above; JSON names, units, quality, and evolution remain application contracts.
• [C-A-P-003] V-A-P-001, local deterministic vector only; no performance, compatibility, security, broker, or universal-size conclusion.
• [V-A-P-001] Exact local deterministic receipt above; Python 3.14.6; executed twice on 2026-07-22.
• [C-A-S-001] Eclipse Sparkplug, Specification 3.0, ratification ballot concluded 2022-10-21.

Primary-source research receipts: Tavily payload boundary request 773f3b12-f6bf-48fc-a33e-07571c34369c and Sparkplug boundary request acb84e15-ec4a-40f7-9486-000e54a1b296, accessed 2026-07-22. Context7 attempts for Eclipse Paho MQTT Python and Eclipse Tahu returned Monthly quota exceeded. Create a free API key at https://context7.com/dashboard for more requests. Official OASIS, Eclipse Sparkplug, and pinned Eclipse Tahu sources are used instead.

Last verified: 2026-07-22.