Skip to main content

Serialisation overview

The code generated by Smithy4s is strictly protocol agnostic. One implication is that the data-types generated by Smithy4s are not tied to any particular serialisation format or third-party library. Instead, Smithy4s generates an instance of a smithy4s.schema.Schema for each data-type (see the relevant section). From this schema can be derived encoders and decoders for virtually any serialisation format.

Smithy4s provides opt-in modules implementing serialisation in a bunch of formats, including JSON, XML and Protobuf. The modules cross-compile to all combinations of platforms (JVM/JS/Native) and scala-versions supported by Smithy4s.

Document (JSON-like adt)​

The smithy4s-core module provides a smithy4s.Document datatype that is used in code-generation when document shapes are used in smithy. Document is effectively a JSON ADT, and can be easily converted to from other ADTs, such as the one provided by the Circe library.

Document also comes with its own Encoder and Decoder construct, for which instances can be derived for every datatype generated by Smithy4s.

import smithy4s.example.hello.Person
import smithy4s.Document

val personEncoder = Document.Encoder.fromSchema(Person.schema)
// personEncoder: Document.Encoder[Person] = smithy4s.Document$CachedEncoderCompilerImpl$$anon$1@3769a29
val personDocument = personEncoder.encode(Person(name = "John Doe"))
// personDocument: Document = DObject(
//   value = Map("name" -> DString(value = "John Doe"))
// )

val personDecoder = Document.Decoder.fromSchema(Person.schema)
// personDecoder: Document.Decoder[Person] = smithy4s.Document$Decoder$$anon$2@61fa0f32
val maybePerson = personDecoder.decode(personDocument)
// maybePerson: Either[smithy4s.codecs.PayloadError, Person] = Right(
//   value = Person(name = "John Doe", town = None)
// )

By default, smithy4s Documents abide by the same semantics as smithy4s-json (see section below).

It is worth noting that, although Document is isomorphic to a JSON ADT, its .toString is not valid JSON. Likewise, the smithy4s-core module does not contain logic to parse JSON strings into Documents. In order to read/write Documents from/to JSON strings, you need the smithy4s-json module. The smithy4s.json.Json entry-point contains methods that work with Documents.

JSON​

The smithy4s-json module provides jsoniter-based encoders/decoders that can read/write generated data-types from/to JSON bytes/strings, without an intermediate JSON ADT. The performance of this module is very competitive are very competitive in the Scala ecosystem.

This module is provided at the following coordinates :

sbt : "com.disneystreaming.smithy4s" %% "smithy4s-json" % "0.18.16"
mill : "com.disneystreaming.smithy4s::smithy4s-json:0.18.16"

The entrypoint for JSON parsing/writing is smithy4s.json.Json. See below for example usage.

import smithy4s.example.hello.Person

import smithy4s.Blob
import smithy4s.json.Json

val personEncoder = Json.payloadCodecs.encoders.fromSchema(Person.schema)
// personEncoder: smithy4s.codecs.package.PayloadEncoder[Person] = smithy4s.json.internals.JsonPayloadCodecCompilerImpl$$anon$1$$anonfun$fromSchema$2@5da5cf7b
val personJSON = personEncoder.encode(Person(name = "John Doe")).toUTF8String
// personJSON: String = "{\"name\":\"John Doe\"}"

val personDecoder = Json.payloadCodecs.decoders.fromSchema(Person.schema)
// personDecoder: smithy4s.codecs.package.PayloadDecoder[Person] = smithy4s.json.internals.JsonPayloadCodecCompilerImpl$JsonPayloadDecoder@79e8f4e2
val maybePerson = personDecoder.decode(Blob(personJSON))
// maybePerson: Either[smithy4s.codecs.PayloadError, Person] = Right(
//   value = Person(name = "John Doe", town = None)
// )

By default, smithy4s-json abides by the semantics of :

XML​

The smithy4s-xml module provides fs2-data encoders/decoders that can read/write generated data-types from/to XML bytes/strings. It is provided at the following coordinates :

sbt : "com.disneystreaming.smithy4s" %% "smithy4s-xml" % "0.18.16"
mill : "com.disneystreaming.smithy4s::smithy4s-xml:0.18.16"

The entrypoint for smithy4s.xml.Xml. See below for example usage.

import smithy4s.example.hello.Person

import smithy4s.Blob
import smithy4s.xml.Xml

val personEncoder = Xml.encoders.fromSchema(Person.schema)
// personEncoder: smithy4s.codecs.package.BlobEncoder[Person] = smithy4s.xml.internals.XmlPayloadEncoderCompilerImpl$$anonfun$fromSchema$2@69796c83
val personXML = personEncoder.encode(Person(name = "John Doe")).toUTF8String
// personXML: String = "<Person><name>John Doe</name></Person>"

val personDecoder = Xml.decoders.fromSchema(Person.schema)
// personDecoder: smithy4s.codecs.package.BlobDecoder[Person] = smithy4s.xml.Xml$$anon$1$$anon$2@58cb6d01
val maybePerson = personDecoder.decode(Blob(personXML))
// maybePerson: Either[smithy4s.codecs.PayloadError, Person] = Right(
//   value = Person(name = "John Doe", town = None)
// )

By default, smithy4s-xml abides by the semantics of :

Protobuf​

The smithy4s-protobuf module provides protocol-buffers codecs that can read/write generated data-types from protobuf-encoded bytes.

sbt : "com.disneystreaming.smithy4s" %% "smithy4s-protobuf" % "0.18.16"
mill : "com.disneystreaming.smithy4s::smithy4s-protobuf:0.18.16"

The entrypoint for XML parsing/writing is smithy4s.protobuf.Protobuf. See below for example usage.

import smithy4s.example.hello.Person
import smithy4s.protobuf.Protobuf

val personCodec = Protobuf.codecs.fromSchema(Person.schema)
// personCodec: smithy4s.protobuf.ProtobufCodec[Person] = smithy4s.protobuf.ProtobufCodec$$anon$1@2b66bf74
val personBytes = personCodec.writeBlob(Person(name = "John Doe"))
// personBytes: smithy4s.Blob = ArraySliceBlob(..., 0, 10)
val maybePerson = personCodec.readBlob(personBytes)
// maybePerson: Either[smithy4s.protobuf.ProtobufReadError, Person] = Right(
//   value = Person(name = "John Doe", town = None)
// )

By default, smithy4s-protobuf abides by the semantics of :

  • alloy protobuf traits. These semantics are the exact same semantics that smithy-translate uses to translate smithy to protobuf. This implies that the Smithy4s protobuf codecs are compatible with the codecs of other protobuf tools, generated from the .proto files resulting from running smithy through smithy-translate. In short, Smithy4s and ScalaPB can talk to each other : the ScalaPB codecs generated from protobuf after a translation from smithy are able to decode binary data produced by Smithy4s protobuf codecs (and vice versa).
┌────────────────────┐                        ┌────────────────────┐
│                    │                        │                    │
│                    │                        │                    │
│                    │                        │                    │
│                    │                        │                    │
│     Smithy IDL     ├────────────────────────►    Protobuf IDL    │
│                    │   smithy-translate     │                    │
│                    │                        │                    │
│                    │                        │                    │
│                    │                        │                    │
└─────────┬──────────┘                        └─────────┬──────────┘
          │                                             │
          │                                             │
          │                                             │
          │                                             │
          │                                             │
          │                                             │
          │ Smithy4s codegen                            │ ScalaPB codegen
          │                                             │
          │                                             │
          │                                             │
          │                                             │
          │                                             │
┌─────────▼──────────┐                        ┌─────────▼──────────┐
│                    │                        │                    │
│                    │                        │                    │
│                    │                        │                    │
│                    ◄────────────────────────┤                    │
│    Smithy4s code   │  Runtime communication │     ScalaPB code   │
│                    ├────────────────────────►                    │
│                    │                        │                    │
│                    │                        │                    │
│                    │                        │                    │
└────────────────────┘                        └────────────────────┘