This is the third chapter of the Kotlin Serialization Guide. In this chapter we'll take a look at serializers in more detail, and we'll see how custom serializers can be written.
Table of contents
Formats, like JSON, control the encoding of an object into specific output bytes, but how the object is decomposed into its constituent properties is controlled by a serializer. So far we've been using automatically-derived serializers by using the @Serializable annotation as explained in the Serializable classes section, or using builtin serializers that were shown in the Builtin classes section.
As a motivating example, let us take the following Color class with an integer value storing its rgb bytes.
@Serializable class Color(val rgb: Int) fun main() { val green = Color(0x00ff00) println(Json.encodeToString(green)) }
You can get the full code here.
By default this class serializes its rgb property into JSON.
{"rgb":65280}
Every class marked with the @Serializable annotation, like the Color class from the previous example, gets an instance of the KSerializer interface automatically generated by the Kotlin Serialization compiler plugin. We can retrieve this instance using the .serializer() function on the class's companion object.
We can examine its descriptor property that describes the structure of the serialized class. We'll learn more details about that in the upcoming sections.
fun main() { val colorSerializer: KSerializer<Color> = Color.serializer() println(colorSerializer.descriptor) }
You can get the full code here.
Color(rgb: kotlin.Int)
This serializer is automatically retrieved and used by the Kotlin Serialization framework when the Color class is itself serialized, or when it is used as a property of other classes.
You cannot define your own function
serializer()on a companion object of a serializable class.
For generic classes, like the Box class shown in the Generic classes section, the automatically generated .serializer() function accepts as many parameters as there are type parameters in the corresponding class. These parameters are of type KSerializer, so the actual type argument's serializer has to be provided when constructing an instance of a serializer for a generic class.
@Serializable @SerialName("Box") class Box<T>(val contents: T) fun main() { val boxedColorSerializer = Box.serializer(Color.serializer()) println(boxedColorSerializer.descriptor) }
You can get the full code here.
As we can see, a serializer was instantiated to serialize a concrete Box<Color>.
Box(contents: Color)
The serializers for the primitive builtin classes can be retrieved using .serializer() extensions.
fun main() { val intSerializer: KSerializer<Int> = Int.serializer() println(intSerializer.descriptor) }
You can get the full code here.
Builtin collection serializers, when needed, must be explicitly constructed using the corresponding functions ListSerializer(), SetSerializer(), MapSerializer(), etc. These classes are generic, so to instantiate their serializer we must provide the serializers for the corresponding number of their type parameters. For example, we can produce a serializer for a List<String> in the following way.
fun main() { val stringListSerializer: KSerializer<List<String>> = ListSerializer(String.serializer()) println(stringListSerializer.descriptor) }
You can get the full code here.
When in doubt, you can always use the top-level generic serializer<T>() function to retrieve a serializer for an arbitrary Kotlin type in your source-code.
@Serializable @SerialName("Color") class Color(val rgb: Int) fun main() { val stringToColorMapSerializer: KSerializer<Map<String, Color>> = serializer() println(stringToColorMapSerializer.descriptor) }
You can get the full code here.
A plugin-generated serializer is convenient, but it may not produce the JSON we want for such a class as Color. Let's study alternatives.
We want to serialize the Color class as a hex string with the green color represented as "00ff00". To achieve this, we write an object that implements the KSerializer interface for the Color class.
object ColorAsStringSerializer : KSerializer<Color> { override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Color", PrimitiveKind.STRING) override fun serialize(encoder: Encoder, value: Color) { val string = value.rgb.toString(16).padStart(6, '0') encoder.encodeString(string) } override fun deserialize(decoder: Decoder): Color { val string = decoder.decodeString() return Color(string.toInt(16)) } }
Serializer has three required pieces.
The serialize function implements SerializationStrategy. It receives an instance of Encoder and a value to serialize. It uses the encodeXxx functions of Encoder to represent a value as a sequence of primitives. There is an encodeXxx for each primitive type supported by serialization. In our example, encodeString is used.
The deserialize function implements DeserializationStrategy. It receives an instance of Decoder and returns a deserialized value. It uses the decodeXxx functions of Decoder, which mirror the corresponding functions of Encoder. In our example decodeString is used.
The descriptor property must faithfully explain what exactly the encodeXxx and decodeXxx functions do so that a format implementation knows in advance what encoding/decoding methods they call. Some formats might also use it to generate a schema for the serialized data. For primitive serialization, the PrimitiveSerialDescriptor function must be used with a unique name of the type that is being serialized. PrimitiveKind describes the specific encodeXxx/decodeXxx method that is being used in the implementation.
When the
descriptordoes not correspond to the encoding/decoding methods, then the behavior of the resulting code is unspecified, and may arbitrarily change in future updates.
The next step is to bind a serializer to a class. This is done with the @Serializable annotation by adding the with property value.
@Serializable(with = ColorAsStringSerializer::class) class Color(val rgb: Int)
Now we can serialize the Color class as we did before.
fun main() { val green = Color(0x00ff00) println(Json.encodeToString(green)) }
You can get the full code here.
We get the serial representation as the hex string we wanted.
"00ff00"
Deserialization is also straightforward because we implemented the deserialize method.
@Serializable(with = ColorAsStringSerializer::class) class Color(val rgb: Int) fun main() { val color = Json.decodeFromString<Color>("\"00ff00\"") println(color.rgb) // prints 65280 }
You can get the full code here.
It also works if we serialize or deserialize a different class with Color properties.
@Serializable(with = ColorAsStringSerializer::class) data class Color(val rgb: Int) @Serializable data class Settings(val background: Color, val foreground: Color) fun main() { val data = Settings(Color(0xffffff), Color(0)) val string = Json.encodeToString(data) println(string) require(Json.decodeFromString<Settings>(string) == data) }
You can get the full code here.
Both Color properties are serialized as strings.
{"background":"ffffff","foreground":"000000"}
In the previous example, we represented the Color class as a string. String is considered to be a primitive type, therefore we used PrimitiveClassDescriptor and specialized encodeString method. Now let's see what our actions would be if we have to serialize Color as another non-primitive type, let's say IntArray.
An implementation of KSerializer for our original Color class is going to perform a conversion between Color and IntArray, but delegate the actual serialization logic to the IntArraySerializer using encodeSerializableValue and decodeSerializableValue.
import kotlinx.serialization.builtins.IntArraySerializer class ColorIntArraySerializer : KSerializer<Color> { private val delegateSerializer = IntArraySerializer() override val descriptor = SerialDescriptor("Color", delegateSerializer.descriptor) override fun serialize(encoder: Encoder, value: Color) { val data = intArrayOf( (value.rgb shr 16) and 0xFF, (value.rgb shr 8) and 0xFF, value.rgb and 0xFF ) encoder.encodeSerializableValue(delegateSerializer, data) } override fun deserialize(decoder: Decoder): Color { val array = decoder.decodeSerializableValue(delegateSerializer) return Color((array[0] shl 16) or (array[1] shl 8) or array[2]) } }
Note that we can't use default Color.serializer().descriptor here because formats that rely on the schema may think that we would call encodeInt instead of encodeSerializableValue. Neither we can use IntArraySerializer().descriptor directly — otherwise, formats that handle int arrays specially can't tell if value is really a IntArray or a Color. Don't worry, this optimization would still kick in when serializing actual underlying int array.
Example of how format can treat arrays specially is shown in the formats guide.
Now we can use the serializer:
@Serializable(with = ColorIntArraySerializer::class) class Color(val rgb: Int) fun main() { val green = Color(0x00ff00) println(Json.encodeToString(green)) }
As you can see, such array representation is not very useful in JSON, but may save some space when used with a ByteArray and a binary format.
You can get the full code here.
[0,255,0]
Now our challenge is to get Color serialized so that it is represented in JSON as if it is a class with three properties—r, g, and b—so that JSON encodes it as an object. The easiest way to achieve this is to define a surrogate class mimicking the serialized form of Color that we are going to use for its serialization. We also set the SerialName of this surrogate class to Color. Then if any format uses this name the surrogate looks like it is a Color class. The surrogate class can be private, and can enforce all the constraints on the serial representation of the class in its init block.
@Serializable @SerialName("Color") private class ColorSurrogate(val r: Int, val g: Int, val b: Int) { init { require(r in 0..255 && g in 0..255 && b in 0..255) } }
An example of where the class name is used is shown in the Custom subclass serial name section in the chapter on polymorphism.
Now we can use the ColorSurrogate.serializer() function to retrieve a plugin-generated serializer for the surrogate class.
We can use the same approach as in delegating serializer, but this time, we are fully reusing an automatically generated SerialDescriptor for the surrogate because it should be indistinguishable from the original.
object ColorSerializer : KSerializer<Color> { override val descriptor: SerialDescriptor = ColorSurrogate.serializer().descriptor override fun serialize(encoder: Encoder, value: Color) { val surrogate = ColorSurrogate((value.rgb shr 16) and 0xff, (value.rgb shr 8) and 0xff, value.rgb and 0xff) encoder.encodeSerializableValue(ColorSurrogate.serializer(), surrogate) } override fun deserialize(decoder: Decoder): Color { val surrogate = decoder.decodeSerializableValue(ColorSurrogate.serializer()) return Color((surrogate.r shl 16) or (surrogate.g shl 8) or surrogate.b) } }
We bind the ColorSerializer serializer to the Color class.
@Serializable(with = ColorSerializer::class) class Color(val rgb: Int)
Now we can enjoy the result of serialization for the Color class.
You can get the full code here.
{"r":0,"g":255,"b":0}
There are some cases where a surrogate solution does not fit. Perhaps we want to avoid the performance implications of additional allocation, or we want a configurable/dynamic set of properties for the resulting serial representation. In these cases we need to manually write a class serializer which mimics the behaviour of a generated serializer.
object ColorAsObjectSerializer : KSerializer<Color> {
Let's introduce it piece by piece. First, a descriptor is defined using the buildClassSerialDescriptor builder. The element function in the builder DSL automatically fetches serializers for the corresponding fields by their type. The order of elements is important. They are indexed starting from zero.
override val descriptor: SerialDescriptor = buildClassSerialDescriptor("Color") { element<Int>("r") element<Int>("g") element<Int>("b") }
The "element" is a generic term here. What is an element of a descriptor depends on its SerialKind. Elements of a class descriptor are its properties, elements of a enum descriptor are its cases, etc.
Then we write the serialize function using the encodeStructure DSL that provides access to the CompositeEncoder in its block. The difference between Encoder and CompositeEncoder is the latter has encodeXxxElement functions that correspond to the encodeXxx functions of the former. They must be called in the same order as in the descriptor.
override fun serialize(encoder: Encoder, value: Color) = encoder.encodeStructure(descriptor) { encodeIntElement(descriptor, 0, (value.rgb shr 16) and 0xff) encodeIntElement(descriptor, 1, (value.rgb shr 8) and 0xff) encodeIntElement(descriptor, 2, value.rgb and 0xff) }
The most complex piece of code is the deserialize function. It must support formats, like JSON, that can decode properties in an arbitrary order. It starts with the call to decodeStructure to get access to a CompositeDecoder. Inside it we write a loop that repeatedly calls decodeElementIndex to decode the index of the next element, then we decode the corresponding element using decodeIntElement in our example, and finally we terminate the loop when CompositeDecoder.DECODE_DONE is encountered.
override fun deserialize(decoder: Decoder): Color = decoder.decodeStructure(descriptor) { var r = -1 var g = -1 var b = -1 while (true) { when (val index = decodeElementIndex(descriptor)) { 0 -> r = decodeIntElement(descriptor, 0) 1 -> g = decodeIntElement(descriptor, 1) 2 -> b = decodeIntElement(descriptor, 2) CompositeDecoder.DECODE_DONE -> break else -> error("Unexpected index: $index") } } require(r in 0..255 && g in 0..255 && b in 0..255) Color((r shl 16) or (g shl 8) or b) }
Now we bind the resulting serializer to the Color class and test its serialization/deserialization.
@Serializable(with = ColorAsObjectSerializer::class) data class Color(val rgb: Int) fun main() { val color = Color(0x00ff00) val string = Json.encodeToString(color) println(string) require(Json.decodeFromString<Color>(string) == color) }
You can get the full code here.
As before, we got the Color class represented as a JSON object with three keys:
{"r":0,"g":255,"b":0}
The implementation of the deserialize function from the previous section works with any format. However, some formats either always store all the complex data in order, or only do so sometimes (JSON always stores collections in order). With these formats the complex protocol of calling decodeElementIndex in the loop is not needed, and a faster implementation can be used if the CompositeDecoder.decodeSequentially function returns true. The plugin-generated serializers are actually conceptually similar to the below code.
override fun deserialize(decoder: Decoder): Color = decoder.decodeStructure(descriptor) { var r = -1 var g = -1 var b = -1 if (decodeSequentially()) { // sequential decoding protocol r = decodeIntElement(descriptor, 0) g = decodeIntElement(descriptor, 1) b = decodeIntElement(descriptor, 2) } else while (true) { when (val index = decodeElementIndex(descriptor)) { 0 -> r = decodeIntElement(descriptor, 0) 1 -> g = decodeIntElement(descriptor, 1) 2 -> b = decodeIntElement(descriptor, 2) CompositeDecoder.DECODE_DONE -> break else -> error("Unexpected index: $index") } } require(r in 0..255 && g in 0..255 && b in 0..255) Color((r shl 16) or (g shl 8) or b) }
You can get the full code here.
Sometimes an application has to work with an external type that is not serializable. Let us use java.util.Date as an example. As before, we start by writing an implementation of KSerializer for the class. Our goal is to get a Date serialized as a long number of milliseconds following the approach from the Primitive serializer section.
In the following sections any kind of
Dateserializer would work. For example, if we wantDateto be serialized as an object, we would use an approach from the Composite serializer via surrogate section.
See also Deriving external serializer for another Kotlin class (experimental)
when you need to serialize a 3rd-party Kotlin class that could have been serializable, but is not.
object DateAsLongSerializer : KSerializer<Date> { override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Date", PrimitiveKind.LONG) override fun serialize(encoder: Encoder, value: Date) = encoder.encodeLong(value.time) override fun deserialize(decoder: Decoder): Date = Date(decoder.decodeLong()) }
We cannot bind the DateAsLongSerializer serializer to the Date class with the @Serializable annotation because we don't control the Date source code. There are several ways to work around that.
All encodeToXxx and decodeFromXxx functions have an overload with the first serializer parameter. When a non-serializable class, like Date, is the top-level class being serialized we can use those.
fun main() { val kotlin10ReleaseDate = SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00") println(Json.encodeToString(DateAsLongSerializer, kotlin10ReleaseDate)) }
You can get the full code here.
1455494400000
When a property of a non-serializable class, like Date, is serialized as part of a serializable class we must supply its serializer or the code will not compile. This is accomplished using the @Serializable annotation on the property.
@Serializable class ProgrammingLanguage( val name: String, @Serializable(with = DateAsLongSerializer::class) val stableReleaseDate: Date ) fun main() { val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00")) println(Json.encodeToString(data)) }
You can get the full code here.
The stableReleaseDate property is serialized with the serialization strategy that we specified for it:
{"name":"Kotlin","stableReleaseDate":1455494400000}
A serializer for a specific type, like Date, can be specified for a whole source code file with the file-level UseSerializers annotation at the beginning of the file.
@file:UseSerializers(DateAsLongSerializer::class)
Now a Date property can be used in a serializable class without additional annotations.
@Serializable class ProgrammingLanguage(val name: String, val stableReleaseDate: Date) fun main() { val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00")) println(Json.encodeToString(data)) }
You can get the full code here.
{"name":"Kotlin","stableReleaseDate":1455494400000}
kotlinx.serialization tends to be the always-explicit framework when it comes to serialization strategies: normally, they should be explicitly mentioned in @Serializable annotation. Therefore, we do not provide any kind of global serializer configuration (except for context serializer mentioned later).
However, in projects with a large number of files and classes, it may be too cumbersome to specify @file:UseSerializers every time, especially for classes like Date or Instant that have a fixed strategy of serialization across the project. For such cases, it is possible to specify serializers using typealiases, as they preserve annotations, including serialization-related ones:
typealias DateAsLong = @Serializable(DateAsLongSerializer::class) Date typealias DateAsText = @Serializable(DateAsSimpleTextSerializer::class) Date
Using these new different types, it is possible to serialize a Date differently without additional annotations:
@Serializable class ProgrammingLanguage(val stableReleaseDate: DateAsText, val lastReleaseTimestamp: DateAsLong) fun main() { val format = SimpleDateFormat("yyyy-MM-ddX") val data = ProgrammingLanguage(format.parse("2016-02-15+00"), format.parse("2022-07-07+00")) println(Json.encodeToString(data)) }
You can get the full code here.
{"stableReleaseDate":"2016-02-15","lastReleaseTimestamp":1657152000000}
Let us take a look at the following example of the generic Box<T> class. It is marked with @Serializable(with = BoxSerializer::class) as we plan to have a custom serialization strategy for it.
@Serializable(with = BoxSerializer::class) data class Box<T>(val contents: T)
An implementation of KSerializer for a regular type is written as an object, as we saw in this chapter's examples for the Color type. A generic class serializer is instantiated with serializers for its generic parameters. We saw this in the Plugin-generated generic serializer section. A custom serializer for a generic class must be a class with a constructor that accepts as many KSerializer parameters as the type has generic parameters. Let us write a Box<T> serializer that erases itself during serialization, delegating everything to the underlying serializer of its data property.
class BoxSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<Box<T>> { override val descriptor: SerialDescriptor = dataSerializer.descriptor override fun serialize(encoder: Encoder, value: Box<T>) = dataSerializer.serialize(encoder, value.contents) override fun deserialize(decoder: Decoder) = Box(dataSerializer.deserialize(decoder)) }
Now we can serialize and deserialize Box<Project>.
@Serializable data class Project(val name: String) fun main() { val box = Box(Project("kotlinx.serialization")) val string = Json.encodeToString(box) println(string) println(Json.decodeFromString<Box<Project>>(string)) }
You can get the full code here.
The resulting JSON looks like the Project class was serialized directly.
{"name":"kotlinx.serialization"} Box(contents=Project(name=kotlinx.serialization))
The above custom serializers worked in the same way for every format. However, there might be format-specific features that a serializer implementation would like to take advantage of.
The Json transformations section of the Json chapter provides examples of serializers that utilize JSON-specific features.
A format implementation can have a format-specific representation for a type as explained in the Format-specific types section of the Alternative and custom formats (experimental) chapter.
This chapter proceeds with a generic approach to tweaking the serialization strategy based on the context.
All the previous approaches to specifying custom serialization strategies were static, that is fully defined at compile-time. The exception was the Passing a serializer manually approach, but it worked only on a top-level object. You might need to change the serialization strategy for objects deep in the serialized object tree at run-time, with the strategy being selected in a context-dependent way. For example, you might want to represent java.util.Date in JSON format as an ISO 8601 string or as a long integer depending on a version of a protocol you are serializing data for. This is called contextual serialization, and it is supported by a built-in ContextualSerializer class. Usually we don't have to use this serializer class explicitly—there is the Contextual annotation providing a shortcut to the @Serializable(with = ContextualSerializer::class) annotation, or the UseContextualSerialization annotation can be used at the file-level just like the UseSerializers annotation. Let's see an example utilizing the former.
@Serializable class ProgrammingLanguage( val name: String, @Contextual val stableReleaseDate: Date )
To actually serialize this class we must provide the corresponding context when calling the encodeToXxx/decodeFromXxx functions. Without it we'll get a "Serializer for class 'Date' is not found" exception.
See here for an example that produces that exception.
To provide a context, we define a SerializersModule instance that describes which serializers shall be used at run-time to serialize which contextually-serializable classes. This is done using the SerializersModule {} builder function, which provides the SerializersModuleBuilder DSL to register serializers. In the below example we use the contextual function with the serializer. The corresponding class this serializer is defined for is fetched automatically via the reified type parameter.
private val module = SerializersModule { contextual(DateAsLongSerializer) }
Next we create an instance of the Json format with this module using the Json {} builder function and the serializersModule property.
Details on custom JSON configurations can be found in the JSON configuration section.
val format = Json { serializersModule = module }
Now we can serialize our data with this format.
fun main() { val data = ProgrammingLanguage("Kotlin", SimpleDateFormat("yyyy-MM-ddX").parse("2016-02-15+00")) println(format.encodeToString(data)) }
You can get the full code here.
{"name":"Kotlin","stableReleaseDate":1455494400000}
In the previous section we saw that we can register serializer instance in the module for a class we want to serialize contextually. We also know that serializers for generic classes have constructor parameters — type arguments serializers. It means that we can't use one serializer instance for a class if this class is generic:
val incorrectModule = SerializersModule { // Can serialize only Box<Int>, but not Box<String> or others contextual(BoxSerializer(Int.serializer())) }
For cases when one want to serialize contextually a generic class, it is possible to register provider in the module:
val correctModule = SerializersModule { // args[0] contains Int.serializer() or String.serializer(), depending on the usage contextual(Box::class) { args -> BoxSerializer(args[0]) } }
Additional details on serialization modules are given in the Merging library serializers modules section of the Polymorphism chapter.
If a 3rd-party class to be serialized is a Kotlin class with a properties-only primary constructor, a kind of class which could have been made @Serializable, then you can generate an external serializer for it using the Serializer annotation on an object with the forClass property.
// NOT @Serializable class Project(val name: String, val language: String) @Serializer(forClass = Project::class) object ProjectSerializer
You must bind this serializer to a class using one of the approaches explained in this chapter. We'll follow the Passing a serializer manually approach for this example.
fun main() { val data = Project("kotlinx.serialization", "Kotlin") println(Json.encodeToString(ProjectSerializer, data)) }
You can get the full code here.
This gets all the Project properties serialized:
{"name":"kotlinx.serialization","language":"Kotlin"}
As we saw earlier, the regular @Serializable annotation creates a serializer so that Backing fields are serialized. External serialization using Serializer(forClass = ...) has no access to backing fields and works differently. It serializes only accessible properties that have setters or are part of the primary constructor. The following example shows this.
// NOT @Serializable, will use external serializer class Project( // val in a primary constructor -- serialized val name: String ) { var stars: Int = 0 // property with getter & setter -- serialized val path: String // getter only -- not serialized get() = "kotlin/$name" private var locked: Boolean = false // private, not accessible -- not serialized } @Serializer(forClass = Project::class) object ProjectSerializer fun main() { val data = Project("kotlinx.serialization").apply { stars = 9000 } println(Json.encodeToString(ProjectSerializer, data)) }
You can get the full code here.
The output is shown below.
{"name":"kotlinx.serialization","stars":9000}
The next chapter covers Polymorphism.