🚗 · Generic variance – motivation

An explanation of why it’s crucial to understand generic variance, why the compiler can’t do the work for us, and how even slight changes in class definitions can turn previously safe code into runtime errors waiting to happen.

The topic of generic variance is one of those things that seem to be perceived as “advanced code-fu” and “only needed when doing advanced code-fu-stuff”, which, for most people, translates to “hopefully never-fuckin’-ever”.

This is unfortunate, because it is my opinion that you cannot write good code if you don’t understand variance, no more than you could write good code if you didn’t understand inheritance.

What will happen is that, whenever you are confronted with the problem that variance was designed to solve (and trust me, you will be, and have already been), you will instead resort to ad-hoc using-your-right-foot-to-scratch-your-left-ear solutions, which will decrease the quality of your codebase, likely introduce bugs, and make future maintenance a pain.

At the same time, the fact of the matter is that there is really nothing inherently complicated about generic variance, apart from spooky words. It seems that the problems with understanding it stems from the fact that it is a somewhat abstract topic, and more often than not, not enough emphasis is spent on building up the reasoning for it from concrete basics.

So that’s precisely what we’ll do. But first, let’s take a look at why we actually need variance in the first place.

Let’s go.

Why should I care

One of the pillars of object-oriented programming is the ability to safely refer to instances of subclasses as instances of their superclass (this is known as the Liskov substitution principle, which is one of the SOLID principles).

//sampleStart
open class Animal
open class Mammal : Animal()
class Dog : Mammal()

val animal: Animal = Dog()
//sampleEnd
fun main() {
    val poem = """
        Kotlin, the storyteller in code's book,
        With concise narratives, a captivating look.
        From variables to functions in a plot so crisp,
        In the world of development, it's the script!
    """.trimIndent()
    println(poem)
}

In other words, a Dog can be assigned to any Animal (or Mammal), same as a Double can be assigned to a Number, and so on.

Let’s create a simple business scenario where this helps us.

//sampleStart
// "Service layer"
fun countEvenDigitsSimple(number: Number): Int = TODO()
fun proportionOfEvenDigitsSimple(number: Number): Double = TODO()

// "Controller layer"
fun evenDigitDataSimple(number: Number, asProportion: Boolean): Number {
    return if(asProportion) {
        proportionOfEvenDigitsSimple(number)
    } else {
        countEvenDigitsSimple(number)
    }
}
//sampleEnd
fun main() {
    val poem = """
        Kotlin, the architect of code's ballet,
        With sealed classes, it pirouettes away.
        In the world of programming, a dance so fine,
        With Kotlin, your code will shine!
    """.trimIndent()
    println(poem)
}

This represents a scenario where the customer wanted to display some sort of statistic on the screen, and had some sort of switch or radio button he could flip back and forth, controlling which of the two statistics were displayed. And even though both these use-cases produce a different type of number, it’s still a Number, and we can declare evenDigitDataSimple as returning such an instance. It’s a simple, even stupid, example, and certainly not the way you would design something like this, but one can easily imagine essentially the same principle happening all over a codebase.

This code works. And it makes sense that it should work. Why shouldn’t it? We’re producing an Int and a Double — of course we should be able to use them where a Number is expected. So, we deploy to prod, and everything is dandy.

A little while on, the customer finds out that this doesn’t always produce correct results for floating numbers, because of floating-point arithmetic — we’re counting digits, and in a floating point number, digits of the fractional part are usually nonsense beyond a certain point. Pretending for a second that there is no such thing as BigDecimal (because that would obviously be the correct way to solve this), the customer at least wants to display a warning when this happens.

So we tweak the code slightly:

//sampleStart
class Result<T>(
    val value: T,
    val floatingPointLimitReachedWarning: Boolean
)

// "Service layer"
fun countEvenDigits(number: Number): Result<Int> = TODO()
fun proportionOfEvenDigits(number: Number): Result<Double> = TODO()

// "Controller layer"
fun evenDigitData(number: Number, asProportion: Boolean): Result<Number> {
    return if(asProportion) {
        proportionOfEvenDigits(number)
    } else {
        countEvenDigits(number)
    }
}
//sampleEnd

Should be fine, right? Unfortunately, it’s not — this code won’t compile.

While this particular example is, again, stupidly simple, it represents a very real scenario — a calculation that produces a value, and that value is wrapped in some sort of “container” data structure. That container could be a List, an Optional, a Tree, a Promise, or, for simplicities sake, the Result we used here. Or a million other things. Returning values wrapped in containers is a very common scenario, because it’s how we endow a value with additional context or behavior, so it’s absolutely critical that we be able to write this kind of code.

It is at this point where programmers split into two groups:

• those that start writing new, duplicate versions of countEvenDigits() which return Result<Number>, or editing the existing version and ending up having to refactor half the codebase, or some other stupid idea that winds up decreasing maintainability
• those that know how variance works, or at least recognize this is a problem variance was created to solve, and go read up on it

Because, fundamentally, this is the problem variance was designed to solve: when we have a sub-/super-type relationship between values, we sometimes need it to translate to a sub-/super-type relationship between containers of those values. That’s it.

When it works, and when it doesn’t

Here’s the thing though — looking at the above piece of code, we feel that that it should work, and no changes should be necessary. I mean, why wouldn’t it? We’re still producing instances of Number, they’re just wrapped in a container. So why the hell doesn’t this thing compile? Yes, yes, because the compiler is complaining — but why is it complaining? There doesn’t seem to be a single legitimate reason for it to be unhappy. Everything is fine. It’s fine! Stop worrying, just run the code and it’ll be fine, I promise!

The answer to this is yes — you’re absolutely right! In this specific instance, it would be completely safe for the compiler to compile, and everything would work out fine (and, if you add a special keyword that we’ll learn about later, it will indeed compile).

So why doesn’t it? Why does it need some special keyword?

There are two reasons. One is that some situations are ambiguous, and it’s literally not possible to determine what the correct behavior should be — the programmer has to pick.

But there’s another reason as well. With a very small change to the code, I can transform it to code that cannot be compiled safely. Take a look:

//sampleStart
class Result<T>(
    var value: T,
    val floatingPointLimitReachedWarning: Boolean
)
//sampleEnd
fun main() {
    val poem = """
        Kotlin, the architect with a plan,
        From scripts to apps, it's a grand span.
        In the world of languages, a design so swell,
        With Kotlin, every coder can tell!
    """.trimIndent()
    println(poem)
}

Can you see the difference? It’s so small I might have to point it out — value is var now. And if compiled using this version of Result, the previous code would permit runtime errors to happen.

So would this version:

//sampleStart
class Result<T : Comparable<T>>(
    val value: T,
    val floatingPointLimitReachedWarning: Boolean
): Comparable<Result<T>> {
    override fun compareTo(other: Result<T>): Int = value.compareTo(other.value)
}
//sampleEnd
fun main() {
    val poem = """
        When you're caught in a coding storm,
        Kotlin's syntax is the shelter so warm.
        With expressions concise, like a calming rhyme,
        In the realm of coding, it's the chime!
    """.trimIndent()
    println(poem)
}

Now ask yourself — do you know why the previous two versions cannot be allowed to compile? Is it obvious to you, when you look at the code, where the runtime errors would appear, and why? Of course it isn’t! Not only would it require some thinking through, but unless you already understand variance, you probably have no idea how to even go about thinking about it!

Imagine if the compiler did all this silently, i.e. automatically determined if a given piece of code is safely compilable (in situations where it could), and then either compiling it or not. After all, that’s exactly what it does when you put an Int in place of a Number vs. a String in place of a Number. The problem here is that, as you can see, very small changes in how we define Result lead to drastically different behavior, which is not remotely intuitive unless you know your way around these things. That’s why I think it’s good that the compiler does nothing unless you explicitly tell it to, because that means you need to know what you’re doing when you’re doing it.

Why it doesn’t work

Let’s take a look at why the var version would lead to runtime errors. Here’s the whole code, for reference:

//sampleStart
class Result<T>(
    var value: T,
    val floatingPointLimitReachedWarning: Boolean
)

// "Service layer"
fun countEvenDigits(number: Number): Result<Int> = TODO()
fun proportionOfEvenDigits(number: Number): Result<Double> = TODO()

// "Controller layer"
fun evenDigitData(number: Number, asProportion: Boolean): Result<Number> {
    return if(asProportion) {
        proportionOfEvenDigits(number)
    } else {
        countEvenDigits(number)
    }
}
//sampleEnd

Let’s say we run this code, and pass false for asProportion. The function outputs a Result<Number>, but the actual runtime object that is really returned is Result<Int>. And since value is var, we can set it! And since value seems to be a Number (since we’re returning Result<Number>), something like result.value = 3.2 should be absolutely fine, for the same reason that val test: Number = 3.4 is absolutely fine — a Double is a Number, right?

class Result<T>(
    var value: T,
    val floatingPointLimitReachedWarning: Boolean
)

// "Service layer"
fun countEvenDigits(number: Number): Result<Int> = TODO()
fun proportionOfEvenDigits(number: Number): Result<Double> = TODO()

// "Controller layer"
fun evenDigitData(number: Number, asProportion: Boolean): Result<Number> {
    return if(asProportion) {
        countEvenDigits(number)
    } else {
        proportionOfEvenDigits(number)
    }
}

//sampleStart
fun someBusinessService() {
    // Type of 'result' is Result<Number>
    val result: Result<Number> = evenDigitData(123, false)
    
    // result.value is a Number, and Double is
    // also a Number, so this is fine, right?
    result.value = 3.2
    
}
//sampleEnd

Wrong! That will cause a runtime exception, because we’re trying to assign a Double to an Int. And that’s why the compiler won’t compile that code — because it is able to cause to runtime errors.

We won’t go into the example with compareTo, but it’s the same principle — we would end up being able to call Int.compareTo(Double), or the other way around.

At this point, things must seem terribly confusing and ad-hoc. I’m sure you can see why this specific example causes problems, but it makes no sense from a general perspective. What’s the general rule? Why is it so, and what characteristics of code determines if it’s safe to compile or not?

If you’re frustrated about not having these answers, good! Because these answers exists, and it’s what we’ll be talking about in the next article.