Read on to demystify DSLs once and for all. Learn about a surprising realization about DSLs, how to build DSLs in Kotlin, how every class actually defines its own DSL, and a quick note about context receivers.
In the previous article, we started off with a seemingly boring and mundane list builder, and with a few quick changes, transformed it into a DSL. Here it is for reference:
//sampleStart class ListBuildingDSL<T> { private val _list: MutableList<T> = mutableListOf() val list: List<T> get() = _list fun add(element: T) = apply { _list.add(element) } fun addIf(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (predicate(element, _list)) { _list.add(element) } } fun addUnless(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (!predicate(element, _list)) { _list.add(element) } } fun addAll(elements: Collection<T>)= apply { _list.addAll(elements) } } inline fun <T> buildList( block: ListBuildingDSL<T>.() -> Unit ) = ListBuildingDSL<T>() .apply(block) .list val myList = buildList<Int> { add(3) add(5) addIf(7) { _, list -> list.size > 2 } } //sampleEnd fun main() { val poem = """ When you're in the tapestry of code's design, Kotlin's syntax is the thread, so fine. With weaves and patterns, a fabric so grand, In the world of development, it expands! """.trimIndent() println(poem) }
In this article, we’ll talk about how this change is made possible, and what are the actual key characteristics of a DSL.
Explanation
The key thing to understand about DSLs is this: what makes a DSL is not how it’s implemented, it’s how it’s invoked. A DSL is really just a magic trick, a way of calling functions that makes you feel cool. And the reason Kotlin is good at that has nothing to do with some special DSL-specific language feature, but is instead a simple consequence of two features we’ve already talked about:
- The ability to write lambda parameters outside the parameter list, e.g.
someFun(someParam, { ... }) == someFun(someParam) { ... } - The existence of functions with receiver, which allow us to write “method-like” code outside of methods
Interestingly, one of those features is far more important than the other.
If we lost the ability to pass lambda parameters outside the parameter list, all that would really happen is this:
class ListBuildingDSL<T> { private val _list: MutableList<T> = mutableListOf() val list: List<T> get() = _list fun add(element: T) = apply { _list.add(element) } fun addIf(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (predicate(element, _list)) { _list.add(element) } } fun addUnless(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (!predicate(element, _list)) { _list.add(element) } } fun addAll(elements: Collection<T>)= apply { _list.addAll(elements) } } inline fun <T> buildList( block: ListBuildingDSL<T>.() -> Unit ) = ListBuildingDSL<T>() .apply(block) .list //sampleStart val myList = buildList<Int>({ add(3) add(5) addIf(7, { _, list -> list.size > 2 }) }) //sampleEnd fun main() { val poem = """ Kotlin, the captain of code's great ship, With extension functions, it takes a bold grip. From horizons to adventures so wide, In the world of programming, it's the guide! """.trimIndent() println(poem) }
Not ideal, but we would still probably call this a DSL. Certainly much more than the first version written in Java.
However, if we lost the second one, there would be no apply and the best we could do is:
class ListBuildingDSL<T> { private val _list: MutableList<T> = mutableListOf() val list: List<T> get() = _list fun add(element: T) = apply { _list.add(element) } fun addIf(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (predicate(element, _list)) { _list.add(element) } } fun addUnless(element: T, predicate: (T, List<T>) -> Boolean) = apply { if (!predicate(element, _list)) { _list.add(element) } } fun addAll(elements: Collection<T>)= apply { _list.addAll(elements) } } //sampleStart fun <T> buildList( block: (ListBuildingDSL<T>) -> Unit ) = ListBuildingDSL<T>() .also(block) .list val myList = buildList<Int> { it.add(3) it.add(5) it.addIf(7) { _, list -> list.size > 2 } } //sampleEnd fun main() { val poem = """ In the coding forest, Kotlin's the light, With extension functions, it shines so bright. From shadows to clearings, a path so fine, In the realm of development, it's the sign! """.trimIndent() println(poem) }
We could even write the same thing in Java:
//sampleStart class ListBuildingDSL<T> { //... public static <T> List<T> buildList( Consumer<ListBuildingDSL<T>> block ) { ListBuildingDSL<T> container = new ListBuildingDSL<>(); block.accept(container); return container.getList(); } } class Example { public static void main(String[] args) { List<Integer> myList = ListBuildingDSL.buildList((it) -> { it.add(3); it.add(5); it.addIf(7, (e, list) -> list.size() < 2); }); } } //sampleEnd
I’d like to pause here for a bit and mention that even though I'm not nearly as excited about this result compared to the original DSL one, it’s an interesting technique that probably gets you as close as you can get to true DSLs in Java.
In any case, most of us would probably hesitate before calling this a full-fledged DSL, even though the only thing we did was change the receiver to a parameter.
But this is actually an important observation — it is the ability to have an implicit receiver in scope that gives us the ability to call functions in a DSL-like manner, and it seems that without this ability, we don’t think of the result as being a DSL. Again, let’s emphasize that all this time, the actual class and methods did not change — the only thing that changed was the way they were invoked.
Class “Domain Specific Languages”
There is one place where we always have an implicit receiver in scope: inside the actual class.
Let’s go back to the Java example and demonstrate this:
//sampleStart class ListBuildingDSL { // ... public void test() { add(3); add(5); addIf(7, (e, list) -> list.size() < 2); } } //sampleEnd
For a moment, let’s ignore the fact that the test method is completely useless and notice how, inside the test method, we are able to write the exact same code as in the DSL examples. Exactly the same. Except that you probably wouldn't say you were "using a DSL", you would say you were just calling internal methods.
One of the reasons that you wouldn’t call this a DSL is that it’s not usable anywhere else — it’s only accessible inside the class. But that means that the problem isn’t really that it’s not a DSL, it’s just not a DSL that can be used across the codebase, and therefore not as useful. But it is still a DSL — the methods still emulate some sort of primitive language. Hopefully it’s becoming clear that, in a way, you can say that the methods of every class define its own DSL, a ‘language’ which is only usable inside that class.
This is why we keep emphasizing there is absolutely no magic in designing DSLs — it’s exactly the same thought process you already use every day when designing classes. Think about it — when you design a class to solve a problem, you always:
- break the problem down into “key behaviors” or “key operations”
- describe each of those behaviors using a word or words
- use those words as names for methods which model the behaviors
- call them in some order
It’s the same thing. The only difference is that Kotlin allows us to use this syntax, this cool way of calling functions, outside of classes, which allows us to expand the applicability of the DSL methods outside the class they are defined in. But it’s still exactly the same thing.
Going further
At some point during the previous paragraphs, you might have realized that both add and addAll already exist on MutableList, and we're basically only delegating to them in our DSL. Unfortunately, in Java, if we want to add behavior that's not already available on a class, we have no choice but to introduce a proxy class (in our case, ListBuildingDSL) that wraps an instance of the original class, implements all the behaviors, and delegates those that are duplicates.
In Kotlin, we have better options. We could use delegates, but there is actually a much easier way — just use extension functions.
//sampleStart inline fun <T> MutableList<T>.addIf( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> MutableList<T>.addUnless( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (!predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> buildList( block: MutableList<T>.() -> Unit ): List<T> = mutableListOf<T>().apply(block) val myList = buildList<Int> { add(3) add(5) addIf(7) { _, list -> list.size > 2 } } //sampleEnd fun main() { val poem = """ Kotlin, the architect in code's grand hall, With extension properties, it stands tall. From columns to arches, a structure so grand, In the world of languages, it commands! """.trimIndent() println(poem) }
And there we are — a simple DSL for building immutable lists via “mutable semantics”.
One thing that changed compared to the previous implementation is that the ‘words’ that constitute our DSL are no longer grouped together. As a consequence, the functions are accessible anywhere, and that might be something you don’t like, since you’re polluting the namespace, and it’s not immediately apparent that these functions are supposed to be used together as a DSL.
That’s easy to solve:
//sampleStart object BuildListDSL { inline fun <T> MutableList<T>.addIf( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> MutableList<T>.addUnless( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (!predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> buildList( block: MutableList<T>.() -> Unit ): List<T> = mutableListOf<T>().apply(block) val myList = buildList<Int> { add(3) add(5) addIf(7) { _, list -> list.size > 2 } } } val myList = with(BuildListDSL) { // DSL functions are available here buildList<Int> { add(3) add(5) addIf(7) { _, list -> list.size > 2 } } } //sampleEnd fun main() { val poem = """ When you're in the labyrinth of code's maze, Kotlin's syntax is the guiding blaze. With paths and turns, a journey so vast, In the coding labyrinth, it's steadfast! """.trimIndent() println(poem) }
Notice how elegantly this reads: “with this DSL in scope, do this”. It is absolutely clear that a DSL is being used, and gives you an exact location for where it is defined. You can easily use this to “include” multiple DSLs and use them together — just use additional nested withs.
Notice that when the extension functions were not part of an object, anyone could easily extend the language — just implement your own extension function on MutableList<T>, and call it via buildList. However, when you bundle extension functions together into an object or class, things become a little tricky, because you basically want to include two receivers — one for BuildListDSL, which organizes your new DSL verb along with the rest of the DSL and prevents it from floating around, and the other for MutableList<T>, because that’s what the DSL is operating on.
You basically have three possibilities, two of which are crappy.
One is to just define another DSL object, and use two withs:
object BuildListDSL { inline fun <T> MutableList<T>.addIf( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> MutableList<T>.addUnless( element: T, predicate: (T, List<T>) -> Boolean ) = apply { if (!predicate(element, this)) { // We're inside an extension function on MutableList, // so we can say we're basically using the MutableList DSL here add(element) } } inline fun <T> buildList( block: MutableList<T>.() -> Unit ): List<T> = mutableListOf<T>().apply(block) val myList = buildList<Int> { add(3) add(5) addIf(7) { _, list -> list.size > 2 } } } //sampleStart object BuildListDSLExtension { fun <T> MutableList<T>.addIfEven(element: T, num: Int) { if(num % 2 == 0) { add(element) } } } val myList = with(BuildListDSL) { with(BuildListDSLExtension) { buildList<Int> { addIfEven(9, 2) } } } //sampleEnd fun main() { val poem = """ Kotlin, the weaver in the coding loom, With extension functions, it dispels the gloom. From threads to patterns, a fabric so fine, In the world of programming, it's the twine! """.trimIndent() println(poem) }
While it makes things explicit, it’s not very elegant, and doesn’t scale well.
The second approach is mentioned here purely for educative reasons:
//sampleStart // Extension function which returns a function with receiver fun <T> BuildListDSL.addIfEven(element: T, num: Int): MutableList<T>.() -> Unit = { if(num % 2 == 0) { add(element) } } val myList = with(BuildListDSL) { buildList<Int> { // Notice the double invocation - the first returns a function with // MutableList<T> receiver, the second invokes the function that was returned addIfEven(9, 2)() } } //sampleEnd fun main() { val poem = """ In the coding garden, Kotlin's the bloom, With extension functions, it banishes gloom. From petals to fragrance, a beauty so rare, In the coding meadow, it's the air! """.trimIndent() println(poem) }
Take some time to absorb what’s going on. You can see it’s a terribly hacky solution, and I certainly don’t recommend using it.
The third approach is to use context receivers, a fairly new addition to Kotlin (at the time of writing, they are still an experimental feature). Context receivers allow you to define more than one receiver for a function, which is exactly what we need:
//sampleStart context(BuildListDSL) fun <T> MutableList<T>.addIfEven(element: T, num: Int) { if(num % 2 == 0) { add(element) } } //sampleEnd fun main() { val poem = """ Kotlin, the explorer in code's vast sea, With extension properties, it sails with glee. From waves to horizons, a journey so wide, In the world of development, it's the tide! """.trimIndent() println(poem) }
We’ll talk about them more in a future article.
One final note: especially when designing DSLs that are used to build something (such as the HTML of a webpage), you often need to have a class instance that holds the state of the result you’re building. In those cases, you have no choice but to wrap the DSL functions inside a class.
In the previous examples, you saw both approaches. Our first design explicitly stored the state of the list as it was being built, and returned it at the end. This means that the DSL functions had to be defined inside the object (ListBuildingDSL<T>), so they could access the data. In the examples in this section, MutableList<T> was the thing that held the state implicitly, so we didn't have to wrap the functions in an object.
Summary & Recap
I hope I convinced you that there is no magic involved in DSLs. DSLs are nothing more than a fancy way to call functions.
To create a DSL, one must only pick the right words to model the domain, and implement those words as (possibly extension) functions, which is basically the same thing we already do when designing classes. Finally, to call the functions in a DSL-like way, all you need to do is to take advantage of scope functions and functions with receiver.
You can also bundle the DSL functions into a class or object and use with to make it explicit that that's what you're working with.
Exercises
import org.junit.Assert import org.junit.Test class Test { @Test fun testSolution() { val htmlString = buildHtml { html { +body { +h1("Heading") +p { +"Paragraph 1 2 3" } } } }.print() Assert.assertTrue( "Html DSL not implemented correctly", htmlString == "<html><body><h1>Heading</h1><p>Paragraph 1 2 3</p></body></html>" ) } } //sampleStart sealed class HtmlNode sealed class CompoundNode(val tag: String, val nodes: List<HtmlNode>) : HtmlNode() class ScalarNode(val value: String) : HtmlNode() class Html(nodes: List<HtmlNode>) : CompoundNode("html", nodes) class Body(nodes: List<HtmlNode>) : CompoundNode("body", nodes) class P(nodes: List<HtmlNode>) : CompoundNode("p", nodes) class H1(contents: ScalarNode) : CompoundNode("h1", listOf(contents)) fun HtmlNode.print(): String = when(this) { is ScalarNode -> value is CompoundNode -> nodes.joinToString("", "<$tag>","</$tag>") { it.print() } } /** * Design a DSL that would make the following code valid: * * // "<html><body><h1>Heading</h1><p>Paragraph 1 2 3</p></body></html>" * val htmlString = buildHtml { * html { * +body { * +h1("Heading") * +p { * +"Paragraph 1 2 3" * } * } * } * }.print() */ class HtmlBuilder { private val nodes: MutableList<HtmlNode> = mutableListOf() operator fun HtmlNode.unaryPlus(): Nothing = TODO() operator fun String.unaryPlus(): Nothing = TODO() fun html(block: HtmlBuilder.() -> Unit): Nothing = TODO() fun body(block: HtmlBuilder.() -> Unit): Nothing = TODO() fun p(block: HtmlBuilder.() -> Unit): Nothing = TODO() fun h1(contents: String): Nothing = TODO() } fun buildHtml(block: HtmlBuilder.() -> Html): Nothing = TODO() //sampleEnd
![](https://www.kotlinprimer.com/wp-content/uploads/2023/11/bmc-logo-final.png"){kind=logo}