Kotlin multi-platform projects structure

Previously, we talked about how to share code across platforms in kotlin world. But as your code base grows, you will encounter a scenario where some code in the projects is cross, but some other is platform specific, so will not be shared and there may be a version of this code for any or all platforms. Things like separating the platform specific implementation to another repo in order to reuse them in a future project. Well, lucky you, this page covers that topic.

1. Overall structure

  • Project platforms
    • This repo contains the platform implementation. The repo provides you with a universal API across platforms as well as the platform implementation.
    • This is the code base where you need to implement each API twice or thirds depends on how many platforms you want to support.
  • Project lib
    • This is the `common` code which you want to share across all platforms without any change. Such that, you can invoke `UserService.findAll()` to retrieve all the users no matter which platform you are working on.
    • This code will rely on `project platforms`. For instance, an API like `UserService.findAll()`, it needs to do an HTTP call where the actual implementation of sending an HTTP request is different across platforms. While using `project platforms`, this `project lib` can purely focus on the core business logic.
  • Project app
    • This is could be a native app which will call `project lib` for preparing all the data, while it only needs to focus on the UI and UX. Or maybe some native platform-specific feature.
This is a reasonable approach, at least to me. Where every `project` could focus on their own business.

2. Setup the project platforms

This is the easiest one. Because it’s just a typical kotlin multi-platform project setup. You can read the official document for more detail.
It should contain the platform implementation and the related tests.
The folder structure is straightforward:
  • common: for `common` code which contains all the `expect class`.
  • platforms:
    • jvm: contains `actual class` implementation for a jvm
    • js: contains `actual class` implementation for a js

3. Setup the project lib

The folder structure is like:
  • common: business logic that we want to share across platforms
  • bundle
    • jvm: Where we actually test and build the `common` for the `jvm` platform.
    • js: Where we actually test and build the `common` for the `js` platform.
This is one is a little bit tricky because we want to make it `common`. To things needs to take care here are:
  1. We need to add the `common` part from `project platforms` as dependencies.
  2. We want to test this `common` business logic.
  3. When we compile, we need to combine the platform implementations from `project platforms` along with the `common` to yield a platform-specific package.
First, let’s solve them one by one:
1. Add `common` part as dependencies, I tried many ways, including adding the generated `jar` file or `sources-jar` file. Always something wrong here, either the auto-completion stops works, or the internal member can’t be recognized while the namespace could be recognized. Only 2 ways work:
  • Just embed the source code from `project platforms :common` in you `sourceSets`: `main.kotlin.srcDirs += “$kotlin_idiom_path/interfaces/main”`
  • Add that common module as a subproject and add it to the dependencies.
  • I prefer the 1st one because now your gradle settings won’t contain any noises. It will benefit the side-panel of gradle in IDEA as well.
2. In terms of the test, you just write the tests in this `common` folder. But you can’t run the tests, you need to run them against the certain platform, otherwise, they are `common` code, which platform for them to run?
3. This is where the `bundle` coming into play. For building and testing. Let’s take `jvm` for example.
  • First of all, you don’t need to add any code here, this folder, as its name indicates, just for bundling code together. So, under in `:bundle:jvm` subproject, it only contains a `build.gradle` file.
  • You need to use `kotlin-platform-jvm` to compile this module
  • In the `sourceSets` setting: you need to add the source code from all the 3 places, both `common` modules from `project platforms` and this `project lib`, the source code of platform implementation from `project platforms`.
  • You need to add the tests only from `common` module of `project lib`, such that you can run the tests. And it won’t run the tests from `project platforms`, because they all gonna take care there. You don’t need to worry about that. Now run the `gradle :bundle:jvm test` will run the tests.
  • Why I add the source code rather than use the `jar` file? Well, hard lessons learned, this is the only way currently.

4. The end

Now run the `:bundle:jvm build`, it will build a lib to that platform. Try to consume it, it works really well. 🙂
Advertisements

Kotlin code sharing among platforms

When you try to deal with the across platform codes, you need to solve 2 things, one is what to share, the other is the architecture of how to share the code.

Different languages have different techniques to the second problem, but in Kotlin, you can use multiplatform projects to share the code.  With kotlin native, you can even expand the support to natively to iOS and Android.

1. Goals

  • share code among JVM, JS, iOS, and Android.
  • enable the use of XCode and Android Studio to edit the native app, while using another IDE to edit the sharing code part. (But you can use Android studio for editing the KN code as well because they use the same language and the setup will be ready.)
    • I saw some setups. They use root folder as a whole gradle project which will affect the Android project structure been displayed in the android studio because it will add some noise there.
  • share more code between iOS and Android.
    • The main hurdle is the platform-specific API. We can make them as `interface` and still shared the logic around them. But here we will use `kotlin multi-platform` approach to make the code more elegant.
    • Think of an API like this `UserService.getAllorNull()`, then your native side just need to bind the result to the UI. Even though the `HTTP` call is different across platform. But all the exception handling and data processing around the http call are the same and could be shared.
  • a clean structure where everything related should reside together.
    • I mean for the gradle phase as well. Such that, `android` folder is only for Android, `ios` folder is only for iOS. The root folder is for all projects. And the gradle settings for `KN-android` should be in the `android` folder as well as the `iOS` side.
  • enable testing the Kotlin native code as well.
    • It’s native project so anything native can be tested via the strategy on that platform. But what about kotlin native part, what about the `common` code? What about the kotlin native platform specific code for `android` and `ios`? Yes, we have that part covered as well here in the example.

1.1 What if there is no need to care about the platform-specific implementation?

If all you need is the API from kotlin-stdlib, and no platform specific code, then you don’t need to use this setup. You can simply remove the platform folder. And let the common just output an iOS framework for the XCode project to consume. For Android, embed the source code in Android studio to make it as dependency, then it’s fine. Same case for JVM and JS as well. And the setup for JVM and JS is as easy as following the official document.

2. Folder structure

  • ios: XCode project
  • android: Android Studio project
  • common: Common code that is about to share across platforms without any change.
  • platforms:
    • ios: platform specific API for iOS
    • js: platform specific API for js
    • jvm: a dependency for both `android app` and `jvmApp`

Very easy to understand. One interesting thing here is in the platforms folder, besides the common folder. What’s the platforms/android and platforms/ios folder for? Well, they are for platform-specific API.

2.1 How to consume the lib

  • JVM and JS just as easy as including the lib as the dependencies in build.gradle.
  • Android related code will be shared via simply including the according module in the android project.
  • iOS related code will be shared as an iOS framework because Kotlin native will generate that. We will then add it to the XCode building phase. You can see more details on my another [blog](http://www.albertgao.xyz/2018/01/14/how-to-create-kotlin-native-ios-project/).

2.2 Background knowledge for Kotlin multi-platform project

Let’s say you want to have an HTTP kotlin class where you can do the normal GET, POST thing, but the HTTP implementation is different across platforms. If you use the old school interface way, you need to declare an IHTTP interface, then implement it twice in android and ios. Then inject it at runtime to make it work.

But via using the Kotlin multi-platform project, you don’t need to inject, the compiler will only pick up the piece of codes depends on your building target. If you build for Android, it will only grab the Android implementation.

It currently works for jvm, js and kotlin native, oh, that’s a nearly-all-platform code sharing tech right? 😀

And it’s very simple.

You need to expect an implementation of your common code.

expect class Platform {
    fun get():String
}

Then implement the actual code in somewhere else, like android.

actual class Platform {
    actual fun get():String {
        return"android"
    }
}

Or in ios:

actual class Platform {
    actual fun get():String {
        return"ios"
    }
}

Then with some proper setup in gradle, it will work flawlessly.

2.3 How to code the `platforms-ios`

Oh well, you may think that old school interface may better in terms of iOS because you need to implement the iOS API specific thing in swift or obj-c then pass it back to kotlin native. But that is not true. Because there is a 1-to-1 API match in kotlin native such that you can implement the iOS specific API in kotlin as well. And it has the same signature which means you can reuse all the existing knowledge. And kotlin and swift are very similar. And from v6, the building phase will link some of the iOS API (you need to build the others) for you if you are on a Mac.
Which means in an ideal world, your swift side just needs to care about the UI part while all other logic will be handled in the Kotlin native side, shared across platform and tested well.

3. About the example

All the code for setup can be found in this repo.

  • `Sample` class is for code that is sharing across platforms (Which means you have to use API from `kotlin-stdlib-common`).
  • `Platform` class is a class which has been implemented twice for different platforms for showing the platform API case.
  • Open `android` folder from the root in Android Studio, run the app, it will show a string from the `:shared-android` project.
  • Open `ios` folder from the root in XCode, run the app, it will show a string from the `:platforms-ios` project.

In fact, all the apps (jvmApp, jsApp, android and iOS) retrieve the string by invoking the method from the Sample class. And the Sample class invokes the method from Platform class to get the string. When you build an iOS framework, the KN compiler will use :platforms-ios to build along with :common. And when you consume it in android project, the setup will use :platforms-android along with :common. No injection, no affection on the code structure, just that simple. Thanks to the Multi-platform project setup from kotlin.
It is a perfect example of showing how to share the code when you have to deal with the platform API.

Hope it helps. Thanks.

Evaluation of Kotlin Native for mobile: January 2018

We have been looking at Kotlin Native (KN) as a viable solution for mobile development, and as a competitive solution to those like Xamarin – or to completely separate developments for iOS and Android. To be a viable mobile solution in general requires Kotlin Native(KN)  to be workable for iOS as first announced

On my inspection, I believe that KN makes sense, and very good sense in the longer term for mobile development, but Kotlin Native is still far from production ready for the following reasons:

  1. This is the most important reason, all the platform specific API hasn’t been published as maven artifacts, which mean you can’t add them as project dependencies in gradle which leads to many other problems like:
    1. syntax highlight
    2. autosuggestions
    3. your code related to KN is fragile as they said:
      • `warning: IMPORTANT: the library format is unstable now. It can change with any new git commit without warning!`
  2. No documentation for API. And without any support from IDE. It will greatly slow down the job.
  3. Multiplatform project needs to support KN in order to get benefits in terms of architecture. You can still just use the old school way (by declaring `interface`). But this should be ready in 0.6 (My feeling according to slack, still not sure.)
  4. In response to a question on the Kotlin Native slack channel “Any ETA on beta/1.0 version?Nikolay Igotti replied( 25/December/2017) :Not yet, however, being v0.x is mostly an internal thing, in general both compiler and runtime are pretty well tested and known to work on rather complex codebases, such as Video Player: https://github.com/JetBrains/kotlin-native/tree/master/samples/videoplayer

  5. CLion: The benefit of using CLion seems more for the KN dev team, and for projects integrating with the C family, which is not the case for iOS projects.  When the developers deal with the cross-platform setup, they need to dig into the LLVM layer in order to build the bridge. CLion is the only IDE in JB family which supports Kotlin Native at this time, which is problematic for projects looking to go multiplatform with JVM or JS, and for iOS projects which combine with Swift and Xcode.  There is no announced plan for supporting the other JB IDEs. Further, from the project leader’s talk in slack in late January 2018 on support for IDEA: `this is not yet completely decided, but CLion is currently the only option`. And you know, CLion doesn’t support gradle, and they use gradle to build…..    The other possibility is Koltin support in AppCode, which there are suggestions may be coming and could be the best solution.
    • So we have some difficult situation here, which is:
      • CLion doesn’t support gradle. And the issue is there since 2014.
      • The multiplatform project hasn’t support KN yet. But this one is easy and difficult.
        • It’s easy because once the maven dependencies is there, the support will be nearly there. Or we could build the whole thing by `gradle` ourseleves.
        • It’s hard because as I said… The KN libs hasn’t been published as maven dependencies yet.
      • And from the talk of slack, it seems that the team holds the releasing mainly because the KN lib has a completely different format, even for the file extention, it’s called `.klib` now. So, uploading it to maven or JCenter seems not ideal. I assume JB might end up with building a new repository just KN libs.

And when there are some problems on both IDEA and CLion, a potential answer from JetBrains might be a new IDE just for Kotlin native. The following video maybe a provenance for this:

https://www.youtube.com/watch?v=QOIn8Uh3lkE at 6:20 Andrey Breslav said (in Russian) they started development of new commercial product for cross-mobile development, Android and iOS.

But it seems that Appcode with KN support should land first according to the slack chat.

The team leader has said in the slack channel that they will ask for dogfooding once it’s ready. 🙂

If app developers wish to only build using the kotlin-std-lib, and inject the platform-specific API at runtime, it’s doable. But then your codebase will be a mess because you need to build the bridge by yourself  as the kotlin type has been converted to some special interface type, which you need to implement in the swift side as well…. all to cope with an interim solution which will be deprecated in a future version release…

So, 3 things are crucial for using KN in production:

  1. Decent IDE support such that we could inspect the APIs signature(No matter CLion or IDEA or AppCode, this is essential)
  2. Multiplatform project support KN
  3. Able to create an KN project without depending on the KN repo…. Which means they need to publish their platform lib in order to enable us to add them as the project dependencies in gradle. Otherwise, a single build on KN repo takes 2 hours…..on my 2017 i7 15″ retina MacBook pro.

All of the 3 are all needed for writing KN related app.

But I will keep an eye on KN, because I think as I dig into more, I think KN starts to make more sense.

  1. You can really share your logic. The most awesome part is that you can invoke platform-specific API from kotlin side which means you don’t need to deal with the communication between languages. Which means you can really embed heaps of logic in the KN base.
  2. The multiplatform project is a really great way to share code across platforms. It just makes sense. You abstract your code in `common`, `js`, `jvm`, `ios` or `android`. And the gradle will grab the related pieces to compile according to the platform you wanna build against.
  3. This sort of embrace-the-platform-differences-rather-than-write-everything-once-and-run-anywhere concept has granted KN a very promising future compare to Xamarin’s replacing-them-all.

Implementation: what is a practical approach?

Any software team who is considering moving to kotlin, must by definition, be currently using at least one alternative language.  To change languages, and ecosystems, is a big step.  One of the key features of kotlin is how easily and seamlessly a project can migrate from java.  Currently, that same ease of migration is far less real from outside the java ecosystem.

Cold Turkey? Or step by step?

On rare occasions, there may be the opportunity to commence a complete new project and build each component with no basis on any legacy system.  If starting an entirely new project but not already experienced in kotlin, it will still require a huge leap of faith to start an entire development in kotlin.

More often, and in the project we are currently working with, the realistic path is to choose system components that can move to kotlin.

The candidates:

Individual pages discuss these sections, but the spoiler alert is that mobile/android development may not be the logical first choice it would on the surface seem.

Kotlin DSL templates with Python (or any other) Server

The Concept: A replacement template engine written in Kotlin DSL

Kotlin can replace the template system for a python server, or ruby server, or any other server, with no change to the server other than the template system, regardless of what language is used for the server itself.

This allows replacing mako, or jinga2,  Django templates, with kotlin DSL templates.  No kotlin or jvm installation is needed on the server, as the kotlin dsl templates can run as javascript in the browser.

Why? : A more dynamic and concise solution

Template engines are generally considered a method of producing ‘dynamic’ web pages.  While a ‘static’ web page always displays the exact same html, templates produce html which reflects the data presented to the template.  For example, a ‘member’ page will have the information for the member currently logged in,  while a static home page will display the same information to everyone.

However pages generated by templates are not necessarily ‘dynamic’ in a web2.0 manner.  The page is ‘generated on the fly’ by the template engine, but does not necessarily run dynamically in the client browser.  To the client browser, the page appear static.

These kotlin DSL templates are a complete rethink of how templates work, and inherently produce code that is also dynamic on the browser and the entire system makes adding true dynamic content far simpler.

Further, the description of page content becomes more concise than with conventional templates.

More Languages? Or Less?

It could be seen that adding kotlin to do templates means adding yet another language to the toolkit in use with a project, however this is only proposed as substitute for mako, jinga2 or some other template language, so there is also one language no longer required.

Kotlin is far more complex than any template language, but using the kotlin DSL as described here does not require learning the full kotlin language.   The further benefit is that learning a template language just for templates has no other uses, while it is very likely there are other possible uses of kotlin (e.g. Android?)  within a project.  If kotlin has an additional use within the project, then using kotlin for templates could mean one less language overall.

How do these Kotlin DSL templates work?

A conventional template is like a more powerful version of the python format function.

Consider:


"person: {name} age: {age}".format(name="fred",age=35)

This is equivalent to the string being a template that is supplied data in the form of the ‘name’ and ‘age’.  The string is modified by the data, to produce a final string.  Templates just allow more power with modifying the string.

The kotlin DSL templates I am discussing, actually run in the browser, not on the server.  The template can look similar to html, but it is code, not a string.  Using simple python format statement for a simple example of templating, the template might be:


    
person: {name} age: {age}


and by processing the template with our data on the server, the final page is prepared on the server and sent to browser.

However with the kotlin templates discussed, the template is sent to the browser, together will all data for substitution. The advantage to this approach is the client becomes a dynamic application rather than a static page in the browser, right from the beginning of the project.  Moving to a far more powerful dynamic client, capable of changing those substitutions, is possible with no additional layers of code.  The first solution of the static page is slightly simpler – but trying to extend that solution results in far more complex code.

In this paradigm, the layout is sent of JavaScript and just a placeholder div tag resides in the  ‘html’ tag, as sent to the browser. This is outlined fully below.

Different perspective?  HTML vs DOM

One perspective is to think of a web page as the html that describes the page.   Another perspective is to think of the web page is the DOM, and html is just data describing that DOM.  With this second paradigm, we could consider: “what if the DOM itself is described by javascript, not by the html code?”

This makes our web page:


  

<!-- first a json script to hold any json data  -->
{jsondata}

<!-- now here is the div where our main content will be added -->
<div id="page"></div>
<!-- now the script for the page content &gt;-->
  

  

As you can see, the html to describe page content is just an empty ‘div’ in the page.  So the DOM must get the page content part of  DOM from the kotlin DSL.  In fact this same html above, is now used for every page on the web site. The only part that changes is the {jsondata}, and  changing the {templatename} to the actual values for these to be used.  The sample above is perfect for using with a python format to substitute actual names,  but in testing if just sending the html file, then just set these to the values for testing.

What does the Kotlin DSL look like?

Of course the page above does nothing without the javascript,  because the javascript it generating the tags for the main part of the web page.   The only HTML is outside skeleton, and the main content of the page is described in kotlin DSL instead of in HTML.  Here is a very simple ‘main part of the page’ with just a

text

with a heading of ‘heading’ for content.

val div = document.create.div {
h1 { +"heading" }
    p { +"text" }
}

The document.create.div is needed for the very outer layer, and all the html tags inside become very clean and simple.  Using data within the page, or having tags produced in a loop, is all automatic by just using more of the kotlin language.

See the link: kotlin javascript tutorial for more on the DSL for html and how to configure Intellij and install the jar file for kotlinx.html

For a working example of the template scheme described on this page, see the ‘hypothetical programmable calculator’ as described in the page Machine code and Global Memory.  The code for the calculator with code for a kotlin DSL template example can be found in the repository.