Kotlin Limitations vs Python

social-media-failThis page will serve as repository of limitations found when attempting to move kotlin code to python code. Some limitations are a natural consequence of moving to static types, others are effectively inherited from Java.

The list here is based on placing those I find most significant at the top, and the further down the list the less significant.

base class parameter pass through

background

Kotlin, like python, allows for optional parameters, a feature not found in Java.  A consequence of this feature is that libraries with heavily used classes can over time evolve to have a large number of optional parameters in the most heavily used methods.  A larger number of parameters should only occur in code in a language that has optional parameters, as without the parameters being optional, every extra parameter  would introduce overhead on every call.  These methods with a large number of parameters are often frequently used methods, and the goal is that well chosen defaults will mean that most of these optional parameters are only provided in exceptional cases.  While a typical instance of the class may require only 2 or 3 parameters, while there may be as many as 20 to chose from.

To find examples, I simply thought: “what are the most widely used python packages I can think of?”.  The first three that occurred to me to check were sqlalchemy,  attr, and Django, and for each of these I looked for the most basic usage cases I could find. The first examples I found are:

  • sqlalchemy:   Table Class  20 optional parameters (see ‘parameters’)
  • attr: atrr.s() function – 9 optional parameters (see attr.s function)
  • Django: Field class – 22 optional parameters (see __init__)

While such a large number of parameters should not normally occur in Java due to lack of optional parameters, I think these example establish that the pattern is common in well regarded python packages.

The Problem.

Consider the Django Fieldclass . This class serves as a base class for several other classes, such as  TextField, TimeField, BinaryField etc.  To code examples like these cleanly, some type of language mechanism is needed which effectively achieves:  “include all fields from the base class constructor (or other method from the base class) as also parameters to this class unless specifically declared in this class“.

Python uses the *args,**kwargs system, which for several reasons is not an appropriate solution for kotlin, but is at least a solution.  There are some more elegant solutions possible for kotlin, and perhaps one will be added at a later time.

In Python the code for defining the BinaryField Class is as follows (not exact code for simplicity):

class BinaryField(Field):

    def __init__(self, *args, **kwargs):
        kwargs['editable'] = False
        super().__init__(*args, **kwargs)
        // other init code goes here

while in kotlin (slightly changed for name conventions and simplicity) the code becomes:

class BinaryField(
        verboseName:String?=null, name:String?=null, primaryKey:Boolean=false,
        maxLength:Int?=null, unique:Boolean=false, blank:Boolean=false,
        nulled:Boolean=false, dbIndex:Boolean=false, rel:String?=null,
        default:Any?=null, //editable:Boolean=true, - force 'false' for editable
        serialize:Boolean=true,
        uniqueForYear:Int?=null, choices:String?=null, helpText:String="",
        dbColumn:Int?=null, dbTablespace:Int?=null, autoCreated:Boolean=false,
        validators:List<Any>?=null, errorMessages:String?=null
 ):Field(verboseName=verboseName, name=name, primaryKey=primaryKey,
        maxLength=maxLength, unique=unique, blank=blank, nulled=nulled,
        dbIndex=dbIndex, rel=rel, default=default, editable=false,
        serialize=serialize, uniqueForYear=uniqueForYear, choices=choices,
        helpText=helpText, dbColumn=dbColumn,
        dbTablespace=dbTablespace, autoCreated=autoCreated,
        validators=validators, errorMessages=errorMessages) {
  // class code here
}

Clearly, the call to the base constructor will be much shorter if not using named parameters, which is a choice, but in a list this long I would use named parameters.

The code (or almost identical code) will be repeated TextField, TimeField and the over 12 other fields that inherit from Field. Any update to the parameter list for the base Field class is tedious to say the least.

This is a case where kotlin requires boilerplate that is not required in python. What is needed is some way to say “accept all parameters to the base default constructor not specifically named in the base call, and pass these parameters through“. Given this problem will not occur in Java libraries which have no default parameters, it may be some time before kotlin designers consider this, if ever.  In the mean time, messy.

Constructor Calls

(to be added: by Nov 13)

Intricate, preset order, class data initialisation

Kotlin has what can appear a rather strange way of implementing the overriding of properties when extending classes.  The result is that when extending classes, the behaviour of initcode can be unexpected.  The way to avoid this is to use lazy properties in place of initialising during init.

open class Base{
    val simple = 3
    open val open = 3
    open val openGet = 3
    open val getter get()= 3
    open val getOpen get()= 3
    open val getDelg by GetDelg(3)
    init {
        println("base simple $simple open $open openG $openGet "+
                "getOpen $getOpen getter $getter "+
                " getDelg $getDelg")
    }
    //open fun add(a:Int,b:Int) = a + b
}

class SubClass:Base(){
    override val open = 4
    override val openGet get()= 4
    override val getter get() = 4
    override val getOpen = 4
    //override val getDelg by GetDelg(4)  //uncomment for null pointer
    init {
        println("sub simple $simple open $open openG $openGet "+
                "getOpen $getOpen getter $getter "+
                " getDelg $getDelg")

    }
}
class GetDelg(val value:Int){
    operator fun getValue(thisRef: Any?, property: KProperty<*>): Int {
        return value
    }
}

The print from instancing a SubClass object is:

base simple 3 open 0 openG 4 getOpen 0 getter 4 getDelg 3
sub simple 3 open 4 openG 4 getOpen 4 getter 4 getDelg 3

 

Open is 0 in the base class init because the code to set the value has not yet been run, but not it is not 3 as you would expect.

openG, a value overridden by a get() method, perhaps unexpected returns the override value in both init() methods

getOpen, a get() method in the base overridden by a simple initialisation, behaves as a simple initialised value overridden by a new initialise, which is to be unitialised in the base init() method

getter() , a get() method overidden by another get() method returns the override value as does openG

getDelg() actually generates a null pointer exception if called during  the base init() method, as the overridden value has not been calculated

Note: part of this behaviour is base on the fact that overridden properties are actually new distinct properties, so the do not inherit values from base class property which is still accessible via super.<name>.  This means, counterintuitively, that open in the base init() method, returns 0while super.open in the subclass init()will return 3

I will update with more on the use of lazy to avoid this issue, but the main point is to think carefully before initialising values in a base class that are open.

*args, **kwargs (to be added)

A specific use of *args and **kwargs has already been covered in base class parameter pass through.

Outside of that specific use, kotlin does have effective equivalents to most use cases, but may depend on the reflections library, which is not available on all platforms at this time.

vararg parameters capture the equivalent to a *args list, and allows for using the list for calling in a manner very similar to python.

callBy provide most of the functionality of **kwargs when used for calling functions, but some code is needed to map parameter names to KParameters.  A link to such code may be added to this page is someone asks 🙂

For cases where it is desired to capture parameters in map for, using a map actually makes better sense in every case I have found, but I will update this further if I find a case where this is not true.

Advertisements

When will your project ‘grow up’ and require typesafe code?

originalThere is common belief that after an initial very agile development period,  “grown up projects should switch to a static typed language”.  This also raises the question, “are type-safe languages less suitable for early stage development?”  This page considers the evidence for these beliefs, plus considers if there is a benefit to starting dynamic and switching to static, what is the crossover point?

TL;DR? As usual, read the headings and choose only read beyond the heading when you choose.  But in summary, modern statically typed language, and specifically kotlin, are bringing forward the point where adoption is logical, to the point it can now be argued that any project that justifies its own repository, would benefit from starting out as a kotlin project. Continue reading “When will your project ‘grow up’ and require typesafe code?”

Class Level Objects, Companion Objects & Instances

Topics for this section:

  • Class Level Objects & Companion Objects:  Key Terminology.
    • the simplest case – one object type
    • Python – an additional object type requires precise terminology
    • Kotlin: the companion object equates to the Python class object
  • Class & Companion Variables and Methods vs instance methods and variables
  •  Terminology for Clarity: Class Level Object, Variables and Methods
  • instance vs class variables in python
  • an alternative explanation of instance vs class for python

Class Level Objects & Companion Objects: Settling on Terminology

This bliki assumes a reader has knowledge of what class is, and what is an object is, but precise usage of the terminology does change from language to language, so please  tolerate a review of very basic concepts in order to be specific about exact terminology

Consider a Person class, that will have two instances, one for “Jack” and one for “Jill”.

The simplest case: the base of confusion

With a compiler language, the compiler will read the class definition and generate the code for all methods required by the class.  The compiler will also store in an internal data structure of the compiler all relevant information about Person class.  For any code relating to the class, the compiler will use the information it has stored about the class to enable generation of the appropriate code.

At run time the Jack and Jill instances Jack and Jill of the person class are created.  So there are two object, both instances of same class: the Person class.  In this environment, any reference to a Person object must be a reference to the Jack or Jill instances.

Python: An additional object type requires precise terminology.

Now consider Python.  The python language also builds the internal data structure containing the relevant information about the person class, and in fact that data structure is an object of the ‘type’ class.  But with python, there is not the same distinction between compile time and run time.  The running program can also access the ‘internal data structure’ – which in fact makes it no so internal.  At run time, there will be an instance of the type class : Person, and once Jack and Jill are instanced, there will be two instances of the Person class: Jack and Jill.  As there is only one Person class regardless of how many instances of Person are generated,  there will always be one Person class level object, regardless of whether there are zero or one hundred instances.  So with the Jack and Jill example with python, there are now three objects available to the running program.  A type object: Person, and two Person Objects: Jack and Jill.  When we say ‘Person object’, do we mean the object describing the class, or one of the two objects which are instances of the class?

In python, the convention is that the object describing the class is the class, and a class variable is a variable within that object describing the class, as opposed to a variable within an instance of the class such as Jack and Jill.  The variables within these instances are instance variables.  This contrasts with the previous simple case, where, with only one instances type of object for any class, the term class variable can only mean an instance variable.

Kotlin: The companion Object equates to the Python class object

Kotlin also provides for an object holding data at the class level, the companion object. Unlike Python, not all classes have a companion object.  Classes with a companion object at class level, must specifically declare a companion object. Keeping the companion object as optional allows the metaprogramming power of Python, or the closer to C++ internal simplicity of Java.

Like the Python class object, there is one companion object for a class, regardless of how many (or even any) instances of the class are created.

Class & Companion Variables and Methods vs instance methods and variables

Variables.

With Kotlin, the developer can declare variables within the companion object, and with Python class variables can be declared at class level.  In both cases, the provides a single variable shared by every instance of the class, and accessible even without the need for an instance.

Java and several other languages have static variables, which have the same property of  a single shared variable regardless of instances of the class and the possibility of access independent an any instance.   However,  there the key difference is with Python and Kotlin, this shared variable is also contained in an object, while a static variable has name scope within a class, but beyond that name, there is no relationship between a static variable and any object.  An example of this difference is that a companion or class object can be part of a family of related classes and implement an interface or have inheritance or duck typing, allowing passing the object as parameter allowing code to operate on the class/companion variable without knowing the specific class.

Simply put, the companion/ class level object is a more object oriented approach that supports more metaprogramming.

Methods

Python supports both class methods and static methods.  Static methods being an equivalent to other static methods in that they have name scope, but have no other relationship with any object.

Kotlin has companion object methods with are similar to python class methods, in that they have access to the object containing all settings, class variables and methods.  They are just like instance methods, but operation on a companion type, or type object, in place of operation on an instance of the class itself.

Terminology for Clarity: Class Level Object, Variables and Methods

The python term ‘class method’ or ‘ class variable’ has a very high protentional for confusion with regular instance variables or instance methods.

The proposal current in use within this bliki is to use the term ‘class level method‘ or ‘class level variable‘.  Another contender in place of ‘level’ was ‘singleton’ but the simpler level is the choice…at least for now.

 

Implementation: what is a practical approach?

Any software team who is considering moving to kotlin, must by definition, be currently using at least one alternative language.  To change languages, and ecosystems, is a big step.  One of the key features of kotlin is how easily and seamlessly a project can migrate from java.  Currently, that same ease of migration is far less real from outside the java ecosystem.

Cold Turkey? Or step by step?

On rare occasions, there may be the opportunity to commence a complete new project and build each component with no basis on any legacy system.  If starting an entirely new project but not already experienced in kotlin, it will still require a huge leap of faith to start an entire development in kotlin.

More often, and in the project we are currently working with, the realistic path is to choose system components that can move to kotlin.

The candidates:

Individual pages discuss these sections, but the spoiler alert is that mobile/android development may not be the logical first choice it would on the surface seem.

Kotlin DSL templates with Python (or any other) Server

The Concept: A replacement template engine written in Kotlin DSL

Kotlin can replace the template system for a python server, or ruby server, or any other server, with no change to the server other than the template system, regardless of what language is used for the server itself.

This allows replacing mako, or jinga2,  Django templates, with kotlin DSL templates.  No kotlin or jvm installation is needed on the server, as the kotlin dsl templates can run as javascript in the browser.

Why? : A more dynamic and concise solution

Template engines are generally considered a method of producing ‘dynamic’ web pages.  While a ‘static’ web page always displays the exact same html, templates produce html which reflects the data presented to the template.  For example, a ‘member’ page will have the information for the member currently logged in,  while a static home page will display the same information to everyone.

However pages generated by templates are not necessarily ‘dynamic’ in a web2.0 manner.  The page is ‘generated on the fly’ by the template engine, but does not necessarily run dynamically in the client browser.  To the client browser, the page appear static.

These kotlin DSL templates are a complete rethink of how templates work, and inherently produce code that is also dynamic on the browser and the entire system makes adding true dynamic content far simpler.

Further, the description of page content becomes more concise than with conventional templates.

More Languages? Or Less?

It could be seen that adding kotlin to do templates means adding yet another language to the toolkit in use with a project, however this is only proposed as substitute for mako, jinga2 or some other template language, so there is also one language no longer required.

Kotlin is far more complex than any template language, but using the kotlin DSL as described here does not require learning the full kotlin language.   The further benefit is that learning a template language just for templates has no other uses, while it is very likely there are other possible uses of kotlin (e.g. Android?)  within a project.  If kotlin has an additional use within the project, then using kotlin for templates could mean one less language overall.

How do these Kotlin DSL templates work?

A conventional template is like a more powerful version of the python format function.

Consider:


"person: {name} age: {age}".format(name="fred",age=35)

This is equivalent to the string being a template that is supplied data in the form of the ‘name’ and ‘age’.  The string is modified by the data, to produce a final string.  Templates just allow more power with modifying the string.

The kotlin DSL templates I am discussing, actually run in the browser, not on the server.  The template can look similar to html, but it is code, not a string.  Using simple python format statement for a simple example of templating, the template might be:


    
person: {name} age: {age}


and by processing the template with our data on the server, the final page is prepared on the server and sent to browser.

However with the kotlin templates discussed, the template is sent to the browser, together will all data for substitution. The advantage to this approach is the client becomes a dynamic application rather than a static page in the browser, right from the beginning of the project.  Moving to a far more powerful dynamic client, capable of changing those substitutions, is possible with no additional layers of code.  The first solution of the static page is slightly simpler – but trying to extend that solution results in far more complex code.

In this paradigm, the layout is sent of JavaScript and just a placeholder div tag resides in the  ‘html’ tag, as sent to the browser. This is outlined fully below.

Different perspective?  HTML vs DOM

One perspective is to think of a web page as the html that describes the page.   Another perspective is to think of the web page is the DOM, and html is just data describing that DOM.  With this second paradigm, we could consider: “what if the DOM itself is described by javascript, not by the html code?”

This makes our web page:


  

<!-- first a json script to hold any json data  -->
{jsondata}

<!-- now here is the div where our main content will be added -->
<div id="page"></div>
<!-- now the script for the page content &gt;-->
  

  

As you can see, the html to describe page content is just an empty ‘div’ in the page.  So the DOM must get the page content part of  DOM from the kotlin DSL.  In fact this same html above, is now used for every page on the web site. The only part that changes is the {jsondata}, and  changing the {templatename} to the actual values for these to be used.  The sample above is perfect for using with a python format to substitute actual names,  but in testing if just sending the html file, then just set these to the values for testing.

What does the Kotlin DSL look like?

Of course the page above does nothing without the javascript,  because the javascript it generating the tags for the main part of the web page.   The only HTML is outside skeleton, and the main content of the page is described in kotlin DSL instead of in HTML.  Here is a very simple ‘main part of the page’ with just a

text

with a heading of ‘heading’ for content.

val div = document.create.div {
h1 { +"heading" }
    p { +"text" }
}

The document.create.div is needed for the very outer layer, and all the html tags inside become very clean and simple.  Using data within the page, or having tags produced in a loop, is all automatic by just using more of the kotlin language.

See the link: kotlin javascript tutorial for more on the DSL for html and how to configure Intellij and install the jar file for kotlinx.html

For a working example of the template scheme described on this page, see the ‘hypothetical programmable calculator’ as described in the page Machine code and Global Memory.  The code for the calculator with code for a kotlin DSL template example can be found in the repository.