Implementation: what is a practical approach?

Any software team who is considering moving to kotlin, must by definition, be currently using at least one alternative language.  To change languages, and ecosystems, is a big step.  One of the key features of kotlin is how easily and seamlessly a project can migrate from java.  Currently, that same ease of migration is far less real from outside the java ecosystem.

Cold Turkey? Or step by step?

On rare occasions, there may be the opportunity to commence a complete new project and build each component with no basis on any legacy system.  If starting an entirely new project but not already experienced in kotlin, it will still require a huge leap of faith to start an entire development in kotlin.

More often, and in the project we are currently working with, the realistic path is to choose system components that can move to kotlin.

The candidates:

Individual pages discuss these sections, but the spoiler alert is that mobile/android development may not be the logical first choice it would on the surface seem.

Kotlin DSL templates with Python (or any other) Server

The Concept: A replacement template engine written in Kotlin DSL

Kotlin can replace the template system for a python server, or ruby server, or any other server, with no change to the server other than the template system, regardless of what language is used for the server itself.

This allows replacing mako, or jinga2,  Django templates, with kotlin DSL templates.  No kotlin or jvm installation is needed on the server, as the kotlin dsl templates can run as javascript in the browser.

Why? : A more dynamic and concise solution

Template engines are generally considered a method of producing ‘dynamic’ web pages.  While a ‘static’ web page always displays the exact same html, templates produce html which reflects the data presented to the template.  For example, a ‘member’ page will have the information for the member currently logged in,  while a static home page will display the same information to everyone.

However pages generated by templates are not necessarily ‘dynamic’ in a web2.0 manner.  The page is ‘generated on the fly’ by the template engine, but does not necessarily run dynamically in the client browser.  To the client browser, the page appear static.

These kotlin DSL templates are a complete rethink of how templates work, and inherently produce code that is also dynamic on the browser and the entire system makes adding true dynamic content far simpler.

Further, the description of page content becomes more concise than with conventional templates.

More Languages? Or Less?

It could be seen that adding kotlin to do templates means adding yet another language to the toolkit in use with a project, however this is only proposed as substitute for mako, jinga2 or some other template language, so there is also one language no longer required.

Kotlin is far more complex than any template language, but using the kotlin DSL as described here does not require learning the full kotlin language.   The further benefit is that learning a template language just for templates has no other uses, while it is very likely there are other possible uses of kotlin (e.g. Android?)  within a project.  If kotlin has an additional use within the project, then using kotlin for templates could mean one less language overall.

How do these Kotlin DSL templates work?

A conventional template is like a more powerful version of the python format function.

Consider:


"person: {name} age: {age}".format(name="fred",age=35)

This is equivalent to the string being a template that is supplied data in the form of the ‘name’ and ‘age’.  The string is modified by the data, to produce a final string.  Templates just allow more power with modifying the string.

The kotlin DSL templates I am discussing, actually run in the browser, not on the server.  The template can look similar to html, but it is code, not a string.  Using simple python format statement for a simple example of templating, the template might be:

<html>
    <body>
person: {name} age: {age}
</body>
</html>

and by processing the template with our data, the final page is prepared on the server and sent to browser.

However with the kotlin templates discussed, our layout inside the ‘html’ tag, is not present inside html sent to the browser, and in place of the content will normally be an emply placeholder ‘div’ tag.  Javascript code to instance the ‘body’ and ‘p’ tags inside the placeholder ‘div’ is sent as the template, and this code will read the jsondata (where present) and provide the correct content inside the div tag.

Different perspective?  HTML vs DOM

One perspective is to think of a web page and the html as basically synonymous.  Another perspective is to think of the web page is the DOM, and html is just data describing that DOM.  With this second paradigm, we could consider: “what if the DOM itself is described by javascript, not by the html code?”

This makes our web page:

<html>
  <body>

<!-- first a json script to hold any json data  -->
<script id="data" type="application/json">{jsondata}</script>

<!-- now here is the div where our main content will be added -->
<div id="page"></div>
<!-- now the script for the page content >
  <script src="{templatename}"></script>

  </body>
</html>

As you can see, the html to describe page content just an empty ‘div’ in the page.  So the DOM must get the page content part of  DOM from the kotlin DSL.  In fact this same html above, is now used for every page on the web site. The only part that changes is the {jsondata}, and  changing the {templatename} to the actual values for these to be used.  The sample above is perfect for using with a python format to substitute actual names,  but in testing if just sending the html file, then just set these to the values for testing.

What does the Kotlin DSL look like?

Of course the page above does nothing without the javascript,  because the javascript it generating the tags for the main part of the web page.   The only HTML is outside skeleton, and the main content of the page is described in kotlin DSL instead of in HTML.  Here is a very simple ‘main part of the page’ with just a

text

with a heading of ‘heading’ for content.

val div = document.create.div {
h1 { +"heading" }
    p { +"text" }
}

The document.create.div is needed for the very outer layer, and all the html tags inside become very clean and simple.  Using data within the page, or having tags produced in a loop, is all automatic by just using more of the kotlin language.

See the link: kotlin javascript tutorial for more on the DSL for html and how to configure Intellij and install the jar file for kotlinx.html

For a working example of the template scheme described on this page, see the ‘hypothetical programmable calculator’ as described in the page Machine code and Global Memory.  The code for the calculator with code for a kotlin DSL template example can be found in the repository.

OOP vs Functional: No contest!

Moving to kotlin, the question can arise: “should I program in OOP (Object Oriented Programming) style, or move to Functional programming?”.  This page examines reviews what Object Oriented programming really means, what functional programing is, and outlines how there is no incompatibility if the correct approach to OOP is in place, and the two paradigms are best when combined.

Functional Programming vs Programming with functions.

What is FP?

Almost all modern programs use functions, but just using functions is not Functional programming.  What is required is functions that are ‘functional’ in the mathematical sense, not programming in the ‘using functions’ sense.  A mathematical function, or ‘pure function’ operates on the supplied arguments and returns a result and does nothing else. No ‘side effects’. Nothing changed by the function, no internal variables altered that will result a future call of the same function dealing with different values.

The best test for a ‘pure function’ is: will the result, and all aspects of calling the function be identical every time the function is invoked?

Limitations of ‘pure’ Functional Programing.

Any ‘side effect’ of calling the function disqualifies the function from being a ‘pure function’, as the only output from the function is the result of the function.  The reality is a program without any ‘side effects’ is actually useless, as input/output is generally a side effect.  The answer can to a problem may be able to be calculated using ‘pure functions’, but actually showing the answer is going to require input/output.  So while a program may incorporate ‘pure functions’, no program will be entirely functional program.   The second important point is that ‘pure functions’ can be built in any language.

Language requirements.

Languages that supports functional programming give tools to make it easy to see what code will qualify as a ‘pure function’, and what does not.

OOP vs non OOP programming.

What is OOP?

OOP means different things to different people.  To some the ‘class’ statement is the essence of object oriented programing, yet while ‘javacript’ has no class statement, but supports object oriented programming.

A more useful view of object oriented to think of ‘classes’ simply as ‘types’.  A type permits operations on that type. These operations are ‘methods’ of that type, that is computing that can be done with that type.  When different types (e.g. ‘int’ and ‘string’) each have their own interpretation of the same operation (e.g ‘+’ or plus) then we have one of the anchors of being object oriented: polymorphism.  As almost every language has several different types which each have their own version of ‘+’, if follows that every language has some elements of object oriented. The core concepts of object oriented is present in almost every program, but object oriented programing is more than just some exposure to the concepts.  An object oriented program does not just use the type system and which will already have some degree of object oriented approach, rather, and object oriented program extends the type system creating its own ‘types’ or ‘classes’.  Further, a an object oriented program, is about using this approach as the basis of the solution.

The limitations of pure Object Oriented

Pure object oriented is usually stated to have four ingredients:

  • polymorphism: the ability of different classes to effectively all act the same in some way but implementation their own version of one or more functions or operations to operate in an equivalent way
  • encapsulation: the implementation of the class (or type) is coded within the class and  code using the class has no need to consider how the class is implemented and will not be impacted if the implementation changes
  • inheritance: classes or types can inherit or ‘sublcass’ from a ‘superclass’.  This enables both reuse of code, and is a key tool for building polymorphism when more than one class subclasses from the same parent has its own implementation of the methods of the superclass
  • abstraction: this is giving the class an interface designed around the logical use of the class, and not simply a reflection of the mechanics of the underlying code

Of these four usually quoted points, all but one are mostly about techniques to do OOP well, while encapsulation is really a test of OOP or not OOP. ‘Pure’ OOP code be built from purely from Objects with the code encapsulated within the classes.

The language Java provides a perfect illustration of why ‘pure’ OOP is a mistake. Where efficiency dictates, the despatch tables within classes required for encapsulation are a performance killer.  For performance.  low level, close to the computer objects are best being reduced by the compiler to direct inline code.  This does not stop such low level object classes adhering to the object model, or at least linguistically, being objects.  However Java even goes as far a simply dropping all pretence of objects, or Object Oriented code for ‘native types’.  In contrast to Java itself, the Java must write all code in classes.  The simple example of the ‘hello world’ which results in a pointless class wrapper containing the ‘main’ method, illustrates how even enforcing there are classes, does not create an Object based approach, and that for some code, there is no object based approach.

Language Requirements.

In python, a programmer could implement their own class system using dictionaries of methods and properties. But creating new objects requires allocating memory, keeping track of when an object is no longer needed and freeing that memory. In fact some ‘object oriented’ languages (like c++) require programs to manually control garbage collection (returning memory to ‘free memory’ when objects are no longer needed).  Every feature needed for OOP can be implemented with no special features at all, however is most languages ‘DIY’ would be laborious and distract from the application. No special feature is needed to do OO, but to do OO well, a good syntax and feature set is essential.

Functional vs OOP: No contest!

The optimal FP approach, is avoid side effects wherever possible.  The optimal OOP approach is for all data in the program to be an object.  FP sounds like focusing on the purity of the logic steps, OOP is about focusing on the data.  But FP programs still need data, and operations on data, and regarding that data as objects and empowering the data makes sense. There is no problem combining both paradigms in a single program.

Some argue that the OO philosophy encourages programming in a ‘state-driven’ manner that is the antithesis of FP, but this argument ignores the foundations of the best FP languages have OO based type-systems.

OO itself makes it easy to break FP principles, but also can make it easy to follow FP principles.  Kotlin does give more tools to support using OO together with FP style than python does, but this can be done with either language.  As a simplistic example, a ‘val’ in kotlin is a variable which can never be reassigned, with python a programmer can still have variable which is never reassigned, but the programmer will need document this and check themselves that the rule is followed.

Misguided OOP vs the OOP and FP combination.

Recently, I was speaking with a programmer who declared, “Object Oriented is dead, Functional Programming is the correct way!” He had a link to a video describing how OOP programs could break all the principles of FP.  OOP programs that behave this way are simply bad OOP programs! I have not found that video again, but I can recommend this (rather long) video by ‘uncle’ Bob Martin on functional programming (see 50:00 minutes in for the points on OOP with functional programming).  Misguided OOP tries to force everything, including functions, into objects.  A classic example of this is with Java where even the main function of a program has to be wrapped within an object that adds nothing.  Kotlin moves from the misguided OOP of Java in many ways, for example by making functions first class objects.  As this guide progresses, the theme of Kotlin adapting true, more mature OOP into the system, and in a Functional Programming compatible way, while still maintaining java compatibility, is repeated over and over.

Java: The king is dying

(originally written 23 May 2017)
TL:DR … but just read the headings in that case.
Java is the king.  In a similar way that the queen of England is ‘the queen’. Java is not everyone’s king, and some have another king, but even they may see java as the most recognised king.  Certainly, Java is the ‘king’ with the health problems.

Birth: 1995, An innovation, free software with slick professional marketing at the right time!

Java was born in San Francisco as a child of James Gosling in 1995, in the age before the internet had really taken off or allowed the collaboration we know today. The promise of the language was ‘write once, run anywhere’ and the language was fully designed to be ‘fully object oriented’.  The truth is Java did not deliver on either premise, but Java did bring the innovation of an efficient, and stable specification for the intermediate code interpreter referred to as the ‘Java Virtual Machine’. ‘C’ was the dominant language, and the early ‘C++’ extension did a very messy job of facilitating OO development, so there was a vacuum to fill.
Sun embarked on a mission to try and deliver ‘write once, run anywhere’.  They approached netscape about running in the browser, and when the answer was ‘it wont’ work, they asked ‘well could we call what would work java?’.  The approached Patrice Peyret about running in a smart card and ended up buying his company and naming another ‘non-java’ in JavaCard. They approached Cardsoft and STIP on another platform but people tried to really make it java and so it died.  But they never acheived ‘java everywhere’ except in having something ‘java’ in the name,   The OO design was that it forced applications to use objects, the language itself had no first class functions, and most variables were not objects. Messy, because design was in a vacuum compared to languages today.
But Java became cool.  A cool logo.  Named after an remote ‘exotic’ island with a reputation as an origin of coffee. Fun in a world of boring ‘Basic’, and ‘C’ and ‘Delphi’. ‘Java One’ conferences were the event to attend.  Sun was already ‘cool’ and Java bathed in that, and the marketing and became the language a hipster would program in. Lack of anyone pushing a real alternative, together with great marketing and the language being free resulted in Java taking off driven by its own momentum. Java became king.

Meanwhile….: 2000-2009 lost years in language, java holds on

By 2000, Java is already starting to decline from the peak. Microsoft now has its ‘won java; in C#, but with Microsoft not cool at the time and C# still new, the strongest competitor to java is still the cumbersome C++.  Rapid coding solutions scripting languages perl and php are on the rise and the better  ‘scripting’ languages like Python and Ruby gain some traction, but they still have ‘rough edges’. All these are regarded as ‘scripting languages’ that are not for real programmers, suffer performance limitations(particularly on artificial benchmarks), and more importantly, programs are difficult to distribute unless distributed as source.  The reality is businesses like yahoo, google others are all built using the competitive advantage of rapid coding with scripting languages, and as their software runs on servers the software distribution is not an issue. So the tech giants grow using newer technologies, but the ‘IT Deparaments’ stay with Java. By late 2004/2005 the secret of companies like google is not only revealed, google actually hires the original developer of python.  Around this time python jumps in popularity and Java declines, but the ‘quick and dirty’ php is still a favourite with millions of simple web sites.  Form 2008 through to 2012 android has halted the demise of java, the python 2/3 problem is in full swing, and nothing seems ready to fill the void.

The plotter gather: 2010 – 2017 contenders mount challenge

Every one realises java has problems, but with no clear successor to the throne, there is a void.  Scripting languages are still working on their limitations, and python is in the 2/3 phase.  Scala(first released beta 2004), feels it is ready for the main game, but it joined by Clojure (2007) and then GoSu, Frege, Kotlin, Ceylon, Fantom   ….so many contenders that it is hard to choose.  All are an improvement on Java, but which one improves on the others?  Plus there are new native languages R, Go, Swift and perhaps moving back to native is better?  Too many choices, so procrastination wins in many cases.

May 2017: A successor announced!

Google endorses Kotlin as a first class language on android.  This may seem, insignificant, but this is google anointing Kotlin as a successor to Java on Android.  Google is even promoting just how much of an improvement Kotlin is over Java, showcasing the savings in code of data classes over Java and other examples.  Not just to any old language is being ‘pushed aside’…  the ‘king’ is pushed aside.  When Apple anointed Swift as the successor to  objective-c, there was no real use of objective-c other than Apple and the language was far from a ‘king’.  Apple anointed Swift, but Swift was their own project, the anointment meets with some scepticism from many developers, just as would have happened if Google chose ‘go’ on Android.  What is interesting is just what happened to objective-c popularity.  See graph for the popularity of Objective C following Swift announcements. image2017-5-23_17-1-6

The Future Impact: A new king, but who will it be?

Expect Kotlin to jump into the top 50 on tiobe within around 1 month, and become the highest ranked Java alternative very soon.  Kotlin will enter PyPl and Java will come under threat of losing its top position on that scale.
Overall Java will have a significant dip, and Kotlin will have a ‘bullet’ following the announcement.  Ok, Java will not dip to same extent objective-c did, but it will be significant.  So Java will have a first phase of ‘dip’ because of the announcement, and then a second dip as a consequence of the first dip reinforcing that Java is now an ‘out of favour’ as a language.   Similarly Kotlin will get at least two ‘boosts’.  Most likely within a year it will be on all top 10 lists, and possible even ahead of the 3yr old Swift, although Apple is really doing some pushing with swift.
The real impact is that unlike Swift, which ‘poached’ programmers from Objective-C,  Kotlin has the potential to poach from Java, C, C++, Python, Javascript, and even others. There may even be enough impact on challegers to Java’s crown that these also lose some share and delay the inevitable abdication of Java,  but Kotlin being endorsed as effectively a better choice than Java has the potential to really shake up the industry.  Where it goes will depend on future decisions that may be likely, but not set in stone, just as google anointing Kotlin was likely, but not set in stone.  The less predictable part was that google would fund Kotlin moving to its own non-profit (but likely well subsidised) entity.
So now, a single language already targeting the JVM, the browser, and native code right down to Arduino with LLVM. This is new ground for computing.  Kotlin promises not write once run anywhere, but write key code once and reuse everywhere that code is relevant.  Only CLR is missing as a target, although there is a windows solution emerging.  How an ever more open Microsoft will respond is not known, but JetBrains is already producing a .NET IDE!   Another big question is whether a taste of success will derail JetBrains!
The future is does have questions, but this anointment of kotlin has the potential for a real change to the industry.  There will be a new king, and while Kotlin is an anointed heir to Java, it is not clear who the new king will be yet. While Kotlin may one day take the throne, it is unlikely to be the very next king.