Closures, methods, static methods, class methods, lambdas and callbacks

Everything you ever wondered about closures, and a few things more:

Closures: just what are closures?

Lets start with an example, and then discuss how it works.


def get_plus(increase):
    def plus(start):
        return start + increase
    return plus
}}} add6 = get_plus(6)
}}} add6(10)
16
}}} add8 = get_plus(8)
}}} add8(10)
18
}}} add8 is add6
False

The function get_plus returns the plus function.  It (get_plus) does not call plus(), but just returns the function plus without calling the code in plus.  When a program calls get_plus, at runtime a ‘stackframe’ is created for the get_plus function​. This stackframe will hold the parameters for get_plus together with any local variables of get_plus. In this case, the only local data is the parameter ‘increase’, but there could be any number of local variables and parameters… anything in the scope of get_plus is part of the environment of get_plus.

The ‘plus’ returned by get_plus, is actually not just  ‘plus’ but plus together with the stack frame, or local environment of get_plus, at the time plus is returned. This ‘code’ + environment is what is called a closure.

So get_plus(6) returns a copy of plus where increase in the environment is 6. Also get_plus(8) would return a copy of plus where increase in the environment is 8. Even though the code is the same, the environment is different for each plus, so the two calls do return something different.  A different closure, where a closure is the combination of function and environment the function can access. Thus add8 is add6 returns False, and the two closures returned are not the same.

Closures as code, packaged with Parameters

The plus function adds two numbers, start and increase. These can be considered the two parameters to plus, one from the parameter list, the other from the surrounding scope.  Every call to get_plus(), returns a copy of plus,  packaged together with a value for increase .  The same plus code used every time, but prepacked each time with a value for increase.

Methods as closures: Python.

Consider the following example


class Adder:
    def __init__(self, increase):
        self.increase = increase
    def plus(self, start):
        return self.increase + start

}}} plus6 = Adder(6).plus
}}} plus8 = Adder(8).plus
}}} plus6(10)
16
}}} plus6 is plus8
False

The above code shows how a class can provide a very similar solution to the prior closure example. A method of an instanced object behaves exactly in the manner of a closure, with ‘self’ packaged in the environment of the method.  When calling ‘plus6’, there is no need to supply the object created by Adder(6), because plus6 has this self in its environment.

Also note that for two objects of the same class, the methods are actually the same.  The is test gives false when comparing the plus method of Adder(6) with the plus method of Adder(8).  Each object has its own plus, not all sharing the same plus method.  That is because each plus is actually a closure with the self of the relevant object,

Now consider the types.


// using class Adder from previous example.
}}} add6 = Adder(6)
}}} type(add6.plus)
{class 'method'}
}}} type(Adder.plus)
{class 'function'}
}}}Adder.plus(add6,8)
14

The ‘plus’ attribute of the ‘​add6‘ instance of ‘Adder’ is a method, but the attribute of class ‘Adder’ itself is a function.  In fact, within the Adder class, ‘plus‘  is just a regular function, and can be called when provided regular parameters including with an object for self as in the example with ‘Adder.plus(add6, 8)‘.  Each time an Adder object is created, that object gets its own closure of plus with bound with self for that object, and these closures are called methods.

Methods as Closures: Kotlin

In Kotlin, methods also function as closures, but in a different way.  The code in a python method accesses the object through the ‘self‘ object, in the same way code outside the class could instance an object of the class, call that object self, and access the attributes of the class. Kotlin code does not access attributes through an object, and ‘this‘ when used within the class is a name qualifier, ‘this.plus‘ signals does not signal attribute access using this as an object.  All data for the instance of the object is within name scope.  The instance of the object is the closure environment.  Defining a function within a class is just like defining a function within a function.  For a function within a function, all attributes from the scope of the outer function are available to the inner function.  A method in a class is just like that outer function.

So in Python, all attributes of the instance are available indirectly through the ‘self‘ object being in the closure environment, in Kotlin, all attributes are available directly as they are all in the closure environment.

Static Methods.

In python, static methods are simply functions that although declared within the class, are simply functions, and like regular functions, do not have any closure.  Kotlin does not have static methods, and instead uses

Consider a static method added our python Adder class.

 

class Adder:
def __init__(self, change):
self.change = change
def plus(self, start):
return start + self.change

@staticmethod
def minus(self, minus):
return start- self.change

}}} add6 = Adder(6)
}}} type(add6.minus)
{ class 'function' }
}}} add6.minus is Adder.minus
True
}}} add6.minus(add6, 10)
4

The above code shows the following:

  • the static function remains a function, and does not generate a method
  • the static function is the same as the function in the class level object, and would be the exact function for every instance of Adder
  • an instanced object can be passed as a parameter to the static function, just pass it as any normal parameter

Class level methods / companion object methods

Python and Kotlin class level objects are already covered here, but the thing to add for completeness on this page is that Python Class Methods provide a closure on the class level object itself, allowing indirect access to all other class level attributes, and Kotlin companion object methods execute in the closure of the companion object itself, allowing direct access to all companion object attributes.

Callbacks, Lambdas and putting the knowledge to use.

Callbacks.  A GUI button may need to call back to the application, in order for the code linked to the button to execute. A communication routine may send a message, then execute a call back to call the application routine to handle the reply when it is received. Call backs are a common way have code triggered by events that may occur at some future time.  But how does the call back get the required parameters?  Answer, closures. Either simple closures, method closures, or a combination of both. A method can be supplied as a callback, and without the caller needing to know what object the method is related to, being a closure, the method has access to all object data without the caller needing to supply the object as a parameter.

Lambdas for callback: However, the best method for any callback is to provide a lambda to the routine requiring the callback.  An example could added here is anyone asks.

Advertisements

OOP: Using Kotlin to improve OOP Code

Ok, Object Oriented Programming (OOP)is not exactly new, and there are lots of pages on OOP on the web.  This page comes from a background of both Python and Kotlin, and how improved programming styles offered by Kotlin can change how you program.  Although Kotlin automates newer techniques and provides assistance coding, you can ‘backport’ many of the ideas to also improve Python Code.

First a recap of the basics, then how OOP is evolving and how the newer Kotlin enables newer thinking that is more difficult to embrace while only coding with Python.

Recap – The Concept

The first programs only dealt with numbers. Then, the letters were mapped to numbers and then, even though computers were still dealing with numbers, they could appear to dealing with letters and text. It turns out anything can be represented as set of numbers, the number of ‘types’ of data expanded.  This created the complication that when operating on data, ‘add’ or ‘print’ needs to execute different code depending on the data type. OOP represents a breakthrough in that the program can define its own types, and an number of operations for each type, allowing the possibility of any real world ‘object’ to be represented within a program.  At least in theory anyway.

Basics Recap: The four pillars

There are four traditional pillars of OOP.

  1. Abstraction
  2. Encapsulation
  3. Polymorphism
  4. Inheritance

Abstraction.

Strangely,  web search for some a good explanation found instead several answers that confused abstraction and encapsulation. These describe abstraction as hiding the details inside the class, which is not abstraction, but encapsulation.  Abstraction is to define what an object is in an abstract manner, so the details are not needed by code that uses the class.required.  For example, consider a duck.  A duck quacks, waddles and flies… and what else?  Well, each duck has size for example. At an abstract level exactly how it does these things is immaterial, so perhaps ‘waddle’ is just how a duck ‘walks’?  The idea is to reduce the ‘attributes’ of a duck to the simplest form, and only those that a relevant.   Another example is a automatic coffee machine. There can be a button that grinds beans, tamps the ground coffee, heats water to the ideal temperature and by pressure forces the heated water through the grinds, then froths milk and adds it to coffee.  But at an abstract level we describe this button as ‘cappuccino’.  We don’t care how it works, as long as it makes a cappuccino.  Now what else does our machine do that is relevant.

Encapsulation.

We have our abstract concepts of the attributes of object. Encapsulation is ensuring all the details of how this is achieved is kept internal to the object. Think medicine inside a capsule.  We see the outside of the capsule, not what is inside unless the capsule breaks! For our coffee machine, encapsulation means that if cappuccino is a function of the machine, then using the machine should not require understanding what steps are in the process.   With software, the idea is the internal steps can change if they remain internal.  If a separate ‘grind’ button is offered, then it will need to always work the same way, but if ‘grind’ is only a step within ‘cappuccino’ we don’t have to grind the coffee ourselves with a separate control if . If the machine does not have a button for the operator to do their own grinding, the how the grinding is activated can change during production and the instructions will not need changing as this become an internal change.  Encapsulation means the instructions to use the class, and thus code that uses the class, will not need changing as the internals of the class are updated.

Polymorphism.

We had a class of object as ‘duck’.  This allows for having any number of objects of the duck class.  Lots of ducks.  Now consider these are either robot ducks, or 3D animations, and we want to have them quack and move.  Our duck has waddle, fly and quack methods and each duck can have a size. So we can have ducks the quack and move in two ways.  But now consider if we also want other animals?  Perhaps we want a dog that could chase the ducks?  The dog does not waddle, fly or quack so the logic to control the dog must be completely different…. unless….. what if we redefine our duck interface as move(speed)  talk(howLoud).   Moving below a certain speed for the duck would be waddling and above the threshold flying.  Talking for the duck is quacking .The dog moves by walking or running depending on speed, and bark to talk.   Now we have a common interface, and ‘polymorphism’ as we can morph both duck and dog to the one interface.

Inheritance.

Polymorphism described a common interface for Duck and Dog which we could call ‘Animal’.   With Inheritance, a base class of ‘Animal’ is declared, then Duck and Dog classes would inherit from the base class.  A class that inherits is also called a ‘subclass’ as it can be said that ‘Duck’ is a subclass of ‘Animal’.  Animal would have methods for move(speed) and talk(howLoud) as well defining the property ‘size’.  The two methods would not need an implementation as the idea is that ‘Animal’ as a class is designed only for subclassing, called an ‘abstract class’, and the methods for Duck and those for Dog will likely have little in common, so an implementation in the base class would not be very useful.

Duck could also be subclassed to ‘SpottledWhistlingDuck’ and other species of Duck.  The SpottedWhistling could use common code for move  (both fly and waddle), but may need a very different ‘talk’.

What has Changed with OOP.

So how has OOP changed over time?  One of the most significant ways is ‘composition over inheritance‘, which basically means that most often, the role of inheritance should actually be best achieved by ‘composition’, which was not one of the original four principles.  Multiple inheritance is now particularly frowned upon, and where possible, composition is considered the preferred way to code, not even single inheritance.  There are many articles on composition over inheritance online, and here is one specific to python, and the author astutely uses ‘ducks’ in his examples.  Further reading encouraged if this is new to you.

OOP in Python.

Python allows almost everything you could want in OOP, but there are many things you have to do yourself without actual language support.

Key features:

  • python (unlike Javascript initially) does have classes and a unified type system with a common base class (object).
  • All data in python is an object
  • inheritance:  single and multiple inheritance support
  • visibility modifiers:  unusual, but effective
  • abstract classes: no Python language support, and no support while coding but these can be implemented to work at runtime, but it is tedious
    • Use
      • from abc import ABC, abstractmethod
      • this works at run time only, errors are detected only when and if an object is instantiated in testing
  • interfaces:  no real support, but the result can be achieved, although all checking will be at run time, and requires additional unit tests to be safe code
  • delegates: again, no real language support, but that doesn’t stop python having delegates, it just makes the code a little less clean and means there is no IDE tool support when coding

OOP in Kotlin

Kotlin not only allows almost everything you could want in OOP, but gives language level support, delivering compile time checks that eliminate errors possible with python, enables tool support when coding, and provides a guide.

Key features:

  • Kotlin  (like Python and unlike the initial Javascript) does have classes, and a unified type system with a common base class (Any).
  • All data in Kotlin is an object
  • inheritance: single inheritance only, multiple inheritance by interface(as generally recommended by coding guides)
  • visibility modifiers: full extensive language support
  • abstract classes: fully supported at language level
  • interfaces: fully supported at language level
  • delegates: fully supported at language level

Summary

While almost everything can be done with Python, visibility control, abstract classes, interfaces and delegates, which are all key OOP tool, and supported at language level with Kotlin making perfecting the techniques easier.  Perhaps learn and perfect in Kotlin, then backport to Python.

Building Applications: Gradle DSL Kotlin 1.0

Big News!  (Aug 2018)

After what seems like the longest time, Gradle support for using kotlin to drive gradle builds is now at version 1.0.  This is the best time for anyone to move to gradle.  This page is an introduction to gradle, using kotlin DSL

  • What is a build tool?
  • Which Build Tool?
  • Build Basics in Gradle
    • Plugins
    • Dependencies and Artifact Repositories
  • A Simple Practical Example
  • Varying the Source Locations
  • More Options

What is a build Tool?

As Building software becomes ever more complex, build tools come to the rescue.  A build tool is simply software designed to automate the build process, and at the same time, keep building as simple as possible.

Think of a simple batch/command/shell file (for windows/maxOS/unix respectively).  In the simplest form, it can capture a command or sequence of commands and save typing them and remembering all the parameters.  Working with building programs can require just such a sequence of commands, and all build tools are simply ways of running a sequence of commands.

So why not just use a simple batch/command/shell file?  You can, but for complex projects it just becomes a mess.  The reason is that rather than remaining simple, as projects scale up, the logic required can become far more complex, and there are some patterns that frequently occur with the commands needs to build programs, like only do a particular step if the file has changed, which benefit from specific logic.

Which Build Tool?

Make was one of the first build tools, but makefiles became painful to use as software evolved and building needs increased, then followed ‘ant‘ which introduced a configuration language based on xml.

Maven came next, and brought integrating the build process with internet, integrating online repositories of artifacts, and keeping XML configuration..

Now we get to Gradle.  The innovation of gradle was to move to a program for configuration.  Being a specific ‘domain’ the goal was a DSL (domain specific language) and at the time gradle emerged Groovy was chosen for the DSL capabilities of the language.  Later, the Kotlin language appeared, and became even more appealing in general with arguably even more useful DSL capabilities, and now gradle scripts can be written in Kotlin as well as Groovy.

Maven and Gradle access the same artifact repositories, and that system is becoming an industry de facto standard. Gradle at this time (Aug 2018) combines state of the art build capability with configuration in Kotlin.

Build Basics with Gradle.

The build file prepares a configuration for gradle and for any optional extra build tools.  The basic configuration for a build requires:

  • what tools will the build need?
  • where are the source files?
  • what libraries are needed?
  • where are artifact repositories (holding the libraries)?

Consider the simplest case for each of these basics:

  • What tools will be used for the build? A:Kotlin – the jvm version
  • Where are the source files? A: ‘src/main/kotlin’
  • What libraries are needed? A: the kotlin std library
  • Where are the artifact repositories for the libs? A: jcenter

Providing only these for a configuration is the simplest case, and here is a build.gradle.kts file for this simplest case.


plugins { kotlin("jvm") version "1.2.60" }
dependencies { compile(kotlin("stdlib")) }
repositories { jcenter() }

The tools are added through ‘plugins’, in this case the kotlin(“jvm”) plugin is all that is needed.

If the source files are in src/main/kotlin, then as this is the default, no instructions on source files are needed (how to add more source locations is covered below).

The dependency is the standard kotlin compile library, which is in the artifcact repository specificed by ‘jcenter()’.

Plugins.

The main plugin for kotlin requires what type of kotlin as a parameter, and in the current version of Gradle Kotlin DSL (1.0 rc) requires a modifier specifying the version of the plugin (currently  1.2.60).

Dependences and Artifact Repositories.

Dependencies are libraries, code, images… any component of any type that is part of a program distribution is a dependency. The most common dependencies are library functions and classes.

To build a program, the build tool needs to know to find the depenencies, and the normal system is to have the dependencies held in maven style artifact repositories.

This means in addition to the what the depnencies are, it is also necessary to list the repositories to search to find each dependency.

A simple practical Example

  • Step 1 Install gradle
    • if you don’t already have gradle installed, follow on online instructions to install and verify installation
  • Step 2 – build simple example in Intellij
    • Create a new kotlin jvm project in Idea
      • File/New/Project/Kotlin/Kotlin JVM
    • Note the blue ‘src’ folder in the project (blue indicates source files)
    • Add a kotlin file with a main method to the src folder
      • right click on folder, then New/Koltin file/class
      • add a main function with a print or something
        • fun  main(args: Array<String>){
      • check for the green triangle (Play symbol) next in the gutter to the left of ‘fun’
        • argument must of an Array String for the symbol to appear
      • click play to get the program to run.
  • Step 3 – add build.gradle.kts
    • add build.gradle.kts to the project root
      • use New/File .. not New/Gradle_Kotlin DSL build script
        • at the time of writing the DSL build script does not work
      • add the text of the 3 line example in build basics above
      • exit and restart the project
      • select ‘import gradle project’
        • the import may bring up an options window
          • select no itck boxes,  ‘use local gradle’
          • other settings should be able to left at default
    • the green play symbol will disappear as the project does not match the new settings
      • add a ‘main’ folder to src, and a ‘kotlin’ folder to main
      • move the sample kotlin file to the kotlin file
    • The new configuration should now work!
      • find the gradle tab on the right edge of the intellij window
        • open the gradle tab
      • if the gradle tab cannot be located..
        • use the main menu :  view/tool windows/gradle
        • use the expand triangles to locate tasks/build/build click build
  • Congratulations – you are on the road to being a gradle master

gradleGradle Settings.

Auto import can be annoying.

Create directories for empty roots is best for the more experienced.

These settings do work for the example.

Varying the source locations

In the example above, the source file must be moved from ‘src’ to ‘src/main/kotlin’.  There is some sound logic to the move, as some sub folders under ‘src’ make sense, particularly to allow for both ‘main’ and ‘test’.

In place of moving the file to suit gradle defaults, you can move add ‘src’ to be a source file location.


java.sourceSets["main"].java {
    // srcDir{"src/main/kotlin") - this is the default, always set!
    srcDir("src")
}

Why java.sourceSets[“main”].java and not kotlin.sourceSets[“main”].kotlin? Maybe it will change in future, but for this example creating for the jvm [“main”] is for the main file location. [“test”] for the test files locations.

More Options…to be added

Building in Intellij IDEA

This program is relevant for building your first “hello world” and first sample kotlin apps.  However, once you have even hello world running, it is recommended now to move to gradle for building, even when building with Intellij.

The topics for this page:

  • Introduction: why use Intellij at all.
  • The Roles on an IDE
  • What is Building?
  • Installing Intellij and Kotlin.
  • When to Build using Intellij
  • When to Build using Gradle

Introduction.

To install kotlin, the easiest way is to install Intellij IDEA.  Just as ‘idle’ comes with a python install from python.org and provides a GUI environment for typing in and running python programs, Intellij performs the equivalent steps for kotlin programs.

In fact, Intellij can also take over the role as an editor and much more for your python projects as well, which saves keeping two environments for the two different languages. This is not recommended when starting out unless you already use pycharm for python, but worth keeping in mind.

The Roles of an IDE

Hopefully before reading these pages, you have had significant exposure to an IDE for python (or another dynamic language)/ perhaps even using PyCharm from JetBrains.

If you are not already familiar with the role of an IDE, it is already explained in many places. An IDE handles editing, building, debugging and even potentially deploying programs but for more explanations follow the links.  The pages are primarily for people with some experience, moving from other languages to kotlin.

The focus for these pages is how the role of an IDE changes when moving from python (or another interpreted dynamic language) to kotlin. The main difference is that configuring the ‘build’ aspect of a project becomes far more important.

What is Building?

Building is the combining of all the parts of a program into one package ready for the program to run.  Python mostly uses a ‘environment build’ approach, which hides the build in simply programs, just as Intellij does to a lesser extent with Kotlin.

see: Building, no need with Python? For compiled languages like kotlin, the packages (and other resources)  should be combined with the program, so you need a build.  The advantage is that once built, each program contains its own environment, simplifying testing and distribution. See

Build systems have been around for a very long time, and evolution has seen great improvement.  Although some notable large scale python projects like linkedin use build systems, usage in python is scattered so moving to kotlin can be the first exposure to build systems.  Note that in an ideal world, even python project would use the same build system as languages such as kotlin  (as is the case with linkedin).

Installing Intellij and Kotlin.

The only limitation of the official kotlin lang getting started guide is that it is targeted mainly at people who already know java.  First install a Java Runtime Enviroment (JRE). Search “install JRE <your platform>” and you should find clear instructions. For more background read this page, but just installing a JRE is all you need to start.  Note you can install OpenJDK in place of the default with no problems.

Then (after installing JRD/JDK) follow the kotlin-lang “getting started”.
If anyone makes me aware of any problems I will add more comments here.

C-Lion is an alternative IDE for Kotlin, targeted at the production of native applications.  At the time of writing this is a more advanced approach not really suited to those who are not already using kotlin.  TLDR; Use Intelij IDEA before moving to C-Lion for your native apps.

When to build using Intellij IDEA?

tldr: As soon as you can after getting hello world running.

As an IDE, Intellij supports its own internal build system. Intellij also supports the official external build systems of Ant, Maven and Gradle, as well as third parties offering their own build systems to work with Intellij.

If the internal build system of Intellij was a great system, then why support for so many alternatives?  In the end, build systems are an art form of their own, and the sooner a developer moves to a dedicated build system, the better.

The Intellij internal build system is a good choice only for small projects with very simple build requirements.  Once a programmer has mastered an industry standard build system, there is generally no longer any need to ever use the internal build system again.

When to Use Gradle, and when you need Gradle?

If you know how to use gradle, why not use gradle all the time?

Just how workable it is to use gradle for every project is a work in progress for our team right now, and we will update this page soon.

However, the clear trigger of when you need gradle is when you need to use a package or artifact that provides instructions of how to add the package/artifact via maven or gradle.  There is a work around, but for almost all packages instructions are given to use either gradle or maven to add the resource to your project.

Another case where you would logically use gradle is when building your own artifacts either in the equivalent of hosting a project on pypi or for simply sharing within your own team

 

Software lifecycles: Single Use, Single Development or Continuous Development?

stage-of-development-clipart-10A key defining characteristics of software is the software lifecycle. Some of the most significant software ‘screw-ups’ and dissatisfaction can come from applying software development methodologies appropriate for one software lifecycle,  to a very different software lifecycle.  Best practice for one software lifecycle should not be assumed to be best practice for a different lifecycle.

Many software developers and businesses predominantly experience one lifecycle category, so are unaware just how different things are on the ‘other side’. This page looks at things how software changes vastly based on the lifecycle picture:

  • single use software
    • lifecycle: all software development ends after ‘acceptance tests’
    • definition: software requires zero maintenance
    • examples: single use software, control software, prototypes
    • methodology: rapid development
  • single development software
    •  lifecycle: maintenance only after ‘acceptance tests’
    • definition: single acceptance/certification software is ‘complete’
    • examples: IT department projects, some software product versions
  • Continuous development software
    • lifecycle: development continues until software is ‘end of life’
    • agile product software / Software as a service
    • definition: software in continuous ongoing development
    • examples: products software with frequent updates, product web sites such as Amazon, Facebook etc
  • Spaning Lifecycles.
  • The impact of lifecycle on development
  • Conclusion (TL;DR): It is all about the Spec and The Tests.

Single Use Software

Definition: Once it ‘works’, the code will never need to changed.

Lifecycle: The source code is effectively ‘dead’ once the code is judged as complete.

Examples:

  • Tutorials, Education exercises and assignments
  • Data conversion software and single run reports and data analysis
  • Some embedded systems
  • Software for environments that will never change
  • The very first computer programs

Why write software for a single use?  This is not how most of us think of software.  However, consider how we learn to program.  Once the software is marked, it has served its purpose and will normally be abandoned.  In fact, most of us learn to program writing almost only single use software. Over and over we improve this skill.

Once education is complete, the use of the skill in writing single use software is more specialised, but does still happen.  A program to migrate a database from one format to another will no longer be needed once the software has been successfully migrated. Many clinical trials or other scientific data collection collect only one single data set.  Reporting on this data often requires single use software as there is only one set of data to report on.

Some software has only a single use, but that single use occurs over and over without any requirement for the software to ever change.  A toaster could contain embedded software of this type. The operating system will never change, nor will any of the toaster hardware during the production.  If any new model will be a complete redesign, then the software will be single use.  However, if there will be an evolution of products using similar software, then this use case stops being single use.

Single Development Software

Methodology:  Rapid Development.  There is a range of software under this heading, but most single use software is developed rapidly, effectively in a single sprint (one sprint = waterfall), to either a pre-existing spec or to a spec simply in the mind of the developer.  While TDD may be justified, there is no real need for unit tests and there simply wont be repeated testing.

Definition: Bespoke Software, written to meet a specific identified requirement, and then judged as complete once the software passes acceptance tests.

Lifecycle: Once the software meets ‘acceptance tests’, the main software development is complete and ‘mature’, and maintenance is then the only ongoing development.

Examples:

  • Contracted Developments for an IT Department
  • Most IT department Internal Projects
  • Discrete, Large Scale Software Product Releases

Most of the IT industry is involved in the production of Bespoke, Single Development software.  The classic example is a software system written for a government department or large corporation to automate data collection and data processing. While the key characteristic of Bespoke software is an acceptance test process, which could seem to preclude product software, individual releases of product software can be also produced as bespoke, single development projects, with product management in the role of the development customer.

Methodology: Usually waterfall.

Continuous Development Software

Definition: Developers continually modify code to add enhancements with new functional versions available daily or at least monthly, even if many of these updates are labelled ‘early access’ or ‘insiders’. New major releases are usually available on regular intervals and the ‘acceptance test’ is gone and replaced by automated unit and integration tests which are run not to be passed once, but to be passed as often as daily.

Lifecycle: Development never ends while the software is active.

While with Single Use and Single Development software, the software is released for use when development has finished.  The announcement development will finish for continuous development software will lead to all users of the software moving to an alternative.

Examples:

  • Python
    • release whenever chosen stories complete
  • Android
    • release around August each year
  • Windows 10
    • new releases on six month intervals (1803 for 2018-03)
    • Previous Windows seemed more like Single Development
  • Ubuntu
    • Released April and October each year

Methodology: Unit tests and easy maintenance are as important as  functionality. With Single Use/Single Development software the goal is simply to get the functionality working and pass acceptance test. With continuous development software, functionality added today will likely be as waste of time without easy maintenance and unit tests. The code will soon be dropped by a subsequence developer if they cannot understand the code or did not even know the functionality was ever there.

Spaning Lifecycles.

Agile really only came into existence with the agile manifesto in 2001.  Prior to that, waterfall was the most followed approach.  That there are today so many continuous development projects mean some teams and projects move from one development lifecycle to another.

A large bank, may have their own internal website and//or mobile apps now developed as continuous development projects, but there internal banking systems developed as traditional waterfall with the external management giving comfort.

The Impact of Lifecycle on Development.

A continuous development lifecycle requires agile developers in place of waterfall programmers, analysts and project managers, and at least compare to waterfall programmers, this is a greater skill set which can result in a more expensive team.

Further, to reach a given milestone, more effort is generally required.  It takes longer to write concise readable code than just get a solution working.

However, a team developing as a project a series fixed developments for each release will tend to plateaux – with new releases failing to be seen as a step forward, as the less readable code is continually replaced from scratch.  This leaves developers starting again over and over rather than standing on the shoulders of others, and not being able to reach the same heights.

Of course, continuous development without sufficiently readable code will have the same plateaux effect.

Conclusion: its all about the Spec and Tests

The role of the spec.

The form of the spec determines the lifecycle, which in turn determines the suitable development methodology.

  • Single use software often has a spec which simply lives in the authors mind.
  • Single Development software typically has a well formed and detailed spec, and the agreement of that spec, and a timeframe and/or cost to deliver that spec may be the basis on which the development project is launched.
  • Agile Developments have an ever evolving spec, and with continuous development projects, sometimes even evolving requirements.

Single use/development software is coded to a fixed spec, which is optimised more for waterfall, while coding to an evolving spec requires agile.

The software is ‘complete’ when final tests are met.

The Role of Tests

The reality is, in all cases, the tests become specification of the actual system.  The tests represent what the system is measured to deliver, and the spec documents are relegated to ‘design goals’.   Think of a car developed with to a specification, the car engine should develop 100kiloWatts of power. Once the car is developed, the real output is measured, and that goes on the specification.  The original ‘spec’ is now considered a design goal.  You set out to build a bookcase, and want it 1 meter wide.  But the real ‘width’ spec is determined by measuring, not by what the goal was.

If software was developed under contract, then the contract is paid once acceptance tests are passed.  If a feature was in the ‘spec’ but not in the acceptance test, then it may not be in the software.  The spec was really a design document for both the code and the tests.  But in the end, what is measured is the real specification.

The key point is that if a feature is not included in testing, it may not be in the system.  With continuous development, if a feature is not included in tests, it may not stay in the system even if it was there at one time.