AMMERSEDesign+MethodologyHome

Proposal for Compilers and Runtimes to combat Software Entropy

Software Entropy is incurred in all software projects from the start of the first line of code. Software Entropy described by Ivar Jacobson et al is a term used to describe the exponential complexity that evolving software brings. There are no means of stopping entropy. There is no avoiding it. Or can we do something about it?

Code becomes complex because of how we think and how we write it and not all code is equal. Give 100 developers the same problem and you will have 101 solutions.

Let’s look at the Layer architectural pattern and how it can affect the complexity of the code. (There is a brief description here of the Layer, for those who do not know the ways of code directly)

Looking at complexity from the Layers pattern

A Layer is an abstraction. The idea is that the layer (a piece of code) is dedicated to a particular function. Typically you may find a ‘user interface layer’, a ‘business layer’ and a ‘data layer’. These three are common and coders use it to categorize and separate code.

The UI layer may speak to the business layer, which in turn speaks to the data layer. This is known as a closed layer system. The Data layer is not accessible directly by the UI layer. You may want this, to separate concerns and offer some sort of protection from the impact of change.
An open layer system may be where the UI also communicates with the data layer as well. Therefore the UI->Business->Data can exist as well as the UI->Data. The business layer can be bypassed.

Now let’s look at entropy with regards to open and closed layers. To do this, we will simply assign a value of 1 to represent a form of complexity. A layer that communicates with another layer would get +1.

The closed layer system has the benefit that it does not have any interaction with some of the layers. I am aware of Shannon Entropy, but let us assign a value to each dependency to get some numbers to represent complexity instead.

** Closed Layer**
Data is closed to UI. You cannot move from UI directly to Data.

  • UI->Business +1
  • Business->Data +1

Complexity total = 2

** Open Layer**
The same layers, but now UI can move to Business and Data.

  • UI->Business +1
  • UI->Data +1
  • Business->Data +1

Complexity total = 3

A design that implements a closed layer has an advantage over the open layer in terms of entropy. In this example simply a value of 2 vs 3.

Now consider that each layer has 10 functions. The entropy level will be increased exponentially within the open layer design.

** Closed Layer **

There are 100 possible connections between the layers since the UI will call 10 functions in Business, who in turn could call 10 functions in Data.

  • UI->Business 10
  • x
  • Business->Data 10

Complexity total = 100

** Open Layer **
Now if we open this layer up, the complexity is compounded.

  • UI->Business 10
  • x
  • Business->Data x 10
  • +
  • UI->Data 10

TOTAL = 110

It is therefore recommended that a closed layer be used and you can see why. An open layer pattern can create a lot of exponential complexity. But what about unit tests? Will they not help deal with the complexity?

A question of support

With the layer pattern, you would expect that our software frameworks and development environments would cater for these rules. You may think that when you add a dependency of UI->Data, your compiler, or IDE would say declaratively, you cannot make a reference to this library. But no, instead the only means of compiler intervention is circular dependencies.

With a simple piece of architectural metadata, we could introduce the ClosedLayer principle as a rule.

What do our IDE, package managers and compilers say
UI->UI – NO
UI->Business->Data->Business – NO

UI->Data YES – *(this should be NO)
UI->Business YES
Business->Data YES

ADaM – Architectural Design and MetaData

What is ADaM?

I propose a language of architectural design and runtime, that implements rules for IDE/compilers which would be written for a project. The project would abide by the rules of the architecture. The rules are declared for ensuring the quality of the architectural offering. ADaM would control the capabilities of the code, the exposure of the code and be both design and runtime capable.

ADaM would enforce rules, whitelists, blacklists, do’s and don’ts for

  • dependencies
    • on a per-dependency basis
    • on a type/module/class/namespace/package basis
  • Permission of streams/function
    • File/ Memory etc
    • Database access
    • Network
  • Permissions language features
    • Reflection
    • Object-oriented features (for example no abstract classes in this layer)
  • Instantiation
    • Limit FactoryMethod A to objects derived from Type X
  • Dependency Injection
    • Control of
  • Call Stack
    • Disallow calls from function a
    • Only allow calls from function a|b|c
  • and more

Consider the following simple example and accompanied ADaM:

class Application
{    
    constructor(){
    }
    getFirstName(){
    }
    setFirstName(name){
    }
    getLastName(){
    }
    setLastName(lastname){
    }
}
var Application = {
    interfaces: {
        internal : 'all',
        external : [ 
            {
                 'Company.XYZ' : {
                    hide : 'set*'
                }
            },
            {
                'certificate.snk' : {
                    hide : 'setFirstName'
                }
            }
        ]
    }
}

Company.XYZ would see

class Application
{    
    constructor(){
    }
    getFirstName(){
    }
    getLastName(){
    }
}

Clients with certificate.snk, would not see setLastName

class Application
{
    constructor(){
    }
    getFirstName(){
    }
    setFirstName(name){
    }
    getLastName(){
    }
}

This means that we can control what this type exposes to different clients. Remember that you can describe interfaces and expose this in a new component, calling the original, or provide a facade layer, or an adaptor pattern, etc, etc. The point of ADaM, is to create a single point of configuration rules, without impact and changing existing code. This will reduce entropy.

Consider the access members such as public the class. I would opt for removing this from the language, and into the ADaM language.

public abstract class Person {
    //we are explicity defining the rule of the class in code, compiled.
    //any changes, would require a recompile.
}

//throws error because Person is marked as abstract by ADaM
var x = new Person();

var Person = {
    declaration : {
       access : public,
       abstract : true
    }
}

Use cases for ADaM

Creating a language for design

Design Patterns provide the added advantage of language. When we speak the same language, complex ideas, can be easily conveyed and digested. ADaM would provide a language for easy digestion of the architecture model of software. A framework can better serve its developers by providing architectural rules and guidance. There are tools and frameworks for evaluating guidelines and rules, but these tend to be design or compile time, not run-time capabilities. Capabilities for design-time, compile-time and run-time would greatly improve the flexibility of the rules. Typically tools that exist do not have instantiation rules or modify the code access at runtime.

ADaM would cover the architectural as well as idiom level of code.

Design for users

The users of your API, Layer or Component or package vary. You could be catering to a team member, another company, or a package in the community. The audience we provide code to determines the nature of what we offer. A component used internally by a few coders on one project, may not require the same level of design that a framework may get if you are building it for the wider community. For example, do you full document an API used by one developer across the hall?

ADaM would allow customization of the code in terms of what is seen by who.

Simplified API for 3rd Party

Consider providing the same codebase API to a client, but provide them with a facade ADaM. ie. The ADaM you provide, informs their IDE, to hide certain classes and interfaces from their intellisense, and provides a simple API. Once they are accomplished, they can remove the training wheels and see more of the interfaces.

Another level of Security

By default, all interfaces could be compiled with private member access, and only a valid compiled ADaM would unlock certain interfaces. This would work hand in hand with obfuscation and code signing.

Staving off unwanted entropy

By designing the dependencies, and controlling what interface is exposed to who, you are preventing entropy by error. For Example, a developer references a component and gets the job done, but only for you to find out later, that it introduced extra dependencies and was not 64-bit compliant.

Conclusion

Software entropy comes from decisions, lack of decisions, lack of care, laziness, and many other aspects of the code. Software must be controlled and entropy must be minimized by everyday design and everyday decisions. The audience of our code is varied, the environments they develop in, the roles that they play vary considerably and yet, we create one code-base. If we cater for all these actors, the complexity increases as we introduce more interfaces and further abstractions to the original code.

Our development environments should be designed to facilitate the reduction of entropy by providing us with the tools we need to enforce architectural and design decisions on the code, before and after. By the mere fact that we would decrease interfaces and abstractions, means we gain more productivity and less entropy as a result.