Design+MethodologyIdeas

The Materialization of Abstractions

The Materialization of abstractions

Almost no one would disagree that software lies within abstractions and that the future lies within more abstractions. If you disagree, it may be because you find abstractions annoying and difficult to handle. However, we require abstractions to further the boundaries of reuse and improve the abilities of frameworks.

Unfortunately its one of the most neglected areas in all software platforms and development tools. Here I discuss what abstractions are, why they are good and why we should be building more abstractions, as long as they come with more tools.

We are in 2017 and coding is largely no different to how it was in the 60’s and 70’s. We all write loops and conditional statements and package it up in different coding paradigms.

We are running command-line arguments for compilers, using switches and options to guide it. We still update versions of dependencies. We are still searching through large text files for functions, debugging an execution stack for a glimpse of a bug. We are pouring through documentation, completing tutorials that don’t work because they are out of date and spending loads of time, just trying to grasp an architectural pattern from a new framework that is far too steep.

When our industry creates abstractions, it’s up to the developers to learn them, and usually its purely with documentation. The age of documentation should have past, but it’s still here. We need to break abstractions into fine manageable pieces, scrap documentation or at least minimalize.

Let’s talk abstractions.

Why abstractions at all?

Abstractions are fundamental to computer science, we are always sitting on top of a layer that attempts to encapsulate the chaos below. We do this because we want a simpler model above.

A simplistic model of computing abstraction:

  • Electronics
  • Binary
  • Assembly languages
  • Compilers
  • Interpreters

Programming Languages are given a tag of ‘generation’ describing the abstraction level at which they sit.

1st Generation language is considered binary, 2nd Generation Hexidecimal (opcodes), with the 3rd generation being compiled and assembled code such as C++, and 4th Generation languages being ‘programmer-less’ graphically ‘configured’ languages.

Benefits of abstractions

There are clear-cut benefits. Thankfully, coders do not code websites in binary. Think about how long it would take for us to develop a website in this fashion.

Encapsulation

When using a tv remote, you don’t need to know the electronics inside, we call this encapsulation. Hiding complexities and providing a simpler interface for the layer above provides an improved environment.

Better Reuse

If a layer consistently reuses a layer below, it benefits from the layer’s existence. The layer below is abstracting complexities and doing more. You get more bang for your buck.

Renewed Focus

Since the layer below is working on the complexities of network handshaking or similar complicated problem, you can focus on your layer. Your layer may be concerned with HTTP, not TCP/IP which sits underneath.

Faster Development time

Even if it’s not faster, you are more focused on your tasks, thus you get more done for your requirements. The developer of the CSS parser in your browser is enabling CSS coders to build CSS frameworks such as Bootstrap, which in turn allows the WordPress jockey the ability to install a theme that works to an existing look and feel.

Abstractions in software development environments

The software industry is awash with abstractions. It has become incredibly popular in software design too. In fact, some might say, certain portions of the developer community revel in the complexity of abstractions.

Abstract vs Concrete (not the Object-Oriented terms)

Without abstractions, we only have material manifestations and concrete implementations. A system can be built in that way, but at the loss of a major tool in the armoury of computer science.

The broader concepts of abstraction include:

  • Hardcoded values
  • Convention over configuration
  • Decoupling
  • Software Layers
  • Assorted Design Patterns
  • Data Object Mappers
  • Dependency Injection
  • Framework Hotspots
  • Interface Definition Languages
  • Component Frameworks
  • Domain Frameworks
  • Domain Specific Languages

Let’s consider the following convention in coding: convention over configuration or coding by convention

A piece of code, is hard-coded to look at a folder named 'myfolder'

This is simplistic.

  • It won’t need to check a configuration file to obtain the name
  • The code could rely on the folder being there, with little checks

Negatives include

  • If it’s not documented, you won’t know it needs this folder
  • It could throw errors if the folder does not exist
  • You cannot configure the path

In order to create a concrete implementation, the software framework requires you to implement IDoThis and register it, with Manager.Register(IDoThis)

Positives include

  • The framework is providing a hotspot for extension
  • The framework has provided an Interface
  • The framework has provided a means for you to register your Interface
  • The interface IDoThis is called when the responsibility and extension is required by the framework
  • You have customised the framework

Some Negatives

  • You need to manually read/learn about the hotspot
  • You need to find the interfaces
  • You need to find the hook in the framework
  • You need to understand when and how the framework calls you, or at least basics of how it works
  • You can only do what the framework has provided and without learning the entire framework, you won’t know what it can do
  • A framework with over 100 hotspots and interfaces becomes complex

The problem with abstractions in software

The above has pointed out some issues to consider, but let me mind dump some further issues.

  • We rely on documentation
    • and it’s usually out-of-date
  • We rely on StackOverflow and forums
  • Documenting software is hard
    • Generated documentation is often worthless
  • Abstractions come with versions and complex dependencies
  • Black-box implementations are ‘magic’
    • difficult to understand
    • difficult to master
  • White-box implementations require firm understanding
    • easy to get wrong
  • Designing a framework that is simple to use is complicated
  • Designing your components inside a framework without complete knowledge is difficult
  • Inadvertently creating bugs by lack of understanding
  • Incomplete frameworks released to market
  • Developer contributions muddy the framework design
  • Badly designed abstractions lead to convolution
  • Your framework is limited by the framework in which it inhabits
  • Object-oriented abstractions rely on interfaces which are designed by contract
  • Strongly Typed system require recompiling
  • Dynamically Typed systems rely on diligent coders
  • Leaning Tower of Pisa syndrome, when layers underneath are not solid to support the implementations above
  • Debugging is really difficult
  • Error finding is difficult

The solution

The solution is a multi-faceted attack on the complexity that abstractions bring. Your IDE needs a way to show you where the abstraction is and how to use it.

The current solution in the real world is this:

abstraction -> documentation -> developer

Let’s work towards this instead:

abstraction -> metadata -> intellisense / hotspot viewer / rules -> documentation -> you

1. ADaM

Abstraction is code and architecture. For my recommendation for architecture can be read in another article, Proposal for Compilers and Runtimes to combat Software Entropy

2. Software Framework MetaData

All abstractions need inline metadata that can be used by reflection. Introducing Framework Descriptive MetaData Language (FDML)

Consider the following class code.


internal class ViewLoader { public ViewLoader(string view) { //load config <views folder="myfolder"> //get folder value //load file from views folder //load contents of file //validate contents to valid HTML //ensure HTML is surrounded by <div data-view=""></div> } }

If you are using this framework, you won’t be able to see the ViewLoader class. You won’t be able to ascertain what the folder in the configuration file is. You won’t know that the view should have a <div> around it. Consider the following code


[Config(&#039;views@folder&#039;, Optional, Default: &quot;myfolder&quot;)] [Assert(File:Content:&quot;&lt;div data-view&gt;$content&lt;/div&gt;&quot;)] [Assert(File:Extention:&quot;html|htm&quot;)] [Assert(File:Permission:&quot;Read&quot;)] internal class ViewLoader{ public ViewLoader(string view){ //load config &lt;views folder=&quot;myfolder&quot;&gt; //get folder value //load file from views folder //load contents of file //validate contents to valid HTML //ensure HTML is surrounded by &lt;div data-view=&quot;&quot;&gt;&lt;/div&gt; } }

Standard code

public abstract class Human : IHuman
{
}

public abstract class Person : Human
{
}

public static class Factory
{
    public static Create(IHuman human){
    }
}

With FDML

public abstract class Human : IHuman
{
}

[Factory(PersonFactory)]
[Configuration()]
public abstract class Person : Human
{
}

internal static class PersonFactory
{
    [Config(&#039;providers\people\person@type&#039;)]
    public static Create(){
    }

    [Creates(IHuman,Human)]
    public static Create(IHuman human){
    }
}

Imagine your IDE understands FDML

If your development environment understands FDML, it can show you what you require. Consider a menu item named ‘inspect usage’ and it’s now on your favourite IDE menu. You include a component reference, and you run ‘inspect usage’. A new window opens and declares a list of things it requires.

Output window


Warning: Assert requires Admin Permissions on interface X TODO: Config is required for Y TODO: IHuman can be implemented and provided to PersonFactory

If you run the ‘inspect usage’ on the Person object, you will get PersonFactory as the creator function. With a carefully constructed FDML syntax and intelligent viewers, we can reduce documentation requirements, decrease and speed up the initial learning curve.

Without FDML, you could ascertain quite a lot from the codebase, but a language like this will provide design control.

3. A Framework for Frameworks (Framework Abstraction Layer)

What is a framework? There are quite a lot of definitions out there but most agree with the following.

  • It has abstract implementations designed for concrete ones
  • Features such as hotspots are required for extending the framework
  • It is focused on a domain
  • It tends to follow a ‘don’t call us, we will call you’ philosophy
  • It is foundational and is a bottom of the pyramid layer

I requested a quick definition of 140 chars or less on twitter from a [post I wrote in February 2004]

“An independent system of cohesive software that performs a general function that controls user specified functions.” – @unclebobmartin

“dumb soundbite: a framework is a program you don’t mind finishing.” – @KentBeck

“Code that attempts to solve a problem that hasn’t been shown to exist yet” – @benfulton

“Framework is designed to support dynamic websites, apps, and services, while alleviating overhead assoc with common activities” – @JRewald2000

“A framework is an API that sits above a language in order to aid developers usually in a specific area of expertise, e.g. web” – @arumbrindy

“Not a solution. But the concept how to create one.” – @BanDoga

“Framework is a painkiller in the way it lets you focus on what really matters. Good for production, not learning purposes.” – @josemotanet

My own definition is

CoalescentPatterns and AggregatedPatterns which exposes mutable and extensible areas, for specialization and change. - @jcrossland

I believed back in 2004 and earlier, and believe it more than ever today. A framework layer is required on which all other frameworks are built. It would be the COM or CORBA of a framework. The cross-platform framework, would provide concrete implementations with a standardized set of interfaces.

4. Software Abstraction Layer

Connecting the Framework Abstraction Layer to existing platforms would be a common interface for libraries. The SAL would provide a common API against which all platforms would be invited to adopt.

5. Common Business Layer

A set of business entities and data types, provided in an open source discovery platform. Tied into the SAL, developers can use common language and interfaces.

I will post a new article dedicated to this topic.

6. The Hive

Our use of files sent to a compiler will appear archaic after new paradigms come into play. The Hive is a potential replacement and will come in another article.

Conclusion

My conclusion is simple. We have to discover better paradigms and build better tools that deal with abstractions. Many more abstractions will be required in software development as the years go by and we have to plan for the next 30 years.