Abstraction of concurrency from software application logic

(a.k.a: “I feel the need… the need for - Concurrency…”)

From Monolith to Component Architectures

In the olden days of computing, everything was combined into a single lump of software - including operating system functions, application logic, data, user presentation. After some time, we realized that creating a separation between the hardware and the software application would be useful, and the operating system was born. Some time later, we realized that managing the data was a distinct task, and databases became a separate entity. Some time after that, we decided that it would be a good idea to split out the user interface, and the era of client/server began.

So it went, the software industry continued to evolve our architectures into more componentized and modular arrangements.

Architecting for Concurrent Computing

Now that multi-core architectures are common and the need for concurrency in software architectures well understood, the next question is how best to architect our applications and how best to structure development organizations to support them. For the moment, let’s talk about the development organization. In the past, concurrency was a task limited to edge cases, but now ubiquitous multi-core hardware is making it much more common.

There are two ways to handle this. First, train all your software developers to be experts in concurrency (yikes…). Second, have concurrency specialists to focus on making your applications parallel. Since training all of your software developers to be experts in concurrency seems daunting, it would seem that the second option is better. But if concurrency is now required through significant portion of your application architecture, how can only a few engineers or architects be responsible for it?

The answer lies in abstracting the concurrency model from the application logic. While this may not be possible for all aspects of your concurrency model, it certainly should be for some - especially for well-defined services (in the SOA definition of the word). Services can be run in multiple instances across multiple threads and multiple cores (even multiple servers) to achieve a significant degree of concurrency without rewriting the code inside the service.

To the extent that you can do this, you can allow your application developers to focus on application logic and your concurrency experts to focus on concurrency. In addition, you can gain quite a bit more flexibility to your application architecture. The more your concurrency is abstracted, the easier it is to change without affecting the application logic. It’s really just an extension of the idea of loose coupling.

The Concurrency Model

This means that the application developer does not have to be the primary owner of the concurrency model. The application developer is able to focus on the application, and a concurrency expert can design about the concurrency model, just as a DBA does with the data model - I wouldn’t be surprised if we start to see something like a Concurrency Model Architect (CMA?).

Anyway, the whole thing relies on being able to separate your concurrency model from your application logic. More on that later.

Patrick

John Markoff of The New York Times published an interesting article on today entitled  “Faster Chips Are Leaving Programmers in Their Dust“.

It’s great to see that this challenge, which I’ve been calling the Multi-core Dilemma is now moving into the mainstream discussion. Of course, I was far from the first to sound the alarm here, but it’s good to see the broader discussion now taking place.  We need to be talking about these issues and delivering practical solutions.  For everyone who writes software, this is a significant challenge but also a significant opportunity.  The approach to addressing this is “concurrent computing” or simply, “Concurrency”.  

Concurrency is a discipline that will become increasingly important in software development.  In fact, expect it to be a skill that employers look for on technical resumes in the not-too-distant future. Of course, Concurrency is not new, it’s almost as old as computing itself.  But as Herb Sutter pointed out in this article in Dr. Dobbs; “The Free Lunch Is Over“, it was only required for a relatively small percentage of applications until recently.  With multi-core and many-core hardware now a reality, Concurrency is already required for a significant percentage of applications and will soon be the norm.  

As cited in the Times article, several companies, including Microsoft, are working on new programming languages that are designed from the ground-up for Concurrency.  That is undoubtedly the right direction, but these efforts are years from being mainstream technology.  When they are ready, they are likely to be applicable mostly to new development - new code will be concurrent from the start.  But where does this leave the billions (yes, billions with a “B”) of lines of code that businesses already have running in their data centers?

There are a variety of threading packages available, including Rogue Wave’s SourcePro Threads for building concurrent applications. These tools make the process easier by handling much of the threading work so that software developers don’t have to. But even with these it is complex and there is still a lot of design and testing work to be done. Testing multi-threaded applications is tricky. Even with thorough testing, problems often don’t arise until the application is in production.There are many factors to consider in the Concurrency discussion, but here is one aspect to consider. With the widening adoption of Service Oriented Architectures (SOA), we need to address Concurrency at the inter-service level as well as intra-service. Threading libraries and packages are well established approaches for intra-service, but there aren’t a lot of good solutions for inter-service Concurrency other than things like clusters - but that is a blunt instrument for this task.

Concurrency at the service level (inter-service) can have several benefits.First, this approach has the affect of abstracting the Concurrency model away from the application logic. This “loose coupling of concurrency” makes it easier to change your concurrency model - such as when you move from a 4-core to a 16-core server - without touching your application code. Second, it is easier to apply to existing applications since it treats the individual service largely as a black box. Third, it can provide the scale-out that traditionally came from clustering but with much more intelligence, since the application logic is part of the mix.

In summary, there are several important needs here that have not been adequately addressed:
1. Solutions that apply Concurrency to existing applications without significant rewriting
2. Inter-service Concurrency to abstract the Concurrency model from the application code

More on this later…