Abstraction of concurrency from software application logic

(a.k.a: “I feel the need… the need for - Concurrency…”)

From Monolith to Component Architectures

In the olden days of computing, everything was combined into a single lump of software - including operating system functions, application logic, data, user presentation. After some time, we realized that creating a separation between the hardware and the software application would be useful, and the operating system was born. Some time later, we realized that managing the data was a distinct task, and databases became a separate entity. Some time after that, we decided that it would be a good idea to split out the user interface, and the era of client/server began.

So it went, the software industry continued to evolve our architectures into more componentized and modular arrangements.

Architecting for Concurrent Computing

Now that multi-core architectures are common and the need for concurrency in software architectures well understood, the next question is how best to architect our applications and how best to structure development organizations to support them. For the moment, let’s talk about the development organization. In the past, concurrency was a task limited to edge cases, but now ubiquitous multi-core hardware is making it much more common.

There are two ways to handle this. First, train all your software developers to be experts in concurrency (yikes…). Second, have concurrency specialists to focus on making your applications parallel. Since training all of your software developers to be experts in concurrency seems daunting, it would seem that the second option is better. But if concurrency is now required through significant portion of your application architecture, how can only a few engineers or architects be responsible for it?

The answer lies in abstracting the concurrency model from the application logic. While this may not be possible for all aspects of your concurrency model, it certainly should be for some - especially for well-defined services (in the SOA definition of the word). Services can be run in multiple instances across multiple threads and multiple cores (even multiple servers) to achieve a significant degree of concurrency without rewriting the code inside the service.

To the extent that you can do this, you can allow your application developers to focus on application logic and your concurrency experts to focus on concurrency. In addition, you can gain quite a bit more flexibility to your application architecture. The more your concurrency is abstracted, the easier it is to change without affecting the application logic. It’s really just an extension of the idea of loose coupling.

The Concurrency Model

This means that the application developer does not have to be the primary owner of the concurrency model. The application developer is able to focus on the application, and a concurrency expert can design about the concurrency model, just as a DBA does with the data model - I wouldn’t be surprised if we start to see something like a Concurrency Model Architect (CMA?).

Anyway, the whole thing relies on being able to separate your concurrency model from your application logic. More on that later.

I’m very pleased to announce the General Availability of SourcePro Edition 10, a very special event for Rogue Wave Software.

SourcePro is our flagship product and the main reason why we can claim to be the market leader in enterprise C++. We are launching the tenth version of SourcePro - a significant milestone that very few application development products reach. It’s a testament to the quality of the product and a demonstration of Rogue Wave’s long term commitment to the product, especially given the recent market trends moving in favor of C++  - most notably, new language standards coming in C++0X and giving C++ the same status as Java in new SOA industry standards such as SCA and SDO.

The full GA version of the product is available for customers now and the evaluation version will be available on our Web site in February.

John Markoff of The New York Times published an interesting article on today entitled  “Faster Chips Are Leaving Programmers in Their Dust“.

It’s great to see that this challenge, which I’ve been calling the Multi-core Dilemma is now moving into the mainstream discussion. Of course, I was far from the first to sound the alarm here, but it’s good to see the broader discussion now taking place.  We need to be talking about these issues and delivering practical solutions.  For everyone who writes software, this is a significant challenge but also a significant opportunity.  The approach to addressing this is “concurrent computing” or simply, “Concurrency”.  

Concurrency is a discipline that will become increasingly important in software development.  In fact, expect it to be a skill that employers look for on technical resumes in the not-too-distant future. Of course, Concurrency is not new, it’s almost as old as computing itself.  But as Herb Sutter pointed out in this article in Dr. Dobbs; “The Free Lunch Is Over“, it was only required for a relatively small percentage of applications until recently.  With multi-core and many-core hardware now a reality, Concurrency is already required for a significant percentage of applications and will soon be the norm.  

As cited in the Times article, several companies, including Microsoft, are working on new programming languages that are designed from the ground-up for Concurrency.  That is undoubtedly the right direction, but these efforts are years from being mainstream technology.  When they are ready, they are likely to be applicable mostly to new development - new code will be concurrent from the start.  But where does this leave the billions (yes, billions with a “B”) of lines of code that businesses already have running in their data centers?

There are a variety of threading packages available, including Rogue Wave’s SourcePro Threads for building concurrent applications. These tools make the process easier by handling much of the threading work so that software developers don’t have to. But even with these it is complex and there is still a lot of design and testing work to be done. Testing multi-threaded applications is tricky. Even with thorough testing, problems often don’t arise until the application is in production.There are many factors to consider in the Concurrency discussion, but here is one aspect to consider. With the widening adoption of Service Oriented Architectures (SOA), we need to address Concurrency at the inter-service level as well as intra-service. Threading libraries and packages are well established approaches for intra-service, but there aren’t a lot of good solutions for inter-service Concurrency other than things like clusters - but that is a blunt instrument for this task.

Concurrency at the service level (inter-service) can have several benefits.First, this approach has the affect of abstracting the Concurrency model away from the application logic. This “loose coupling of concurrency” makes it easier to change your concurrency model - such as when you move from a 4-core to a 16-core server - without touching your application code. Second, it is easier to apply to existing applications since it treats the individual service largely as a black box. Third, it can provide the scale-out that traditionally came from clustering but with much more intelligence, since the application logic is part of the mix.

In summary, there are several important needs here that have not been adequately addressed:
1. Solutions that apply Concurrency to existing applications without significant rewriting
2. Inter-service Concurrency to abstract the Concurrency model from the application code

More on this later…

The new SourcePro release scheduled for launch next week includes the Apache C++ Standard Library (stdcxx), which has recently been promoted to Top Level Project status by the Apache Software Foundation after over two years in incubation (you can read the project history here). Rogue Wave Software is the primary sponsor of this project, which is based on our own standard library. There are many companies with products that have been opened, but few ever achieve the status of being an Apache project. It takes a lot of work to get a project out of incubation and to the status of being a full Apache project.

It’s a big achievement and the primary driver from Rogue Wave Software has been Martin Sebor - congratulations, Martin!

Welcome to the Rogue Wave Software blog. We’d like to take the opportunity to let you know how it all started.

Rogue Wave Software was founded in the late 1980s by Dr. Tom Keffer, a faculty member of Woods Hole Oceanographic Institute. He named the company after his sailboat.

The first commercial Rogue Wave product was based on a University of Washington project, the Data Analysis and Interactive Modeling System (DAIMS), which was the first attempt at solving large numerical problems in C++. It was launched under the name Math.h++, which can now be found, still going strong, as part of SourcePro Analysis.

The product was quickly expanded into broader class libraries with the launch of Tools.h++ in 1990 as the answer for a general data structure library for C++. Tools.h++ is also still in widespread use as part of SourcePro.

It was Dr. Keffer’s systematic approach as a scientist that resulted in the now legandary reliability and performance of these first two products. This focus provided the basis for everything that has come later. They effectively decided the product direction and roadmap for over 15 years: high performance enterprise application development in C++.

So as C++ applications continue to evolve over time to be part of Service Oriented Architectures, we will be there supporting our customers by providing the reliability and performance they have come to expect.