How to integrate memory leak detection into continuous integration
Continuous Integration (CI), the process of frequently integrating changes from development into the existing code repository and automatically triggering a build and test of the changes, has enabled development teams to avoid major integration and merge headaches and resulted in the routine production of quality product builds. Coupling Continuous Delivery (CD) along with CI, also referred to as DevOps, provides repeatable, rapid processes to deploy the built product to users, shortening the amount of time to complete a quality product release cycle.
The CI process automates testing of the software, including unit, functional, and sometimes performance tests. One testing aspect that has been absent from the CI process is validating a C or C++ application's use of memory and looking for memory leaks and illegal uses of memory, such as buffer overwrites. Memory leaks can be catastrophic to an application and, unfortunately, checking for leaks is often done too late in the development cycle or worse, the problem is found by the customer. Let’s take a closer look at the benefits of adding leak detection and memory usage analysis to the CI process.
CodeDynamics memory debugging technology
The CodeDynamics memory debugging solution provides C and C++ leak detection, memory write overrun detection, and a variety of other memory validation checks. Developers typically use the CodeDynamics user interface to find memory problems in their program but there is a script-driven interface available that is useful for automated environments, such as within CI deployments. Our underlying memory-tracking technology does not require any recompilation or special build of the target application. Instead, it dynamically proxies between your program and the underlying system memory function calls to track all memory-related activity. This solution allows for the checking of memory on target applications “on the fly” with minimal runtime overhead.
CodeDynamics’ script interface enables applications to run within an automated test environment. The script generates output logs and memory recording files with leak detection results and notifications of any illegal uses of memory. If memory leaks or memory use errors are detected, developers can load the generated memory recording files into the CodeDynamics user interface and analyze the programs use of memory! The script-based memory debugging functionality provides the basis for integration into your CI system.
Automated leak detection and continuous integration
Continuous Integration systems enable constant feedback of the impact of changes on a codebase. By monitoring changes to source control systems, such as Git, CI systems automatically detect changes to source code, check out the latest changes, build the product, and then kick off automated tests. All failed results are reported and emailed to the team.
Advantages of integrating leak detection into your CI system
Leveraging the automated capabilities of a CI system brings the same advantages for conducting routine memory use analysis as it does for regular functional testing:
• New code changes are immediately tested, making them easier to fix
• Any new illegal uses of memory and memory leaks are detected and reported through the CI system
• The CI system’s test reports provide transparency to a problem that is often undetected until its too late
• Generation of CodeDynamics memory result files allows for follow-on detailed analysis of the application's use of memory
Let’s take a closer look at how to integrate our leak detection and memory debugging capabilities within a typical CI environment.
Setting up CodeDynamics leak detection
From a high level, any changes to a codebase monitored by a CI system, such as Jenkins, TeamCity, or GitLab CI, results in the execution of the following common steps:
1. Build the application
Changes to the codebase are gathered and the CI system kicks off a job to build the application. Configuration of the build job depends on the type of build system you are using. CI systems provide formal support for systems such as Ant, CMake, and others but they also easily accommodate custom Make system builds.
If the application is built successfully, another job is triggered to regression test the application. If a build failure occurs, the CI system records information about the failure in the job build report and notify the team. Testing of the application will not be done if a build failure occurs.
2. Regression test the application
With a successful build of the application, the CI system starts a job to test the functionality by running unit, integration, and system-level tests. CI systems provide various levels of support for formal testing frameworks but also provide hooks for gathering test results from custom test harnesses too. Test results are gathered by the CI system and presented as part of the results of the test job and reported to the team.
3. Memory test the application
Using CodeDynamics to test the memory usage of an application under the CI system is not much different than regression testing the application. Most times, looking for memory issues simply involves running the test suite again but, this time, running each application test under the control of the CodeDynamics memory debugging technology. Depending on the test suite, this might be as simple as adding an argument.
After each memory usage test, CodeDynamics generates a test report log and, if instructed, memory result files.
The next section discusses a high-level approach to incorporating the memory test results into the CI system's reporting mechanism.
Leak detection report
For each test job that is run, the CI system gathers the test results and presents them as part of a report for the team. When CodeDynamics memory tests are run, the results are deposited in a log file. Using facilities provided by the CI system, the log file can be picked up as part of a post-build action and sent to your team members. Application run logs also provide details on the location of the log files and memory result files so that they can be examined.
CI systems provide many benefits for simplifying the build and test process, shortening development cycles, and improving code quality. Integrating CodeDynamics leak detection and memory usage capabilities into your CI process helps your team find and eliminate memory problems as a routine part of the CI process, giving your teams the transparency they need to find and resolve memory issues before they get noticed by your customers.
Next time, we'll discuss how to integrate leak detection into Jenkins.