SAN MATEO (03/27/2000) - Whether they are creating applications for internal or external customers, today's C++ developers are in a tough position. They know they should spread comments liberally throughout their code. They know they should design before coding and document as they go. Programmers know the rules, including the most important one: Create test harnesses that exercise every line of the code you create.
In this time of rushed projects, scarce talent, and high turnover, responsible coding too often takes a backseat to the demand to "Just ship it." It's up to project leaders and managers to see that code is solid before it's delivered.
ParaSoft Corp. has introduced a Windows tool that automates the quality testing of C++ code. C++Test analyzes your project and generates code to exercise the methods it contains. I examined a beta release of C++Test using an AMD Athlon workstation running Windows NT Workstation 4.0. The workstation, running at 1GHz and equipped with 384MB of RAM, already had Visual Studio 6.0 (with Service Pack 3) and Visual C++ loaded. I also tested with Windows 2000 Professional and found C++Test's performance and behavior identical to that observed under Windows NT.
C++Test is a Java application with a native (i.e., non-Web) interface. It installed easily and has modest disk space requirements. Like all Java applications of consequence, C++Test requires a great deal of RAM and a fast CPU. My Athlon workstation was configured to run Visual C++ and C++Test side by side. C++Test alone occupied 24MB of RAM after loading, growing to 31MB during the testing of a simple set of objects. C++Test also ran acceptably on a 500MHz Pentium III system with 256MB of RAM. However, on the slower CPU, C++Test's impact on other applications was more noticeable. C++Test's hardware requirements are not unreasonable, but you'll want to keep them in mind when configuring test workstations.
The beta release of C++Test proved very solid. Installation was quick and automatic. A sample C++ project and online help in HTML format accompany the software. To start a new testing session, you point C++Test at your Visual C++ project file. C++Test loads your source code and displays a tree view of files, objects, and methods. You can click on any item in the tree to expose more detail, including syntax-highlighted source code.
C++Test's interface is functional, but I would have preferred to see it integrated with Visual Studio. When I found an error while browsing the source code in C++Test, I had to swing over to the Visual Studio editor (C++Test's source view is not editable). After modifying my code, I returned to C++Test, synced it with the changes I had made, and resumed testing. The C++Test interface lets you assign a code editor -- it uses Notepad by default -- but most Visual C++ developers edit within the Visual Studio environment. Less significant, yet still bothersome, is the Java run time's lack of support for mouse wheel scrolling.
C++Test does a good job of picking apart C++ classes. Once a project is loaded, your next task is to configure method tests. C++Test examines each method's argument and returns value types. It supplies default arguments for simple data types; you can define test data for user-defined types and classes. The interface lets you override C++Test's default arguments and return value tests.
I appreciated C++Test's capability of testing incomplete code. The tool generates stub code for any method or function that doesn't yet exist. This facility also supports simulation of external devices or user input, both of which interfere with repetitive testing.
Once you've defined test parameters for all of your methods, C++Test is ready to exercise your code. The tool generates test code and invokes the Visual C++ compiler to build it. The C++Test interface launches tests at the method, class, file, or project level. As you work through your code/test cycle, C++Test keeps track of the C++ statements you've tested. When you make a change, the tool's coverage-analysis report identifies it as an area that needs retesting.
Most Windows C++ projects make use of COM (Component Object Model) objects, and it's here that C++Test comes up short. C++Test generates code to test C++ classes and methods, but calls to COM object methods are rendered murky by templates, macros, and external library code. NuMega's TrueCoverage (part of DevPartner Studio for Visual C++) does not generate test harness code, but it appropriately analyzes the coverage of the harnesses your developers create.
None of DevPartner Studio's facilities are confounded by COM.
The less-costly ($799 per seat) DevPartner Studio also includes BoundsChecker, an invaluable utility that watches running C++ code for memory leaks, unreleased resources, invalid argument types, bad pointers, and other common gaffes. C++Test does not ferret out these errors during automated testing.
Parasoft sells Insure++ for that.
Automated C++ object testing could ease some of the burden of harried developers. Unfortunately, C++Test's lack of COM support, absent Visual Studio integration, and high price make it likely that Windows programmers will continue to write their own test harnesses for some time to come.
Tom Yager is a senior analyst for the InfoWorld Test Center. He can be reached at tom_yager@infoworld.com.
THE BOTTOM LINE: BETA
C++Test, beta
Business Case: Rushed projects and staff turnover complicate C++ code testing.
C++Test builds and runs the test harnesses your programmers should create, but probably don't.
Technology Case: C++Test analyzes C++ source code, locates objects, and generates test routines for the objects' methods. It requires another product (Insure++) to watch running code for memory leaks, bad pointers, and array-bounds errors.
Pros:
+ Includes a versatile test editor
+ Generates useful code coverage reportsCons:
- Huge Java footprint
- Lacks COM support
- Lacks integration with the Visual Studio interface- High costCost: $3,500 per seatPlatform(s): Windows 98 and Windows NTShipping: April 10, 2000ParaSoft, Monrovia, Calif.; www.parasoft.com.