Book excerpt

Book chapter: Software test methods for the CISSP exam

SearchSecurityChannel.com Staff

    Requires Free Membership to View

In CISSP Study Guide, authors Eric Conrad, Seth Misenar, and Joshua Feldman discuss application development security, including software test methods such as fuzzing and combinatorial software testing.

The following excerpt comes from Chapter 9: Domain 8: Application development security (pdf).

Software Testing Methods

There are a variety of software testing methods. In addition to testing the features and stability of the software, testing increasingly focuses on discovering specific programmer errors that could lead to vulnerabilities which risk system compromise, including a lack-of-bounds checking.

Static testing tests the code passively: the code is not running. This includes walkthroughs, syntax checking, and code reviews. Dynamic testing tests the code while executive it.

White box software testing gives the tester access to program source code, data structures, variables, etc. Black box testing gives the tester no internal details: the software is treated as a black box that receives inputs.

A Traceability Matrix (sometimes called a Requirements Traceability Matrix or RTM) can be used to map customers’ requirements to the software testing plan: it “traces” the “requirements”, and ensures that they are being sent.

Software Testing Levels
It is usually helpful to approach the challenge of testing software from multiple angles, addressing various testing levels, from low to high. The software testing levels of Unit Testing, Installation Testing, Integration Testing, Regressing Testing, and Acceptance Testing are designed to accomplish that goal:

  • Unit Testing: Low-level tests of software components, such as functions, procedures or objects
  • Installation Testing: Testing software as it is installed and first opened
  • Integration Testing: Testing multiple software components as they are combined into a working system. Subsets may be tested, or Big Bang integration testing tests all integrated software components
  • Regression Testing: Testing software after updates, modifications, or patches
  • Acceptance Testing: Testing to ensure the software meets the customers’ operational requirements. When this testing is done directly by the customer, it is called User Acceptance Testing.

Fuzzing
Fuzzing (also called fuzz testing) is a type of black box testing that enters random, malformed data as inputs into software programs to determine if they will crash. A program that crashes when receiving malformed or unexpected input is likely to suffer from a boundary checking issue, and may be vulnerable to a buffer overflow attack.

Fuzzing is typically automated, repeatedly presenting random input strings as command line switches, environment variables, and program inputs. Any program that crashes or hangs has failed the fuzz test.

Combinatorial Software Testing
Combinatorial software testing is a black box method that seeks to identify and test all unique combinations of software inputs. An example of combinatorial software testing is pairwise testing (also called all pairs testing).

NIST gives the following example of pairwise testing. “Suppose we want to demonstrate that a new software application works correctly on PCs that use the Windows or Linux operating system, Intel or AMD processors, and the IPv4 or IPv6 protocols. This is a total of 2 x 2 x 2 = 8 possibilities, but as the table below shows, only four tests are required to test every component interacting with every other component at least once. In this most basic combinatorial method, known as pairwise testing, at least one of the four tests covers all possible pairs (t=2) of values among the three parameters.”

NIST Pairwise Testing Example

Test case

OS

CPU

Protocol

1

Windows

Intel

IPv4

2

Windows

AMD

IPv6

3

Linux

Intel

IPv6

4

Linus

AMD

IPv4

Download the entire chapter (pdf).

Reprinted with permission from Elsevier Inc. Copyright 2011. "CISSP Study Guide" by E. Conrad, S. Misenar and J. Feldman. For more information about this title and similar books, please visit the book’s page on the publisher's web site.

This was first published in September 2011

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.