By James D. McCabe
After taking the steps of seeding the evaluation process and asking peers for advice about choosing networking vendors and service providers to help with your projects, the third step is gathering data on top prospects.
Now that you have developed a complete (or nearly complete) set of (vendor, vendor equipment, service provider) candidates, you need to gather and develop data that will be helpful to your evaluations. Common sources and types of information are:
- Discussions with internal and external groups
- Discussions with vendors and/or service providers
- Independent (third-party) assessments of vendors, equipment, and/or service providers
- Modeling and simulation
- Information from risk assessments
Discussions with internal and external groups can provide information in addition to the candidate discussions. The intent of these discussions is to expand the amount of information by talking with more users and staff, and in greater detail. External groups may be able to relate their experiences with candidate vendors, vendor equipment, and service providers. This is especially useful when your design includes technologies or equipment that are new to you, but that others have experience with (e.g., implementation and operational experience with IPv6). External groups may be other organizations like yours, or groups with a common interest, such as user groups or technical forums.
Since we now have a list of candidates, discussions with vendors and/or service providers on that list are often useful to learn specific information about each vendor, service provider, or piece of equipment. Such discussions can also be helpful in learning what each is capable of doing for your project: providing value-added services, access to labs, testing and validation, and competitive pricing are some examples. However, vendors and service providers should not be allowed to get too close to the evaluation. They are motivated to represent their products and services in the best light and may try to steer the evaluation in their direction. Therefore, it is best to keep such discussions isolated from the evaluations (i.e., don't invite them to any meetings where the evaluations are being discussed).
Along with the above discussions, at times it is useful to get independent (thirdparty) assessments of vendors, equipment, and/or service providers. Independent assessments provide a prospective that is often different from what internal groups, vendors, and service providers will give. Like external groups, independent assessors may be able to provide you with implementation and operational experience with technologies and equipment that are new to you. For independent assessments I tend to choose small companies or individuals (consultants) instead of large consulting firms.
Modeling and simulation of all or part of the network architecture can provide valuable information for your evaluations. For example, you can use computer models to make comparisons of how your network will perform using different service providers, based on their services offered and on their infrastructures. You may already have some of this information if you have done modeling and simulation to refine your network analysis and/or architecture.
In addition to the above, any information from risk assessments that is applicable to vendors and/or service providers can be useful in evaluations. Your risk assessment, performed early in the analysis process, may reveal certain risks for your project. One or more of these risk factors may be applicable to your evaluation. For example, your project may be adverse to the risk of applying a new protocol, technology, or service. If vendors or service providers apply one of these, it can be used as a part of your evaluation.
All of this information is usually compiled as a document in support of the evaluations and is carried forward into criteria refinement and ratings development. In addition, this information can be very helpful in developing formal documents needed in the procurement and implementation of your design. One example of this is development of a request for purchase (RFP) for products and services.
For example, using the set of initial evaluation criteria from Figure 10.7, data gathering would probably allow us to expand that set based on input from various organizations. Vendors would probably be able to add technology and standards-specific information, while we may be able to learn about risks from other organizations that have already completed similar projects. We may be able to learn specifics regarding performance, from these same organizations or from groups that do equipment and system testing.
Evaluating vendors and service providers for networking projects
How to choose vendors, tools and service providers
Seeding the evaluation process
Having conversations about prospective networking partners
Gathering data on prospective networking partners
Refining your criteria for prospective networking partners
Developing ratings for prospective networking partners
Modifying the list of prospective networking partners
Determining the order of evaluations for networking partners
Reproduced from Chapter ten of the book Network Analysis, Architecture, and Design by James D. McCabe. Copyright 2007, Morgan Kaufman Publishers, an imprint of Elsevier Science. Reproduced by permission of Elsevier, 30 Corporate Drive, Burlington, MA. Written permission from Elsevier is required for all other uses.