Quality Assurance

Vanguard Solutions, being a newly formed organization, has yet to stand up an independent Quality Assurance Department. However, this does not mean that quality assurance is not critical to Vanguard Solutions success.  Vanguard Solutions treats quality assurance as it does Risk Management, in that it is everyone's responsibility for ensuring that quality assurance is adhered to throughout the life of the project.  Quality Assurance begins with the Quality Management Plan, which stems from quality planning.

The Quality Management Plan encompasses the quality assurance and quality control of the project.  The quality management plan addresses both the management of the project and the product outcome of the project.  Vanguard Solutions Quality Management Plan stresses constant improvement to processes, and recognizes the importance of Client/Customer satisfaction. In short, Quality Assurance drives Vanguard Solutions, leads to our success, and enhances our pride as a customer service provider.

Quality Assurance Processes

One method that Vanguard utilizes is Quality Assurance through requirements management.  Regardless of the program's size and complexity, requirements management is critical to a program's success and user/customer acceptance of the results. Establishing realistic achievable requirements is the first step in this process, and this is generally accomplished by documenting the user's requirements in an Operational Requirements Document (ORD), Statement of Objectives (SOO) and/or a Concept of Operations (CONOPs).

The earlier the program "team" (Government and the contractor) can become involved in establishing achievable requirements, the greater are the chances for success.  These requirements are generally stated at a relatively "high level" and reflect the user's operational needs for a system to perform real-world tasks.  Once established in an ORD-like document, these requirements are the primary means by which the end product is evaluated.  From the ORD, generally a specification is developed, such as a System/Subsystem Specification (SSS).  Other system design documentation may contain "detailed requirements" (lower level requirements that support and are scoped by SSS requirements; sub-components or functions of SSS requirements that provide unique or specific development items to focus the understanding of program needs), "design requirements" (second or third indenture requirements, in the form of Software Requirement Specification (SRS) documentation, which provides further delineation and/or refinement of requirements – these must be traceable to the SSS requirements) and/or "derived requirements" (those lower level requirements deemed necessary to achieve the higher level system requirements; a requirement created to support, but within the scope of, one or more system and/or detailed requirements).

Also derived from the basic ORD requirements may be several other important elements that facilitate system design, development and evaluation/assessment, such as Key Performance Parameters (KPPs) and Critical Technical Parameters (CTPs).  Critical Technical Parameters (CTP) are derived from the ORD, critical system characteristics and technical performance measures as per DoD 5000.2-R Mandatory Procedures for Major Defense Acquisition Programs and Major Automated Information Systems.  The CTP list includes major requirements that are most important to the system or those that have high risk. Measures of Effectiveness (MOEs) and Measures of Performance (MOPs) are "derived" directly from requirements so that system tests and operational assessments can be properly performed.

Tracking and Metrics

In order to manage and track the system and detailed requirements and associated data on each program, Vanguard Solutions establishes and maintains a master Requirements Traceability Matrix (RTM). The master RTM contains the requirements text and traceability to the source documents.  The set of source documents may include selected Technical Requirements Documents (TRDs), Functional Requirements Documents (FRDs), ORDs, System Specifications or "A" Specs requirements and white papers from the user/customer.  The requirements identified from the original source documents are analyzed and separated into two groups: system requirements and detailed requirements.  System requirements are consolidated and incorporated into the SSS, organized by an attribute structure such as those developed for the Joint Universal Capabilities List (JUCL) and have traceability to the system level requirements.  The master RTM may further contain the allocation of these requirements to "builds" and/or versions, to hardware and software configuration items (CIs), to domains (common components, models, etc.), and to the IPTs responsible for detailed design and development.  Vanguard Solutions also uses RTMs verification activities as the pieces of the system are systematically integrated and tested.  A Requirements Verification Traceability Matrix (RVTM) can provide a view of the requirements verification activities maintained in the master RTM.  The RVTM can be published as an appendix to the System Test Document (STD) and System Test Report (STR) for each build or version of the system.  To support verification activities, the RTMs developed will contain a minimum of:

• traceability from software requirements to the test cases that support their verification,
• date the test case was executed, and
• status of the verification activity – Passed, Failed, or Conditional.  A Conditional status indicates the unit passed, but stubs or drivers were utilized during the verification test.

Vanguard Solutions' processes and procedures require the RVTM data to be maintained by the organization responsible for performing the verification.  If a System Integration and Test IPT is established, they will also maintain data for SSS requirements verification, as well as verification data for the detailed requirements and the derived requirements.  Vanguard Solutions' typical requirements management process follows a document orientation – linking from the specifications (such as system specs or "A" Specs) to downstream design specifications.  The traceability is captured by referencing between document paragraphs with embedded tags, and summary trace tables at the back of the document.  A number of COTS tools have automated this process by allowing you to tag text, and store the links between paragraphs in a database.  Other tools also support this approach to traceability, but support more advanced ways of dealing with requirements (i.e., requirements as objects).  These tools allow capture and management of requirements the traditional way, but also allow growth as new and improved management practices are brought to bare, and when requirements links into the system are required.  This is known as "integrated requirement management", defining what the requirements mean to the design.  This approach to managing requirements has a number of benefits, shown below.

• Requirements can be managed and attributed – which allows for requirements to be assigned priority, responsible parties, verification schemes, etc., not just text.
• Requirements may be grouped in interesting ways.  For example, these are all the requirements having to do with safety. In addition, inheritance flows through the requirement links.  This provides a means of flowing information about "parents" down to the "children".
• Behaviors can be associated with the requirement objects, allowing the enforcement of processes such as a change process for requirements.

Vanguard Solutions, using automated requirements management tools such as IcConcepts^TM or Doors^TM, can provide an integrated view of the system complete with requirements.  This means:

• that requirements can be allocated to system elements where their impact can be directly seen (a requirement is driving a feature or a function),
• decompositions can look up-stream to see where they come from, and why, and
• design decisions can be made in the context of the requirements every time.

These tools can also drive requirements through its hierarchies into other downstream tools such as CASE or EE design tools – enabling integrated decisions in the downstream disciplines.  Requirements can also be linked upstream to external standards and regulatory databases, tying the entire organization into regulatory/standards change impact web.  Automated requirements management tools that Vanguard Solutions personnel have used to track system requirements include such COTS RTM database tools such as SLATE^TM, IcConcepts^TM (a.k.a. RTM Tool^©, copyrighted by GEC-Marconi), Proj-Net and Doors^TM.

Vanguard Solutions uses a variety of tools in the application of Quality Assurance and Quality Control, which are identified within the Vanguard Solutions Quality Assurance Plan.  Vanguard Solutions has an extensive reporting process on a regular interval for each project.  Program reviews are the first stage of Quality Assurance.  These reviews are conducted by the Program Manager as well as the Vanguard Solutions Project Manager, on a weekly basis at the Program Manager level, and a monthly and semi-annual basis at the Project Manager level.  These reviews address the program, and are confirmation of meeting the requirements and/or identify deficiencies in the process and/or product.  Vanguard Solutions also utilizes an aggressive survey process to ensure customer/client satisfaction with the services and products provided.  The program reviews and reports, coupled with the surveys conducted, feed the process improvement goals of Vanguard Solutions.

Quality Control of a program is the measurement of products and processes to ensure they comply with the established parameters.  Quality Control is the validation that the product and/or process meet the specifications.  Vanguard Solutions utilizes several methods to collect and analyze data to ensure that the project/program stays within established parameters. Dealing with the management process, Vanguard Solutions addresses extensive reporting and review procedures.  These Management Reviews, conducted by Program Management, also encompass Quality Reviews.  The quality reviews are utilized to ensure that the processes employed are performing to ensure success, or whether they need improvement and adjustment.  This is a continuous cycle that Vanguard Solutions utilizes in their internal process improvement.  For Quality Control processes, one must define what is being measured, then analyze the data, make improvements, and continue to control the process.  For this, Vanguard Solutions follows the processes of DMAIC, which is an acronym that stands for D-Define, M-Measure, A-Analyze, I-Improve, and C-Control.  DMAIC is a Six Sigma structured, disciplined, and aggressive approach to process improvement.  The defining phase defines the process, and determines what is required to be measured.  The measure phase gathers information on the current situation, and involves establishment of a baseline and results in data that pinpoints problem areas or occurrences. The analyze phase analyzes the data, with the goal being identification of root causes.  This will result in a validated theory that will form the basis of corrective action and/or solutions within the improvement phase.  The improve phase implements solutions, addresses root causes, and the output is planned.  Action should eliminate or reduce the identified root causes, and a new plan is created as to how the results of the implementation will be evaluated during the next evaluation period.  The control phase evaluates the solutions, and the plan that was implemented ensures the gains accomplished are maintained by standardizing the process.  Each phase of the 5-step process utilizes tools to assist in the gathering of required information.  Tools utilized are numerous, and vary by project/program.  Tools to be utilized will be specifically outlined in the Quality Assurance Plan.  The most common tools utilized by Vanguard Solutions personnel consist of Microsoft products such as Excel, and Project, other tools are documents produced by Vanguard Solutions during the Management Process.  The common mistake is identifying incorrect information to record.  This is why the data collection plan is critical, and must have Stakeholder "Buy In" to be successful.  The identification of Key Measures and Clarification of Goals is critical to a solid Data Collection Plan.  The important function in the measure phase is to establish a baseline from which everything is evaluated.  The key measures are identified within the data collection plan, and the tools to be utilized are spelled out.

Providing "Gold Standard" support to America's Warfighters