One of the most sought after
technologies for computational geometry software code
developers is layout test cases from their customers.
For the customer, such test cases are treasured as
well and they have been reluctant to give them out when they
contain layout data considered proprietary. Not only
are good test cases difficult to acquire, good test
cases are difficult to create in a way that
gives good code coverage. Furthermore, few
developers make the same mistake twice so, the test
cases must be rigorous in their scope and coverage.
gmTest combines
mathematical analysis of computational geometry
algorithms, intelligent automated layout synthesis,
geometric hashing and code coverage methods to create
a system which automates the verification of CAD
and EDA tools without such a heavy need for real
layout test cases. Such a task was daunting to say the
least. The sheer volume of test cases required
(probably) has been enough to discourage companies
in the past from creating such an operation-agnostic
palette of test cases.
To build these test cases, we had to develop a high
level control language, language operators, and
language-to-layout converters. To validate (and to
choose which of) the hundreds of millions of test
cases we built, we developed source geometry,
established algorithms, heuristics and
statistical measures.
Simply speaking, the test case suite is
instance-based meaning we can parse a language into
instances, variants of those instances (and slightly
less equally) varied levels of complexities of
combinations of instances that are able to
capture errant code development whether syntax or
matters of interoperability, functionality,
compliance, correctness, placement accuracy, etc.
Each test case is linked back to extensive
(electronic) documentation justifying the existence of
the test case, whether it is a positive (or negative)
test case, whether it is an equivalent and a view of
the test case if the test cases is positive, whatever
the format. We organize the documentation, files names
and indexes by the instance we plan to target. This
lessens the need to share the test case with any
company who does not have the test cases themselves.
There are 23 modules and approaching 250,000 test
cases.
The test cases are intended for code developers and
for customers who want to expand their test case
repertoire (to increase coverage) to validate software
from their external suppliers. The use of commercial
test cases like these can be a win-win for both the
supplier and customer. Our compliance test cases
quickly resolved bugs and matters of interoperability
in the field much more quickly than had the test cases
not been available. We exercised the same level (if
not more) of diligence in our development efforts for
gmTest and therefore we expect confidently that we
will be able solve more problems during development
and more effectively solve problems in the field.
