Testing

XCOMPUTE’s test framework has evolved over the years, but two categories of tests remain constant: Unit Tests and Integration Tests. As client-side testing is limited in scope, the content here focuses on server-side testing,

Unit Tests

Unit tests are intended to test broad low-level functionalities to ensure that more complex machinery can be built reliably from smaller pieces. There are over 40 unit tests with ~250 assertions. 100% of unit tests are expected to pass.

Integration Tests

Integration tests are intended to test specific high-level capabilities that mirror workflows or physics whilst omitting the complexity of an active server session. Currently, there are over 70 integration tests…most are passing but still some refinement is required for 100% compliance.

Benchmarking

Benchmarking is a high-level wrapper that drives Integration Tests using several hundred standard cases with a built-in profiler. A report is generated to summarize status.

Across the xcompute platform, common/status.h defines the enumeration:

enum Status : unsigned int
{
    Undefined    = 0,
    Error        = 1, 
    Danger       = 2,
    Warning      = 3,
    Caution      = 4, 
    Alert        = 5,
    Notice       = 6,
    Disabled     = 7,
    Abort        = 8,
    Skipped      = 9,
    Incomplete   = 10,
    Stopped      = 11,
    NoConvergence= 12,
    Modified     = 13,
    Initializing = 14,
    Ready        = 15,
    Queued       = 16,
    Preparing    = 17,
    Busy         = 18,
    Executing    = 19,
    Finishing    = 20,
    Complete     = 21,
    Correct      = 22,
};

Thus a passing test will return status code 21 or 22, while a failing test returns 1 and an undefined outcome would be 0.