| Home | Trees | Indices | Help |
|---|
|
|
A 'Result' describes the outcome of a test.
A 'Result' contains two pieces of data: an outcome and a set of annotations. The outcome indicates whether the test passed or failed. More specifically, the outcome may be one of the following constants:
'Result.PASS' -- The test passed.
'Result.FAIL' -- The test failed.
'Result.ERROR' -- Something went wrong in the process of trying to execute the test. For example, if the Python code implementing the 'Run' method in the test class raised an exception, the outcome would be 'Result.ERROR'.
'Result.UNTESTED' -- QMTest did not even try to run the test. For example, if a prerequiste was not satisfied, then this outcome will be used.'
The annotations are a dictionary, mapping strings to strings.
The indices should be of the form 'class.name' where 'class' is the name of the test class that created the annotation. Any annotations created by QMTest, as opposed to the test class, will have indices of the form 'qmtest.name'.
The annotation values are HTML. When displayed in the GUI, the HTML is inserted directly into the result page; when the command-line interface is used the HTML is converted to plain text.
Currently, QMTest recognizes the following built-in annotations:
'Result.CAUSE' -- For results whose outcome is not 'FAIL', this annotation gives a brief description of why the test failed. The preferred form of this message is a phrase like "Incorrect output." or "Exception thrown." The message should begin with a capital letter and end with a period. Most results formatters will display this information prominently.
'Result.EXCEPTION' -- If an exeption was thrown during the test execution, a brief description of the exception.
'Result.TARGET' -- This annotation indicates on which target the test was executed.
'Result.TRACEBACK' -- If an exeption was thrown during the test execution, a representation of the traceback indicating where the exception was thrown.
A 'Result' object has methods that allow it to act as a dictionary from annotation names to annotation values. You can directly add an annotation to a 'Result' by writing code of the form 'result[CAUSE] = "Exception thrown."'.
A 'Result' object is also used to describe the outcome of executing either setup or cleanup phase of a 'Resource'.
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
RESOURCE_SETUP = |
|||
RESOURCE_CLEANUP = |
|||
TEST = |
|||
FAIL = |
|||
ERROR = |
|||
UNTESTED = |
|||
PASS = |
|||
CAUSE = |
|||
EXCEPTION = |
|||
RESOURCE = |
|||
TARGET = |
|||
TRACEBACK = |
|||
START_TIME = |
|||
END_TIME = |
|||
TIMEOUT_DETAIL = |
|||
kinds = A list of the possible kinds. |
|||
outcomes = A list of the possible outcomes. |
|||
|
|||
Construct a new 'Result'. 'kind' -- The kind of result. The value must be one of the 'Result.kinds'. 'id' -- The label for the test or resource to which this result corresponds. 'outcome' -- The outcome associated with the test. The value must be one of the 'Result.outcomes'. 'annotations' -- The annotations associated with the test. |
Return a representation of this result for pickling. By using an explicit tuple representation of 'Result's when storing them in a pickle file, we decouple our storage format from internal implementation details (e.g., the names of private variables). |
Return the kind of result this is. returns -- The kind of entity (one of the 'kinds') to which this result corresponds. |
Return the outcome associated with the test. returns -- The outcome associated with the test. This value will be one of the 'Result.outcomes'. |
Set the outcome associated with the test. 'outcome' -- One of the 'Result.outcomes'. 'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation. 'annotations' -- The annotations are added to the current set of annotations. |
Mark the test as failing. 'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation. 'annotations' -- The annotations are added to the current set of annotations. |
Return the label for the test or resource. returns -- A label indicating indicating to which test or resource this result corresponds. |
Return the cause of failure, if the test failed. returns -- If the test failed, return the cause of the failure, if available. |
Set the cause of failure. 'cause' -- A string indicating the cause of failure. Like all annotations, 'cause' will be interested as HTML. |
Return a version of string suitable for an annotation value. Performs appropriate quoting for a string that should be taken verbatim; this includes HTML entity escaping, and addition of <pre> tags. 'string' -- The verbatim string to be quoted. returns -- The quoted string. |
Note that an exception occurred during execution. 'exc_info' -- A triple, in the same form as that returned from 'sys.exc_info'. If 'None', the value of 'sys.exc_info()' is used instead. 'cause' -- The value of the 'Result.CAUSE' annotation. If 'None', a default message is used. 'outcome' -- The outcome of the test, now that the exception has occurred. A test class can call this method if an exception occurs while the test is being run. |
Check the exit status from a command. 'prefix' -- The prefix that should be used when creating result annotations. 'desc' -- A description of the executing program. 'status' -- The exit status, as returned by 'waitpid'. 'non_zero_exit_ok' -- True if a non-zero exit code is not considered failure. returns -- False if the test failed, true otherwise. |
Generate a DOM element node for this result. Note that the context is not represented in the DOM node. 'document' -- The containing DOM document. returns -- The element created. |
|
|||
outcomesA list of the possible outcomes. The order of the 'outcomes' is significant; they are ordered from most interesting to least interesting from the point of view of someone browsing results.
|
| Home | Trees | Indices | Help |
|---|
| Generated by Epydoc 3.0.1 on Fri Dec 23 12:30:41 2011 | http://epydoc.sourceforge.net |