Purpose & status
The censhare Tester allows to run tests scripted in XML and Groovy (and partly Java), simulating a client behaviour. While unit-tests usually concentrate on a single aspect of a product, the censhare Tester is focusing on bigger test scenarios, like load-tests.
What is needed
- A running censhare server (version 5.x and higher), with an Oracle DB connection and some valid credentials (username/password) to access the server.
- An XML file that describes the test scenario: what commands to send to the server, and when.
- Possibly other files that are needed for the test, e.g. a file with assets to import or features to add to the DB.
How to run a test
If the censhare Tester has not been compiled yet, run the following command in a Terminal:
|
To run a test, type the following command in the Terminal:
|
The censhare Tester already comes with a number of tests. You can try the following for instance:
|
If you want to run the test inside eclipse or intelliJ, run the class com.censhare.tester.Tester and give the XML file as a parameter. Refer to your preferred IDE on how to do that.
How to write a test
General structure of a test scenario
A test is described by an XML file with the following structure:
|
- Line 3, contains the relevant information for the server, i.e. how to access it. The id attribute allows to reference it later in the XML.
- On lines 6-9, we start an action
TouchAssetAction. This action is a Java class of the same name. Line 7 indicates how to connect to the server, by using the information from line 3. The “action” node can contain additional XML nodes which will tell the action what to do. - On lines 11-12, more actions are defined.
Parameters
A test can be parametrized by using standard java properties. Each property must be declared inside special <params> element together with default value and option type (types are actually useful only with Groovy scripting). These parameters/properties can be used in a two ways:
- By plain string replacement of special placeholders in a form
@{PROPERTY_NAME}:- The replacement is done once, before executing any action
- The replacement is done in the entire XML, including the attributes
- This is useful mainly for server declaration
- As a variable:
- Useful in groovy action
- Remember to declare the correct type, unless it is really supposed to be
String - Shadowed by variable declaration (see later)
|
Parameters can be set when invoking from the command line:
|
The server node
The server node indicates how to access the server. There are three ways of doing this:
RMI access (censhare 4): indicates a configuration name id , a URL rmi_url, and a user and password
<serverid="server1"rmi_url="rmi://dev-load-test:1099/corpus.RMIServer"user="XXX"password="YYY"/>API-Command access (censhare 5): indicate a configuration name id, a URL url, and a user and password
<serverid="server2"url="http://dev-load-test:9000/censhare5/client/"user="XXX"password="YYY"/>Reference to a server configuration: indicates a configuration name idref. The tester will replace the node with the server node that has the same id as given in idref.
<serveridref="server1"/>
Note: Avoid hardcoded hostnames in the server declaration, use parameters instead.
The action node
The action node tells the server which Java class to execute:
|
The Java class should sub-class the abstract class com.censhare.tester.actions.AbstractAction, and the executeInternal()(line 4) method should be implemented. The pre- and post- execution methods (line 2-3) can be overwritten as needed. The class also defines some methods to access the XML configuration of the action (lines 11-14), variables (lines 21-24) or the current session (line 17). See the next section below on variables and sessions.
|
Additionally class com.censhare.tester.actions.AbstractSshAction establishes an SSH connection in its pre-execution and closes the connection in its post-execution.
Statements
Statements are special actions. There are 3 kinds of statements: SessionAction (to create a session with the server), Loop (to iterate over a set of actions), and VariablesDeclaration. They are defined in the package com.censhare.tester.actions.statements.
SessionAction
This statement is in fact a com.censhare.tester.actions.AbstractAction that is specified as a normal action. It opens a session with the server and keeps it during the execution of the actions. The SessionAction will run its children nodes successively if they are part of: variables, loop, action. Other nodes are ignored (except the server node which holds the url/user/pwd but has no actions). This action needs a server configuration with an API-Command access (see The server node).
Example:
|
- line 2: server configuration
- line 3: start of a session with the server (referenced in line 4)
- line 5: declaration of some variables (see below)
- line 6-8: specify some actions. Line 7 is a loop-action (see below)
- line 9: end of the session, the connection with the server will be closed.
Variables
The variables node declares a set of global variables that can later be accessed in Java classes or directly in the XML. It is supported by the Java class VariablesDeclaration. Such a node can occur in any action that subclasses AbstractAction, like for instance the SessionAction or Loop statements.
Declaring a variable
Note: This feature is useful only for pure XML tests, without any Groovy.
A variable has a name (string), a type ( long, double, string, list) and a value:
- long, double and string types: the value is specified as an attribute of the XML node. For long and double, it must be formatted so “new Long/Double(value)” is working in Java.
- list: the value is specified as the content of the XML node. It is a comma-separated list of values and trimmed with spaces. This means that “my first, my second , term3” will result in “my first” , “my second” , “term3”. It is not possible to have elements with commas inside.
Example: Line 2 defines a long named myVar with an initial value of 10, while lines 3-5 define a list myList with 3 values.
|
Accessing a variable
You can access the value of a variable in Java by using getVariable(String) (defined in com.censhare.tester.actions.AbstractAction).
Variables are also available in Groovy. Be careful about their type, though - usually they are strings (unlike for parameters, there is no support for declared type). There is, however, hardly any reason to mix these variable and groovy scripting.
|
You can access the value of a variable inside the XML. Use {$varname} to refer to the variable named varname. It works only inside a node’s attributes and it does not work for lists (i.e. the whole list). Be aware: in order for this to work, you need to get the configuration of the action with getConfigWithValues() (defined in com.censhare.tester.actions.AbstractAction)
Examples:
|
|
Loop
Note: Loops are useful only for pure XML tests. Groovy allows much better flexibility and it should be preferred.
The loop node allows to iterate over a list of actions, in a sequential manner (i.e. like a for-loop), or in parallel. It is supported by the Java class Loop. Such a node can occur in any action that subclasses AbstractAction, like for instance the SessionAction or Loop statements.
The loop has the following attributes (all mandatory unless specified otherwise):
- loopvar: name of the variable loop, so it can be accessed by the actions of the loop (see Variables).
- values: the values to iterate over. It can be the name of a variable of type list, or a number in a range of the form “start..end” (inclusive), with
start <= end - parallel (optional): yes / no . Default to no.
- delay (optional): delay between two iterations, in milliseconds. For parallel iterations, the loop will wait the given delay before starting the next iteration-thread. Default to 0.
Example 1: Perform the actions 10 times, every 500ms, in a sequential manner.
|
Example 2: Perform the actions for each element in a list, and do it in parallel.
|
API-Command actions
Those actions will work only inside a SessionAction. They are using the API-Command to execute and are available only in censhare 5.
com.censhare.tester.actions.server.load.LiveQueryAction
Creates a LiveQuery. It will be destroyed when the parent SessionAction gets disconnected. It also records all received updates and allows CheckAction to check whether their amount is correct or not.
Example: With the use of a variable (see VariablesDeclaration):
|
RMI-oriented actions
Those actions are using the RMI to execute. Some of them will be deprecated once censhare 5 has a stable API-Command. This is a non-exhaustive list, please refer to the Java implementation or the files in the config directory (e.g. config/actions.xml)
com.censhare.tester.actions.server.load.ModifyLoadTestAssetAction
Performs some modifications on assets:
- Modify assets: search for some assets (via a query), and modify the given feature for all of the returned assets.
- Delete assets: search for some assets (via a query) and physically delete them.
Nodes inside the action XML:
- Node query (mandatory): Specifies the assets that will be modified, only one query node can be used per action.
- Node params (at least 1): Specifies on what to do depending on its attributes:
- mode (mandatory):
modifyFeature,deleteFeature,deleteAssets(last bullet above). - featureKey (mandatory if mode != “deleteAssets”): name of the feature to modify/delete. TODO : not all special features are implemented: only asset.description , and features of type String.
- value (mandatory only for mode=“modifyFeature”): value of the feature. Only strings are handled. Future versions should have a better handling of feature types.
- append (optional, used only for mode=“modifyFeature”): append to existing value. If equals to date or assetID , will use the current date or the current assetID.
- mode (mandatory):
The action will perform the modifications one params-node after the other and then send the updated assets to the server. Exception: with mode=“deleteAssets”, the action will delete the assets immediately and return (ignoring the rest of the ‘params’ nodes).
Sample config (note line 1: we name the server configuration serverrmi in order to make a distinction with possible API-Command access configurations):
|
Other actions
All classes are in the package:
PrintLiveQueryStatisticsAction: Prints the LiveQuery statistics with the logger. You should also call ResetLiveQueryStatisticsAction to have proper stats.ResetLiveQueryStatisticsAction: Resets the LiveQuery statistics.WaitLiveQueriesAreUpToDate: It will ask every 2 seconds for the list of pending queries. Stop if all are UpToDate or Cancelled.TermQueryAction: Execute fulltext queries with a list of terms on the target server. See some sample config in the javadoc.TouchAssetAction: Execute asset updates on a random asset from a query. See some sample config in the javadoc.
More actions can be found in the package com.censhare.tester.actions.server:
AdminClientBatchActionAssetImportActionCreateDatabaseActionDropDatabaseActionInstallServerActionMetaImportActionRemoveServerActionServerTestActionStartServerActionStopServerActionWaitTasksAction
Actions with no connection to the server
Those command don’t use a connection to the server and can be used outside a SessionAction
com.censhare.tester.actions.server.load.Wait
Waits a number of milliseconds. Sample config for waiting 100 ms:
|
Check action
This special action allows (in cooperation with other actions) tests to actually test something by checking the expected behaviour. It defines what should be checked at a given place and records the results.
The checks themselves are not actually executed by this action; they must be implemented in some other action, executed somewhere before the check. CheckAction just calls the callbacks of these actions, registers and passes them to its configuration.
A typical example (and at this moment, the only supported check) is LiveQueryAction which counts the received notifications from a promise and allows them to be checked
|
Implementation note: A check implementation should return a list of results (which is then added to the global results), but it can also just throw an AssertionError - a result of type ERROR is automatically added.
The results of all checks are printed at the end of the entire test run, in plaintext form.
Test examples
Creates 100 LiveQueries and perform some database updates. This test concentrates on the time it takes to send many updates.
|
The following test creates a LiveQuery that will receive a long list of documents. The structure is somewhat similar to the previous test.
|