QCumber - KX Developer (2024)

qCumber is a unit testing library for q, which offers assertion testing, benchmark testing, and property-basedtesting. This library is provided both as a q API and as a q command line script. Additionally, there is integrationwithin Developer which provides Right-click > Test capabilities on repositories and test files.

Tests are stored in files with the .quke extension. An example of a simple test file (count.quke) is shown below.

feature Count should return the count of its input expect count on an atom to return 1 1 ~ count `a expect count of 1 drop list to return n - 1 n:1 + first 1?10; l:1 _ n?`8; count[l] ~ n - 1

The above feature test (feature) contains one described behavior (should) which contains two assertions (expect)to check the the validity of the behavior. The feature, should, and expect blocks contain natural language,allowing simple organization and structure of tests, as well as clear specification of intent in human-readable terms.

expect assertions are always contained in a should, and should declarations are always contained in a feature. Each .quke test file can have one or more features.

expect assertions contain q code. The final statement in the code block should be true (1b) to result in apassing test. For all other return values, it fails. Local assignments made within an expect block do notimpact other expect blocks.

Running tests

There are three ways of executing tests in qCumber: command line, API, or within the Developer IDE.

Command line

qCumber tests can be run at the command line by using the qcumber.q_ library packaged in ax-libraries.This is useful for both q developers and for automated q builds. Test results can be printed to the console,or written to serialized q, JSON, or JUnit XML formats. See the Libraries section for moreinformation.

$ q $AXLIBRARIES_HOME/ws/qcumber.q_ -helpqCumber: q test runnerUsage: q qcumber.q_ <flags>Flags: -help - display this help -src <dir> - path to a q script to load before running tests -test <dir/file> - path to a quke file or directory of quke files to test [-out] <dir> - path to a file to store results files supported: .json, .xml (junit), .dat (q: get `:file.dat) [-insert] <key=val> - add columns to the output: col1=val1,col2=val2,... [-times] <number> - number of times to run each property block [-color] - colorize the output [-showAll] - print passed and skipped tests as well as failures to stdout [-quiet] - do not print any results to stdout [-reporter] <file.q> - custom report from a q file defining a function 'write[file; results]' [-errorsAsFailures] - convert all errors to failures in final output useful for systems which only look at 'failure' [-exitFailuresWith] <number> - exit code to use when script terminates and there are failing tests defaults to '1' [-breakOnErrors] - propagate errors in unit tests to facilitate debugging [-version] - print the library version

API

The qcumber.q_ library, if loaded into a q process from another script or from a running process provides.qu.runTestFile and .qu.runTestFolder functions, returning a table of test results.

IDE

Tests can also be executed within the IDE by right-clicking on an artifact in the artifact tree, and clickingCode > Run Tests.

Using Code > Run Tests on:

  • a repository or module will run all the .quke files contained
  • a .quke file will run that file
  • functions or data will run associated test files

QCumber - KX Developer (1)

Tests can be associated with functions, data or modules (See Conventions, namespaces, and contexts below).Multiple files, modules, and repositories may be tested simultaneously by multi-selectingartifacts. It is also possible to test an individual file by right clicking an open editor window and clickingTest.

Output

The output for qCumber tests reports the tests that failed, as well as any malformed test filesand any skipped tests.

If running from the command line, test results can be exported to a number of on-disk formats, or printed to theconsole, as shown below:

QCumber - KX Developer (2)

If running within the Developer IDE, the failed tests, malformed files, and skipped tests are nested in the output as expandable lists that can be opened by clicking the arrow.

QCumber - KX Developer (3)

The failed tests section will display the test blocks that failed, the file name and line numberwhere the failed test occurred, and, depending on the type of test, details as to the reason thatthe test failed. These could be failed test conditions or errors in the q code. Clicking the linenumber where the error occurred will open the failed test file directly to the test that failed.

The malformed files section displays a list of formatting errors as well as the file name andline number where the error occurred. These can also be clicked to open the location of the error.

QCumber - KX Developer (4)

Skipped tests show only the identity and location of the skipped tests (see below).

Test setup and teardown

qCumber provides several optional blocks which can be used to initialize state prior to running tests, and to clean up the environment after tests have run. Each are defined under a feature.

  • before will execute arbitrary code before running the first should, bench, or property block for a feature
  • after will execute arbitrary code after running the first should, bench, or property block for a feature
  • before each will execute arbitrary code before running each should, bench, or property block for a feature
  • after each will execute arbitrary code after running each should, bench, or property block for a feature

As with expect blocks, assignments made in a before or before each block will not impact other blocks. Thus,to set state visible to the expect blocks, globally assign to identifiers with ::, as the examples below demonstrate.

As an example, the below test sets initial state and deletes the state after running the should checks.

feature demoBeforeAndAfter before randomNumbers :: 50?50i; after delete randomNumbers from `. should use data created in our before block expect its count to be 50 50 ~ count randomNumbers expect it to be integers `ints ~ typeOf randomNumbers

The example below runs the before each block before each should:

feature demoBeforeEach before each $[`randomList in key system "d"; randomList ,: 1?`8; randomList :: 1?`8]; after each delete randomList from (system "d") should showcase that our list is growing expect the count to be 1 1 ~ count randomList should showcase that our list is still growing expect the count to be 2 2 ~ count randomList should showcase that our list is still growing again expect the count to be 3 3 ~ count randomList

Comparing values

To show the expected and actual values in the test output, the .qu.compare function can be used.The function accepts two arguments, the expected result and the actual result. If these two results match,then the function returns boolean true (1b), and the test passes.If the results do not match, then a dictionary will be passed to the expect block containing the expectedand actual results.

feature flip should test flip operator expect flip to produce a table l : til 6; .qu.compare[ ([] a: 0 1 2 3 4 5; b: 5 4 3 2 1 0); flip `a`b!(l; reverse l)]

Skipping tests

qCumber supports ignoring tests by prepending an x to the front of a given test block. qCumbersupports skipping feature, should, expect, bench, and property blocks.

For example, in the following test the first expect block is skipped. In the Developer IDE,the skipped section is grayed out.

feature count should return the count of its input xexpect to skip this test 1 ~ count `a expect correct count of list of length 2 2 ~ count `a`b

The output from this test would report the skipped test. To view the skipped tests in the output,click the arrow to the right of the Skipped Tests header.

QCumber - KX Developer (5)

Skipping tests conditionally

A skip if blocks allow a feature to be skipped if a condition is true.If the skip if block returns boolean false (0b), then the feature is executed.If a non-boolean value is returned or if an error is thrown the feature aborts.

feature loadFile skip if not .z.o ~ `w64 ...feature loadFile skip if not .z.o like "m*" ...

Benchmark blocks

qCumber provides a facility for benchmarking and comparing code run time. A simple example isthe following:

feature benchmark 1 bench baseline til 100000000 behaviour til 1

Benchmark blocks appear at the same level as should blocks. The simple case above states that the code til 1should be at least as fast as til 100000000.

Setup and teardown of a benchmark is allowed as well, as in:

feature benchmark 2 bench setup .ab.num : 1 baseline til 100000000 behaviour til .ab.num teardown delete num from `.ab

For more information about benchmark blocks, refer to qu in the Function Reference available from theHelp menu.

QCumber - KX Developer (2024)
Top Articles
Latest Posts
Recommended Articles
Article information

Author: Francesca Jacobs Ret

Last Updated:

Views: 6078

Rating: 4.8 / 5 (48 voted)

Reviews: 95% of readers found this page helpful

Author information

Name: Francesca Jacobs Ret

Birthday: 1996-12-09

Address: Apt. 141 1406 Mitch Summit, New Teganshire, UT 82655-0699

Phone: +2296092334654

Job: Technology Architect

Hobby: Snowboarding, Scouting, Foreign language learning, Dowsing, Baton twirling, Sculpting, Cabaret

Introduction: My name is Francesca Jacobs Ret, I am a innocent, super, beautiful, charming, lucky, gentle, clever person who loves writing and wants to share my knowledge and understanding with you.