[Reynolds]

pawn-test

pawn-test is a simple automated testing framework for PAWN code. The framework provides a number of functions for setting up and managing test suites and test cases. It was designed to be compatible with the PAWN compiler that comes with San Andreas: Multiplayer (Pawn 3.2.3664), but with a bit of tinkering it should work with other PAWN projects as well.

Even though there exists a working testing environment for SA:MP (that would be y_testing), it is part of the YSI library, which many scripters do not want to deal with. It also creates a lot of syntactic sugar which may not appeal to everyone. This is a standalone, lightweight library that can be used without additional dependencies.

License

pawn-test is an open source project covered by the MIT License. See the attached LICENSE file for more information or visit http://opensource.org/licenses/MIT.

Using pawn-test

To use pawn-test in a PAWN project, copy the src directory into your project's pawno/include directory. Rename the directory to pawn-test.

To run pawn-test with the default configuration, simply include the bootstrapper in your project:

Код:

#include <pawn_test/pawn_test>

If you want to provide your own configuration, edit the configuration file (config.inc) and change your settings there.

Getting Started

The pawn-test documentation includes an article on how to get started, but a unit test for a simple test function looks something like this:

pawn Код:

#include <a_samp>
#include <pawn_test/pawn_test>

forward CheckAnswer();

stock AddOne(x)
{
    return x + 1;
}

public CheckAnswer()
{
    return PawnTest_Assert( AddOne(5) == 7, "AddOne(5) == 7" );
}

main()
{
    new TestSuite = PawnTest_InitSuite("My First Unit Test");

    PawnTest_AddCase(TestSuite, "CheckAnswer");
    PawnTest_Run(TestSuite);

    PawnTest_DestroySuite(TestSuite);
}

This produces the following output:

Код:

[==========] running test suite "My First Unit Test" w/ 1 test case(s)...
[ RUN      ] test "CheckAnswer" (100%)
             assertion "AddOne(5) == 7" failed.
[     FAIL ] test "CheckAnswer" (1 ms, 0.001 s in total).
[==========] test suite "My First Unit Test" (#0) finished, 1 test(s) ran.
[  FAILED  ] 1/1 test(s) failed.
[==========] testing stopped.

Download

pawn-test is developed on GitHub. The latest stable release can be downloaded as a zip/tarball from the downloads page.

The complete documentation is available in the wiki of the repository. Every API function is documented and the API documentation can be generated using pawncc and a file in the /docs directory.

[Abagail]

I'll have to try this sometime, it looks like a useful tool!

[Crayder]

So it's basically like y_tests, except y_tests is a bit better. I'm not downing yours I'm just saying it can still be better. Nice job!

[Reynolds]

Actually, no.

One of the key powers of pawn-test is that it supports xUnit-style testing. You can create test suites, add test cases to it, dynamically skip testing, there's even macros for floating-point delta comparison. Test functions are simple publics so you can call them outside the test environment, once, twice or even more times in the same test suite. You can also use a test case in different test suites. pawn-test can handle expected failures and most importantly, it has xUnit-like setup and teardown functions (test fixtures), making it a lot easier to test production code with mock players (NPCs). It also means that you can make your test cases independent of the test suite they're currently in.

In my opinion, y_testing is far less powerful than that.

Simplicity and power aren't mutually exclusive: you don't have to use any of these features, it's easy enough to use that any beginner can pick it up and write a simple test like the one in the OP. After all, a test case that's never run or a test library that never gets used is like a tree falling in the woods. I don't think y_testing has the kind of barrier of entry that would enable people to test their code, which is a shame because 80% of SAMP projects fail because of a mysterious bug.

I recommend that you read the documentation, it goes into detail about everything I mentioned. I just thought it'd be better if the post did not contain examples that need to be updated after every minor release.

[Crayder]

Then I have a suggestion that'll do this a lot of good.
Add parameters to PawnTest_AddCase (using #emit assembly code) to make it call the callbacks with more parameters. That would make this possible:

pawn Код:

#include strlib

forward CheckAnswer(var1, var2, and, out);

GiveSome(x, a)
{
    return x + a;
}
TakeSome(x, a)
{
    return x - a;
}

public CheckAnswer(var1, var2, and, out)
{
    return 
        PawnTest_Assert(GiveSome(var1, and) == out, sprintf("GiveSome(%i, %i) == %i", var1, and, out)) & 
        PawnTest_Assert(TakeSome(var2, and) == out, sprintf("TakeSome(%i, %i) == %i", var2, and, out));
}

main()
{
    new TestSuite = PawnTest_InitSuite("My First Unit Test");

    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 8, 12, 2, 10);
    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 6, 14, 4, 10);
    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 0, 20, 10, 10);
    PawnTest_Run(TestSuite);

    PawnTest_DestroySuite(TestSuite);
}

If you don't know assembly, just copy it from someone else's code (great example, sprintf, by Slice)... xD

[Reynolds]

This is normally a bad idea. A unit test should do only one thing and shouldn't contain more than one assertion, also they should be able to run in any order. Moreover, each test should test a single, separable and verifiable segment of code. Passing arguments like that is not productive. pawn-test passes the case id and the test id by default but does not provide an interface for custom arguments.

Your test case tests two functions at the same time. Moreover, the only way for a person to know which case actually failed is if you look at the test suite.

You should create two separate test cases and daisy chain these separate cases in a for loop. If an assertion fails, you return CASE_FAILED.

I will post an example when I get home.

[Crayder]

My example is not the point. Even this would apply:

pawn Код:

#include strlib

forward CheckAnswer(var, and, out);

add(x, a)
{
    return x + a;
}

public CheckAnswer(var, and, out)
{
    return 
        PawnTest_Assert(add(var, and) == out, sprintf("add(%i, %i) == %i", var, and, out));
}

main()
{
    new TestSuite = PawnTest_InitSuite("My First Unit Test");

    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 8, 2, 10);
    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 6, 4, 10);
    PawnTest_AddCase(TestSuite, "CheckAnswer", "iiii", 0, 10, 10);
    PawnTest_Run(TestSuite);

    PawnTest_DestroySuite(TestSuite);
}

Also they will be optional. Think of it like the optional parameters in CallLocalFunction.

[Reynolds]

Again, I would like to quote myself:

Quote:

Originally Posted by Reynolds Moreover, each test should test a single, separable and verifiable segment of code.

The function you've written isn't a unit test because it doesn't test a separable segment of your code and it's not reusable. If it requires extra parameters to work (aside from the test environment created by the text fixture), it can't be called a unit test.

pawn-test's API was designed in a way that makes it easier for the user to do the right thing than the wrong thing.

This is why the simple Assert() function returns PAWNTEST_CASE_FAILED and PAWNTEST_CASE_PASSED instead of true and false. There's an AssertEx() function that does this but since creating more than one assertion in a unit test isn't recommended, it's not the default option. Moreover, you can simply return PAWNTEST_CASE_FAILED and PAWNTEST_CASE_PASSED if you want to, but again, that's not usually necessary so it's a lower-level API feature.

What you should do is refactor your code like this:

Код:

forward CheckVariousAnswers();

add(x, a)
{
    return x + a;
}

public CheckVariousAnswers()
{
    new Values[3][3] = {{8, 2, 10}, {6, 4, 10}, {0, 10, 10}};
    new bool:ok = true;

    for( new i = 0; i<sizeof(Values); i++ ) {
        ok &= PawnTest_AssertEx(    add(Values[i][0], Values[i][1]) == Values[i][2],
                                    sprintf("add(%i, %i) == %i", Values[i][0], Values[i][1], Values[i][2])
                                );
        if( !ok )
            return PAWNTEST_CASE_FAILED;
    }

    return PAWNTEST_CASE_PASSED;
}


main()
{
    new TestSuite = PawnTest_InitSuite("My First Unit Test");

    PawnTest_AddCase(TestSuite, "CheckVariousAnswers");
    PawnTest_Run(TestSuite);

    PawnTest_DestroySuite(TestSuite);
}

CheckVariousAnswers() is now independent of the test environment and can be included in any further test suites that you might have to write.

Read this document for more details on how unit testing works (this is specific to Python but the principles are identical).

[Reynolds]

A new patch version, pawn-test 2.0.1, has come out.

It fixes an off-by-one memory error in functions that use memcpy() and strlen() at the same time.

The patch version can be downloaded from the GitHub repository.

[jonrb]

It makes me very happy that this exists. Nice one!