[Tutorial] Benchmarking - Printable Version

[Tutorial] Benchmarking - Printable Version

+- SA-MP Forums Archive (https://sampforum.blast.hk)
+-- Forum: SA-MP Scripting and Plugins (https://sampforum.blast.hk/forumdisplay.php?fid=8)
+--- Forum: Scripting Help (https://sampforum.blast.hk/forumdisplay.php?fid=12)
+---- Forum: Tutorials (https://sampforum.blast.hk/forumdisplay.php?fid=70)
+---- Thread: [Tutorial] Benchmarking (/showthread.php?tid=218491)

Pages: 1 2

Benchmarking - Slice - 30.01.2011

Every day I see new faulty ways people benchmark their code, so I figured it'd be a good idea to make some sort of standard that allows easy comparison with other bench results.

I've recently revised my benchmarking macros where I added a "warmup" for highest possible precision. I'll get to that in a little bit.

Why benchmark?
Mainly to make sure your code doesn't cause any performance issues for the server.

How?
You can use my macros to make it really easy!

pawn Код:

START_BENCH( measure time );
FINISH_BENCH( name );

Example:

pawn Код:

START_BENCH( 1000 );
{
    floatdiv( 5412.4121234, 2412.1111 );
}
FINISH_BENCH( "floatdiv" );

Note!

Try and place all of your code inside the benchmarking functions; declare variables and such in-between the macros as well.
Use this only for testing - it will freeze your server for a couple seconds (depending on the time you test).
Don't do anything (music, videos, being in-game, etc.) that could compromise the benchmarking results - you can survive 2 seconds without those things.

The results
The results will be presented as following:
Bench for x: executes, by average, x times/ms.

What this means is how many times the code can run in one millisecond. This gives you a pretty good perspective!
First off, anything that executes more than 1 times/ms will be ok. It will probably make your server work hard when many players are on, or if the code is being used frequently.
If you're going to make functions other people can use, try keeping them above 50 times/ms.
For larger pieces of code used in callbacks or things like that, I'd say keep them above 20 times/ms.

The macros
Here are the macros you will be using.

Before your code:

pawn Код:

#define START_BENCH(%0); {new __a=%0,__b=0,__c,__d=GetTickCount(),__e=1;do{}\
    while(__d==GetTickCount());__c=GetTickCount();__d=__c;while(__c-__d<__a||\
    __e){if(__e){if(__c-__d>=__a){__e=0;__c=GetTickCount();do{}while(__c==\
    GetTickCount());__c=GetTickCount();__d=__c;__b=0;}}{

After your code:

pawn Код:

#define FINISH_BENCH(%0); }__b++;__c=GetTickCount();}printf(" Bench for "\
    %0": executes, by average, %.2f times/ms.",floatdiv(__b,__a));}

Human-readable (only for reference)

The macros are compressed version of these snippets:

START_BENCH

pawn Код:

{ // Put all the benchmarking code inside a block so we can use it multiple times without "variable already exist" errors.
    new
             iMeasureTime = %0, // Number of milliseconds to run the code
             iCount = 0, // Number of time the code was executed
             iTick = GetTickCount( ), // Current tick
             iStartTick, // The tick when the benchmarking started
        bool:bWarmup = true // Warmup?
    ;
    
    do { } while ( iTick == GetTickCount( ) ); // Wait and start right after the tick counter goes up (for precision)
    
    iTick = GetTickCount( );
    iStartTick = iTick;
    
    while ( iTick - iStartTick < iMeasureTime || bWarmup ) // While the execution time hasn't exceeded iMeasureTime or warmup
    {
        if ( bWarmup ) // If we're in a warmup..
        {
            if ( iTick - iStartTick >= iMeasureTime ) // ..and the time has passed
            {
                bWarmup = false; // No more warmup
                
                iTick = GetTickCount( );
                
                do { } while ( iTick == GetTickCount( ) ); // Wait for the next tick
                
                iTick = GetTickCount( );
                iStartTick = iTick;
                
                iCount = 0; // Reset the counter since we just did the warmup
            }
        }
        {

FINISH_BENCH

pawn Код:

ґ       }
        
        iCount++; // Increase the count
        
        iTick = GetTickCount( ); // Store the current tick in iTick
    }
    
    printf( " Bench for " %0 ": executes, by average, %.2f times/ms.", floatdiv( iCount, iMeasureTime ) ); // Print out the results
}

Re: Benchmarking - SlashPT - 30.01.2011

awesome and thanks alot for share

Re: Benchmarking - iMonk3y - 30.01.2011

This is going to be very useful!

Re: Benchmarking - On_Top_Non_Stop - 05.02.2011

Very nice Slice.
Normally I would take what needs to be benchmarked and put it in a seperate function, but these macros are alot better. Thanks

Re: Benchmarking - MrDeath537 - 05.02.2011

Thanks for the macros, now it's easier to do speed tests

AW: Benchmarking - Nero_3D - 05.02.2011

Thats handy, thanks

Re: Benchmarking - legodude - 06.02.2011

lol why all the i prefixes. but it will come in handy for sho'

Re: Benchmarking - JaTochNietDan - 06.02.2011

Quote:

Originally Posted by legodude

lol why all the i prefixes. but it will come in handy for sho'

It's called a naming convention, the i is for integer and similarly the b is for boolean. They are used to show what type of variable it is without having to check its initialization, it's good programming practice. If you worked for a company or did a contract for a program, and you didn't use a standard naming convention, you would be fired.

Anyway good work there Slice, I'm sure this will be good for getting a standard of benchmarking tests around the forums for all of the new systems being released.

Re: Benchmarking - legodude - 06.02.2011

but he uses the i for every var?
that's not the right way then is it?

Re: Benchmarking - JaTochNietDan - 06.02.2011

Quote:

Originally Posted by legodude

but he uses the i for every var?
that's not the right way then is it?

Well which variable pre-fixed with i is not an integer in his code?

Re: Benchmarking - [03]Garsino - 06.02.2011

Quote:

Originally Posted by JaTochNietDan

Hmm, could be really useful I must say.

Also, good tutorial, Slice!

Respuesta: Benchmarking - luis_lpv_22 - 06.02.2011

Hi, thanks for this nice tutorial.

Can you give me an example of benchmarking my gamemode full?

Thanks you.

Re: Benchmarking - MadeMan - 06.02.2011

Thanks for that, but what's the use of warmup ?

Doesn't it affect the results?

Re: Benchmarking - Slice - 06.02.2011

Quote:

Originally Posted by legodude

lol why all the i prefixes. but it will come in handy for sho'

Here's some reading on that area (if you're interested): http://en.wikipedia.org/wiki/Hungarian_notation

Quote:

Originally Posted by luis_lpv_22

Hi, thanks for this nice tutorial.

Can you give me an example of benchmarking my gamemode full?

Thanks you.

This is just for benchmarking single functions/code pieces.

Quote:

Originally Posted by MadeMan

Thanks for that, but what's the use of warmup ?

Doesn't it affect the results?

The warmup affects the results in a good way - it makes them more accurate.
I noticed that if I run two benchmarks without any form of warmup, the first one would get a better result. The warmup makes sure this won't happen.

Re: Benchmarking - Lorenc_ - 13.10.2011

Going to give this a bump since it deserves a clap, helped me work out which function should I use (between fastest and slowest)

Thanks slice, you urbanLegend!

Re: Benchmarking - GrimR - 18.10.2011

I tried this out, but you need to re-word as you can't put the FINISH define below your code because as far as calling it is concerned, it doesn't recognize it as existing.

Also not sure why, but if I try this to test my player enum, it never works (i'm doing it in OnGameModeInit), as if it does not exist.... surely all variables declared above the entire mode will be made before anything runs, especially mode init.

Re: Benchmarking - Slice - 18.10.2011

Quote:

Originally Posted by GrimR

I tried this out, but you need to re-word as you can't put the FINISH define below your code because as far as calling it is concerned, it doesn't recognize it as existing.

Also not sure why, but if I try this to test my player enum, it never works (i'm doing it in OnGameModeInit), as if it does not exist.... surely all variables declared above the entire mode will be made before anything runs, especially mode init.

Could I see the code you're talking about?

Re: Benchmarking - GrimR - 18.10.2011

pawn Code:

enum pData
{
  Float: pHealth,
  Float: pArmor,
         pName[MAX_NAME],
  pSex,
  pAge,
  pAcc,
  pConnected,  // Keep track of if this ID is connected.
  pExp,
  pMoney,
  pBank
};

new  pInfo[500][pData];

I'm just trying to test how pInfo[playerid][pConnected] would go up against IsPlayerConnected() with a loop and check. The 500 was just to test if MAX_PLAYERS was somehow an issue.

pawn Code:

new j = 0;

  START_BENCH( 499 );
  {
    if (pInfo[j][pConnected]) {}
    j++;
  }
  FINISH_BENCH( "My Test" );

It just doesn't do anything. If I replace it with IsPlayerConnected(j) instead, it spits out what it should. No compilation errors, or run time errors etc.

Re: Benchmarking - Slice - 18.10.2011

Quote:

Originally Posted by GrimR

pawn Code:

enum pData
{
  Float: pHealth,
  Float: pArmor,
         pName[MAX_NAME],
  pSex,
  pAge,
  pAcc,
  pConnected,  // Keep track of if this ID is connected.
  pExp,
  pMoney,
  pBank
};

new  pInfo[500][pData];

I'm just trying to test how pInfo[playerid][pConnected] would go up against IsPlayerConnected() with a loop and check. The 500 was just to test if MAX_PLAYERS was somehow an issue.

pawn Code:

new j = 0;

  START_BENCH( 499 );
  {
    if (pInfo[j][pConnected]) {}
    j++;
  }
  FINISH_BENCH( "My Test" );

It just doesn't do anything. If I replace it with IsPlayerConnected(j) instead, it spits out what it should. No compilation errors, or run time errors etc.

The parameter in START_BENCH is the time it should be benchmarked (in milliseconds). The code inside the benchmark should run once independently, not taking in account previous repetitions.
Proper usage in this case would be:

pawn Code:

START_BENCH( 1000 );
{
    for (new j = 0; i < MAX_PLAYERS; j++) {
        if (pInfo[j][pConnected]) {}
    }
}
FINISH_BENCH( "My Test" );

Re: Benchmarking - GrimR - 18.10.2011

Aha well that clears that up lol.

I still don't understand why it worked with IsPlayerConnected but not passing my enumeration array, but I will try this when I can and see if I get a better result.