[iLearner]

I am using a while loop to search in a array, with strfind... e.g:

PHP Code:

new myarray[MAX_MESSAGES][7][1000] =
{
    {"hey", "Hello, how are"},
    {"hi", "Hello , how are "}
}; 


Command:

PHP Code:

while(count != MAX_MESSAGES)
    {
        new Index = getIndex();
        printf("%d",count);
        count++;
        printf("%d",Index);
        if(strfind(myarray[Index][0], text, true)!=-1)
        {
            printf("Found it");
            k = Index;
            found = true;
            break;
        }    
        else
        {
            printf("didnt find it");
            found = false;
        }
    } 


Basically that runs the loop until we're at MAX_MESSAGES, but breaks if we found what we're looking for.

I have approx 330 messages (MAX_MESSAGES), using that cmd, it takes around 3-4 second to get response, is it cuz of while, strfind or something else?

[NaS]

Quote:

Originally Posted by iLearner I am using a while loop to search in a array, with strfind... e.g: PHP Code: new myarray[MAX_MESSAGES][7][1000] = {     {"hey", "Hello, how are"},     {"hi", "Hello , how are "} };  Command: PHP Code: while(count != MAX_MESSAGES)     {         new Index = getIndex();         printf("%d",count);         count++;         printf("%d",Index);         if(strfind(myarray[Index][0], text, true)!=-1)         {             printf("Found it");             k = Index;             found = true;             break;         }             else         {             printf("didnt find it");             found = false;         }     }  Basically that runs the loop until we're at MAX_MESSAGES, but breaks if we found what we're looking for. I have approx 330 messages (MAX_MESSAGES), using that cmd, it takes around 3-4 second to get response, is it cuz of while, strfind or something else?

I hope you tested without the prints. If you didn't, try without since print is the slowest PAWN function I know - relative to what it does of course.

Also strfind can be quite slow, it's just another loop going through the whole content of the strings, depending on the search term it has to check every character, and sometimes even several characters multiple times.

It seems to be a bit slow, but considering you have 330 messages (probably lengthier ones as well) it's not going to speed up much by optimization.

Btw what do you need the getIndex function for? You could just use a for-loop for indexes. Or use count as index in the existing code. That's not going to improve much but still removes the overhead of having to get an index from a function, although it's already existing.

[iLearner]

The 2 functions i use:

PHP Code:

new Indexes[MAX_MESSAGES];
stock FreeIndexes()
{
    for(new i =0; i< MAX_MESSAGES; i++)
    {
        Indexes[i] = 0;
    }
}
stock getIndex()
{
    new Index = random(MAX_MESSAGES-1), j;
    for(new i =0; i< MAX_MESSAGES; i++)
    {
        if(Indexes[i] != Index)
        {
            if(Indexes[i] == 0)
            {
                Indexes[i] = Index;
                break;
            }
             j = Index;
        }
    }
    return j;
} 


FreeIndses() is called on starting of the command.


I cant use the normal for loop for the only reason that i want to access random elements within the array, not crescent or decreasing, thus i made the function getIndex which checks if the index hasnt been used already, if not return it & store it into the indexes array... until its full...

I couldnt think of another way... i know i am using alot of loops... but is there any other way?
Response is kinda 4-5 seconds.

[NaS]

Quote:

Originally Posted by iLearner [...] I cant use the normal for loop for the only reason that i want to access random elements within the array, not crescent or decreasing, thus i made the function getIndex which checks if the index hasnt been used already, if not return it & store it into the indexes array... until its full... I couldnt think of another way... i know i am using alot of loops... but is there any other way? Response is kinda 4-5 seconds.

I don't see the reason for accessing them randomly, but you can just shuffle the Index Array. After all this is the same as you do now, but you can do it before executing the main code and save time for the actual calculation. That removes at least one loop from within the main loop.

Also, can you lower the string length? 1000 does not sound reasonable at all. If that's the length of strings you use we don't need to continue trying to make it faster, because that's just how slow it would be!

Otherwise there is not much you can do.
It would be way easier to optimize if you checked for exact matches. That allows for hashing and based on that, other ways to optimize.

I'd suggest you use (my)SQL. PAWN is just bad for matching strings at this scale.

Just think about how many checks are involved. It compares character for character, byte for byte and each byte is read from a global, multi-dimensional array. Even getting one byte involves further calculations because of the structure. Sounds neglible but consdering you have a few million characters, it's not. That's basically the worst case scenario.

[iLearner]

Quote:

I'd suggest you use (my)SQL. PAWN is just bad for matching strings at this scale.

Could you elaborate a bit?

[NaS]

Quote:

Originally Posted by iLearner Could you elaborate a bit?

From CompuPhase's web page:

Quote:

Unlike many languages, pawn is not intended to write complete full-scale applications in. pawn's purpose is to script the functionality provided by an application or by a device.

PAWN is a quite minimalistic language which is meant for small (embedded) systems.
It's bascially designed to coordinate hardware and provide the natives and functionalities of the host program to an exchangable script. Just like it's implemented in SAMP, the server does the actual work (natives, syncing players, network..) and PAWN is used to script it.

What you are trying to do is basically not what it was made for.
PAWN is really fast as such, and perfect for what it was made. But going through a few hundred thousands of cells and then passing them to the server for further checks just takes its time.

(My)SQL should be what you are looking for. You can dynamically add/remove as many data sets as you like, query for matches within milliseconds and all this without the need to recompile a 30MB amx file.

I think you can speed up the code you wrote, BUT not to a degree it wouldn't hang your server notably!

[Lordzy]

The more data you add in, the more slower it goes. I suggest you to consider what NaS has stated, you can use database management system to fetch your result in few milli-seconds which is ALOT faster than to fetch a string from a 3D array in PAWN. Also, if you're ever building something such as what you're doing, you should have a look over packed strings.

[iLearner]

Suppose i have data stored in MySQL / SQL as following:

ID	Question	Answer0	Answer1	Answer2	Answer3
0	Hello	Hey there!	Sup buddy	Yo	How are you
1	...	....	....	....

How can i do something like, go through all of them, and check (i do with strfind) if the parts of given text as input matches somewhere in those records, if yes, select 1 of the answers randomly.

Any idea?

[NaS]

Quote:

Originally Posted by iLearner Suppose i have data stored in MySQL / SQL as following: How can i do something like, go through all of them, and check (i do with strfind) if the parts of given text as input matches somewhere in those records, if yes, select 1 of the answers randomly. Any idea?

You don't even need strfind or anything like it.

Code:

"SELECT * FROM table WHERE Answer0 LIKE '%%searchterm%%' OR Answer1 LIKE '%%searchterm%%' (...)"

Will return all rows (including all fields for all rows) that have a match anywhere in the text (searchterm).
You can then just pick one of the fields if one or more rows have been fetched.

AFAIK this only works for MySQL, SQL requires a bit more than that.
If the searchterm is input from a user, you may want to escape it to avoid injection.

To ignore case sensitivity replace each '%%searchterm%%' with LOWER('%%searchterm%%').

PS: If you were going to use strfind, you would've eliminated the point of using MySQL for this. It would be even slower!

[iLearner]

Hell it freezes the server until its doing the thing... which is now 2-3 seconds.

And no, the messages aint long 1000, the longest one is 180, and most of them are 30-50... i changed the size from 1000 to 180... but it freezes the server for few seconds and sometimes freezes it at all.

Any other way to obtain the thing? using any other thing ? (eg calculations etc on another server / web server) any thing at all?