Redis cache and message queue plugin for SA:MP
#1

Hi everyone! I've been throwing this idea around my head for a while and decided to start work on it and wondered if anyone else has any use for it or is interested in contributing.

At work, I've been working with message queue based systems for microservice communication within a cluster since it has some advantages over HTTP based RESTful APIs. I was going to start this project last year but tried to finish Pawpy instead (which ultimately failed, it was fundamentally flawed, rest in piece Pawpy).

I realised that a message queue based system would be much more flexible than simply running threaded Python scripts. I was going to use Pawpy for delegating tasks to a better language with more utilities such as IO, resource acquisition from the internet (GeoIP, stats, IRC, discord). It works a little bit like this:

Here's an example using Discord and a server event system:

- Gamemode publishes a message on a channel named "events" to say a new event has started
- A Python app is subscribed to the "events" channel and when it receives a message, it uses the Discord API to send a message to the server's discord channel (or IRC channel if you're oldschool)

The advantage here is that this uses a "fire and forget" method, the gamemode doesn't care about the process so why should it waste time with anything else. The plugin call takes almost no time at all since the Redis code is actually run in a separate thread.

Another example, this time using a remote admin interface:

- Gamemode subscribes to "admin_cmd" channel, binds this to a callback named "OnRemoteCommand"
- Discord server, web interface, mobile app fires a message to an API server
- API server picks up that message using whatever API is necessary
- API server then forwards this message to the "admin_cmd" redis channel to trigger the gamemode code

Think of this as a threaded MySQL plugin on steroids, you're not just doing database calls, you're doing anything you want with a familiar event based mechanism at the core. You can write programs in a multitude of languages that interact with the channels, you can have these programs perform MySQL calls (bypassing the need for a MySQL plugin for SA:MP, just do it in Python!).

I'm going to be lazy and copy the list of possibilities from the old Pawpy topic:

- Sockets - just use Python sockets, Golang sockets, Ruby sockets, etc...
- Mail - thousands of email packages for many languages
- SQL - there are drivers for pretty much all the popular databases (MySQL, Postgres, Mongo, etc)
- GeoIP - maxmind have a Python library for reading their data, no more converting to SQL
- Regex - most languages have a fast regex engine
- CTime - all modern languages have some form of time/date module
- GPS - there are loads of packages for this
- MapAndreas - I've rewritten this in Python so you could thread it's calls to ease server load
- Any maths plugins/libs - as complicated as you can think of, there will be some way to do it in numpy/scipy
- Logging - Logstash, sql, flatfile, complex analysis, possibilities are endless!
- JSON - DJson is outdated and doesn't follow spec, SAMPSON only supports reading, most languages have JSON

And then of course there's a bunch of cool stuff you can do:
- Generate graphs of your player data
- Connect to your website/forum/panel with ease
- You like statistics? Python has statistics coming out the wazoo. Log and track anything.
- Neural networks? Want to write an AI that learns something from players? Go for it!
- OCR? somehow? could probably be useful...? A gamemode where players have to write text by driving around
- Why not just hook a Python call up to your coffee machine and allow players to trigger a brew?
- Okay I'm out of ideas

I'd love to hear if anyone else would be interested in such a plugin (I'll release it either way since I'm using it for my own Scavenge and Survive project).
Reply
#2

Hmm, What would be the one thing (the "channel" you mention a couple times) binding all these together? the plugin?
How would codes in other languages subscribe?
I'd love to have this so I can hook it up to my server even easier, the idea of having a hub to connect the server to whatever we need is just the thing we need to survive without the official sa-mp updates (Please don't kill me Kye)
Reply
#3

Channels are a feature of the Redis Pub/Sub mechanism. You can have as many channels as you want and they are really just a method of tagging messages. This is built into Redis and will be exposed in my plugin so you can publish a message on a particular channel straight from Pawn and also subscribe to a channel from within Pawn and let other programs do all the hard work.

Redis in other languages is well documented, there's a Redis client in every popular language so you could write an app in your favourite language which interacts with the gamemode via Redis channels.

You can sort of do this via HTTP but it's largely one way (unless you poll from the gamemode via a timer, not ideal) whereas with this plugin you'll be able to "bind" a channel to a callback and have it react almost instantly to a message.

I've designed the first version API now, it'll look something like this:

PHP код:
Redis_Connect(host[], porttimeout) > ctx // connect to redis server
Redis_Close(ctx// disconnect context from redis server
Redis_Command(ctxcommand[], ...) // issue any Redis command (except SUBSCRIBE and PUBLISH)
Redis_Subscribe(ctxchannel[], callback[]) // waits for messages on channel and calls Pawn callback when message is received.
Redis_Publish(ctxchannel[]) // publish a message on channel 
More info here: https://github.com/Southclaws/samp-r...ster/readme.md
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)