[kaisersouse]

"I don't think its because of the mods" tells me that you have mods, so I'd try removing them all and see if it helps. If you feel the need to add them back, add them one-by-one and test for 30 - 45 minutes each time.

Also, does this happen on any server or just certain server(s)?