Anyone else suffering extreme server instability?

[aRottenKomquat]

It seems the last couple updates have really hurt the dedicated server's stability. I have a VM with 8 allocated CPU cores and 12 GB allocated RAM (11G allocated to the JVM) and the server crashes all the time, generally 2 - 3 times daily if it's active. Though I operate a public server, generally I'm the only one playing so it's usually not under a lot of load. The crashes seem random. Sometimes I come back to the game after not playing for a few days and it's down. Other times it's only been up for a few minutes and mining an asteroid causes it to die. A few days ago something really bad happened and it just wouldn't come back online and I had to wipe all the StarMade files and world and redownload to get it going again.

Any other server owners experiencing this, or have any suggestions to improve it? I know the game is in alpha but it's starting to get frustrating. It would be great if the next patch or two had some stability fixes for multiplayer.

I only made a few tweaks to my server's CFG file. I set the asteroid generation core count to 7 (suggested # of cores - 1), asteroid respawn to 1 hour, and disabled pirates taking blueprints. Maybe there are some other settings in there I should play with?

 

[nightrune]

What error are you seeing in the logs?

 

[Valck]

While there seem to be quite a few database issues (ie. inconsistencies, duplicate or missing entries) with the 0.199.253 builds, I haven't had a single server crash in months, running on Linux, using 4 out of 8 cores, 4 out of 16GiB, scheduled daily restarts, load about the same as yours. Moderate changes to server.cfg and about every customXYZ.xml.

 

[aRottenKomquat]

Okay, "crash" is the wrong word, it's really more unresponsive. The console usually looks normal and the server shows up in the server list. But connection requests always time out. Also when this happens the console does not respond to commands and I have to do "CTRL-C" to kill it instead of a proper shutdown.

I'm using Ubuntu Server 16.04, CLI-only install, though I doubt that matters.

 

[Valck]

Okay, "crash" is the wrong word, it's really more unresponsive. The console usually looks normal and the server shows up in the server list. But connection requests always time out.

Do you have asteroid physics enabled? If so, /sector_info a few asteroid sectors to see if they have duplicate asteroids that generate massive collision lag. The asteroids in question generally have different uids, but identical seeds

Spoiler: Example

/sector_info 5 -7 9
[...]
1476787909 [SEND][SERVERMESSAGE] DatabaseEntry [uid=ENTITY_FLOATINGROCK_1476620882691_5_-7_9_3, sectorPos=(5, -7, 9), type=3, seed=-2900456152527641911, lastModifier=, spawner=, realName=Asteroid (rich in Sintyr, Threns), touched=false, faction=0, pos=(-345.98294, -135.41582, 290.84308), minPos=(-4, -5, -3), maxPos=(3, 4, 2), creatorID=6]
1476787909 [SEND][SERVERMESSAGE] DatabaseEntry [uid=ENTITY_FLOATINGROCK_1476646293111_5_-7_9_3, sectorPos=(5, -7, 9), type=3, seed=-2900456152527641911, lastModifier=, spawner=, realName=Asteroid (rich in Sintyr, Threns), touched=false, faction=0, pos=(-639.5636, -112.946465, 422.1042), minPos=(-4, -5, -3), maxPos=(3, 4, 2), creatorID=6]
[...]

You may want to disable asteroid physics and see if that helps.

 

[aRottenKomquat]

I will try that this evening. I know asteroid physics is enabled, I believe it is by default and I never changed it.

 

[BDLS]

Getting any log spam? I've seen poor performance from basic logic contraptions like spinning radar dishes and simple logic loops.

Just curious, are you running ECC RAM?

 

[aRottenKomquat]

I doubt there's any logic spam, the players on my server typically don't do anything fancy. And no ECC RAM on the current system, it's a consumer-grade system with an AMD FX-8350 and 32 GB DDR3.

 

[kurosawa]

You are not alone, but it could be a coincident unless a few more starts to report server crashes

 

[nightrune]

Please feel free to post logs and I can give any information I have. The others have pointed out some really good stuff to try. You could actually try allocating only 8G instead of 11. Also not running bare metal actually has quite an impact on server performance since on the server side Starmade is CPU bound mostly with collisions.

 

[aRottenKomquat]

I think disabling asteroid physics fixed it.

 

[Valck]

I think disabling asteroid physics fixed it.

If that was indeed what caused your issues, keep an eye out for those asteroids, and possibly on this bug. They may not collide anymore, but they might still multiply.

 

[aRottenKomquat]

I've noticed them multiply before. Right as a sector full of asteroids loads, sometimes one of the asteroids will explode into 3 - 4 identical ones. I haven't noticed any performance problems from it though.

 

[aRottenKomquat]

Just thought I'd check in to definitively say that disabling asteroid physics solved my issue. My server has not been down once since I changed that setting.