Rejected Limit number of blocks per ship to 500k, including the turrets.

    Joined
    Jul 31, 2013
    Messages
    387
    Reaction score
    87
    • Purchased!
    • Community Content - Bronze 1
    • Legacy Citizen 4
    I'm not saying you have not had this problem, but I have run a server for two years and have never had the same problem. I limit ship mass so that players cannot lag out their opponents in battle and to make it less likely to have unbeatable mega titans. That being said, I have loaded some of the largest ships and stations possible with zero problems on the server.

    Strictly technically speaking, it is simply not a server lag problem unless its substandard. The server loads many players with total block counts in the hundreds of millions, not including planets and stations and all nearby sectors loading. The only two possible explanations are client issues or a poor server setup.
     
    Joined
    May 7, 2015
    Messages
    65
    Reaction score
    5
    It may be server configuration or need for game optimization on 1 big ship size. But, imagine for example two factions goes to war, since currently the best ship is the biguer in this game ( no strategy at all), and all members decide to take their biguer ships, what do you think will happen when 20 players x 20 players starts to load all their 5m blocks ships ? Do a test, do you think it will not crash the server in any case ?
     

    Keptick

    Building masochist
    Joined
    Sep 26, 2013
    Messages
    4,062
    Reaction score
    1,841
    • Councillor 2 Gold
    • Railman Gold
    • Thinking Positive Gold
    A server setting to limit ship mass? Sure. As default? No.
     
    Joined
    May 2, 2015
    Messages
    73
    Reaction score
    22
    First I will explain why you get lag with large ships on a server. Then maybe you will understand what the real issue is, and its not ship sizes.

    First you have to understand how those blocks are delivered. An object when brought into an area is broken up into 'chunks' essentially regions of blocks that need to be updated and the system sends you that chunk as a data packet group (I won't cover the networking aspects). We as server admins can control the size of the block count in the chunk, and the server controls the shape of blocks sent and block types. You can see chunks in action if you make a REALLY big door with blast door blocks. When you open it you will see strips of it opening at a time, not all at once. Those are the chunks in action. I can't remember if its 128 or 256 blocks in a chunk by default.

    So two ways we approach this. Smaller chunks are more efficient for network bandwidth use. The server can queue up more of them and it doesn't waste space on small updates. On the other hand more of these individual parts means that you get delivered the information a bit slower and dropped packets on the network side means those chunks need to be resent. All which slows down the processor of the server. On the other hand if you increase the chunk size the network can become more congested from the larger traffic, but it lightens the load on the server.

    Since RedShift was mentioned they do have 3 Million block ships over there and they do fairly well. Koderz and SgtPepper have mostly figured out the balance of Chunks, blocks and server build that works for their community. There are tons of ways to combat the issues from the lag and most of it doesn't require reducing the data count. Data compression, reducing spike behavior, identifying laggy actions and more can be performed for the sake of bandwidth and processor performance. That is why this is best left to admins to control. Different hardware types, network builds and more come into effect here, plus population types and behaviors. For example, an RP server typically has very strict build rules, so ships are naturally reduced. PVP servers have high ship turnover which also can reduce ship size. Plus if we get more of the tools we are looking for (I want to set more diminishing returns on mechanics for example, with XML scripts to change the data on load, to naturally curb supermassives) we as admins can meet our community needs for different builds.

    Plus expect to see more optimizing from the Devs. They aren't done yet and they will be able to perform some optimizations on the code to handle the amount of blocks. Plus preloading regions, better spam controls and more. This isn't a new problem with games, its been going on a long time.
     
    • Like
    Reactions: Lidren

    Jaaskinal

    ¯\_(ツ)_/¯
    Joined
    Jan 19, 2014
    Messages
    1,377
    Reaction score
    646
    • Legacy Citizen 4
    • Wired for Logic Gold
    • Thinking Positive
    Hang on, that thing is never that heavy.
    775M Mass, would be 775.000.000 on the "Mass" info in build mode.
    Or 7.750.000.000 in Blocks count.

    So i think your mixing something up.
    I was confused by a bug in the catalog, this is what the catalog says;

    This is what buildmode says;

    I hadn't touched it in a while :p
     

    Valiant70

    That crazy cyborg
    Joined
    Oct 27, 2013
    Messages
    2,189
    Reaction score
    1,167
    • Thinking Positive
    • Purchased!
    • Legacy Citizen 4
    Well , I am talking about the experience I had playing online, like, we know when a player with 5m block ship is online and using the ship when it starts to lag, even if we are not in the same system, and sometimes, not often, the server crashes...
    I know my machine can process this kind of ships when I am playing single, because I liked to download big ships to look at, so it may be a server cpu lack, but its not an client problem. I sorry, but if all players just start to build biguer ships there will be no server on line for playing. Admins already figure out this, and that is because almost all servers have admin limits of 1m.
    Judging by what Lidren said, you're probably describing connection lag, meaning your server's ISP is weak. (Lidren owns a server, so he would know.)
     
    Joined
    Jul 31, 2013
    Messages
    387
    Reaction score
    87
    • Purchased!
    • Community Content - Bronze 1
    • Legacy Citizen 4
    As for combat, that is an entirely different aspect of the issue than you described in the OP. Of course a 20 on 20 slugfest will lag or crash the server. Heck, 40 people connected at all can lag a server regardless of what they are doing. All I am saying is that a single stationary or even active mega-ship will not in itself lag the server.

    *facepalm* Thanks for the vote of confidence Val, though its experience that guides me. The fact that I own a server alone does not mean much in the long run.
     
    Joined
    May 7, 2015
    Messages
    65
    Reaction score
    5
    Well, so you are affirming that multiplayer in this game is not going to work right ? Since you said this game cant handle 40 peaple.
    Also, about your technical explanation, yep, it may be network bottleneck, but I still saw it happens on the best ping servers.
    I dont beleave the internet average speed will grow in the next years, so this game is doom not to be a multiplayer anyway if you say is true. Just to make my idea clear, what the games tries to sell is a universe with hundreds (maybe thousands) of players competing in wars for territory etc, and it dosent see to be possible in this game, not with this limits.
     

    Jaaskinal

    ¯\_(ツ)_/¯
    Joined
    Jan 19, 2014
    Messages
    1,377
    Reaction score
    646
    • Legacy Citizen 4
    • Wired for Logic Gold
    • Thinking Positive
    Well, so you are affirming that multiplayer in this game is not going to work right ? Since you said this game cant handle 40 peaple.
    Also, about your technical explanation, yep, it may be network bottleneck, but I still saw it happens on the best ping servers.
    I dont beleave the internet average speed will grow in the next years, so this game is doom not to be a multiplayer anyway if you say is true. Just to make my idea clear, what the games tries to sell is a universe with hundreds (maybe thousands) of players competing in wars for territory etc, and it dosent see to be possible in this game, not with this limits.
    Game no worky because alpha-y.
     

    Valiant70

    That crazy cyborg
    Joined
    Oct 27, 2013
    Messages
    2,189
    Reaction score
    1,167
    • Thinking Positive
    • Purchased!
    • Legacy Citizen 4
    As for combat, that is an entirely different aspect of the issue than you described in the OP. Of course a 20 on 20 slugfest will lag or crash the server. Heck, 40 people connected at all can lag a server regardless of what they are doing. All I am saying is that a single stationary or even active mega-ship will not in itself lag the server.

    *facepalm* Thanks for the vote of confidence Val, though its experience that guides me. The fact that I own a server alone does not mean much in the long run.
    Your experience is what I was referring to, although my choice of words was imprecise.
    [DOUBLEPOST=1432929914,1432929717][/DOUBLEPOST]
    Well, so you are affirming that multiplayer in this game is not going to work right ? Since you said this game cant handle 40 peaple.
    Also, about your technical explanation, yep, it may be network bottleneck, but I still saw it happens on the best ping servers.
    I dont beleave the internet average speed will grow in the next years, so this game is doom not to be a multiplayer anyway if you say is true. Just to make my idea clear, what the games tries to sell is a universe with hundreds (maybe thousands) of players competing in wars for territory etc, and it dosent see to be possible in this game, not with this limits.
    This just means we need server hosting services with very high bandwidth connections in and out of the server itself. i.e. servers will need to be hosted on fiberoptic networks to support hundreds of people. In the future, 40 will probably be doable on a normal connection. However, it is unlikely you will ever see battles of the scale that is often seen in games like EVE online.
     
    • Like
    Reactions: Lidren
    Joined
    Jul 31, 2013
    Messages
    387
    Reaction score
    87
    • Purchased!
    • Community Content - Bronze 1
    • Legacy Citizen 4
    Its already about as able as that other block-voxel game to handle players, and the optimization is a process, not a static variable. It handles just fine until people start doing crazy stuff.

    In short, no, that is not at all what I said, or meant.
     
    Joined
    May 2, 2015
    Messages
    73
    Reaction score
    22
    4096(16x16x16) block per chunk, and up to 4096 chunks per region. any amount of regions per entity.
    I think that is logical strorage size. So essentially how it is save as a state save for the region. Not the same as traffic based updates. Could you imagine how a mining array would wreck a server if it was that big. Mining causes a shower of block chunk updates as each beam updates its block damage every 0.025 seconds. (Why they decided humans needed salvage beams to update 40 times a second is beyond me, I think changing this to a much more reasonable number would do a lot for Starmade salvaging and server performance. Plus you maybe can hardcap salvage speeds on the server, need to test that out)

    Code:
    NT_BLOCKUPDATE_QUEUE_SIZE = 127 //how many blocks are sent per update. Huge placements will shot faster, but it will consume more bandwidth and is subject to spamming players
    CHUNK_REQUEST_THREAD_POOL_SIZE_TOTAL = 25 //Thead pool size for chunk requests (from disk and generated)
    CHUNK_REQUEST_THREAD_POOL_SIZE_CPU = 3 //Available threads of total for CPU generation. WARNING: too high can cause cpu spikes. About the amount of available cores minus one is best
    I think this is more what I was talking about. NT_BLOCKUPDATE_QUEUE_SIZE = 127. I have toyed with this and it does allow you to drastically change the performance of the server. Sizes beyond 511 crushed my testbed server though on a local network. Mostly the processor couldn't keep up with the large amount of data moving.

    I haven't messed with the Thread Pool size and Thread Pool Size CPU so I can't tell you the effects of those. On my crappy test server I did have to set it to 2 or 1 CPU (I have a Pentium Core Duo for a test server). I suspect these could ruin the Starmade experience with very little change to them. When I upscale my server to a processor capable of more threads I will try higher settings, the recommendations seem to make sense from an architecture point of view.

    **Edit and Update**
    Just checked my test server. Its at 350 blockupdate queue size, with 2 threads. It still lags a bit with 8 people on the server, but its just for testing code and game changes in a non-production environment. (Pent Core Duo AKA Garbage POS w/ 2GB ram being fed 25Mbps bidirectionally). Often have 1-3M block ships moving about it.
     
    Last edited:
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    I think that is logical strorage size. So essentially how it is save as a state save for the region. Not the same as traffic based updates.
    No, regions are only interesting during auto-saves, otherwise, only the segments consisting of 16x16x16 blocks are interesting.
     
    Joined
    May 7, 2015
    Messages
    65
    Reaction score
    5
    This just means we need server hosting services with very high bandwidth connections in and out of the server itself. i.e. servers will need to be hosted on fiberoptic networks to support hundreds of people. In the future, 40 will probably be doable on a normal connection. However, it is unlikely you will ever see battles of the scale that is often seen in games like EVE online.
    I agree with that. I don't understand why there is no official server yet. To test performance for real and prove what I am talking here, it would be required an official server with nice bandwidth.
    It would help the development, since it would be possible to measure the limits, of bandwidth, of ship limit size, of qtd of players, etc.
     
    Joined
    May 2, 2015
    Messages
    73
    Reaction score
    22
    I don't think I would be happy at my work if my boss said "Go optimize the data handling over a network interface for project X, BTW, we are updating features all the time so its a moving target. Good luck!" I think I would give him the crazy eye and try to wait 10 seconds before I point out the tremendous waste of time he inflicted on me and the waste of resources that was about to happen.

    That is a very specific science to optimizing network use for an application. Part of that science is the application can't keep making large changes beyond bug fixes. You would spend a week getting a baseline, planning changes, identifying bottlenecks, and then toss out the work to just start over the next week when they add in a new big feature.

    TL;DR - Optimization comes close to last in a large project like a game. Be much more patient.