Chunk loader / Sector loader

    Joined
    Mar 11, 2015
    Messages
    141
    Reaction score
    39
    • Community Content - Bronze 1
    • Purchased!
    Sector loader: Simply a block that keeps the sector, it's in, loaded.

    In Minecraft some mods are adding this and it's realy usefull.

    Should be limited to one per player or on per faction. (configurable in server.cfg)

    This would keep your factories and logic working, even if no player is around.
    Also the AI would work in the sector.

    I haven't found this suggestion by searching, but I'm pretty sure I'm not the only one thinking about this :)
    My very own intention in suggesting this is, that i would need a possibility to keep my logic working, cause it's calculations need some time.
     
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    Sector loader: Simply a block that keeps the sector, it's in, loaded.

    In Minecraft some mods are adding this and it's realy usefull.

    Should be limited to one per player or on per faction. (configurable in server.cfg)

    This would keep your factories and logic working, even if no player is around.
    Also the AI would work in the sector.

    I haven't found this suggestion by searching, but I'm pretty sure I'm not the only one thinking about this :)
    My very own intention in suggesting this is, that i would need a possibility to keep my logic working, cause it's calculations need some time.
    adding this effect automatically to homebases would work, I think. Ofc it should be disable-able for each homebase.
     
    Joined
    Mar 10, 2015
    Messages
    122
    Reaction score
    50
    • Community Content - Bronze 1
    • Purchased!
    • Legacy Citizen 5
    Why do you need this? What would you use it for?

    Your ability to keep your factories running is a good use case, but most of the time your factories aren't even running. It doesn't really justify the memory usage associated with this. If I slapped one of these blocks on a planet, or worse - if I made a bunch of alts and slapped a whole bunch of these on a whole bunch of planets - I could cause a lot of problems. I'm not of the opinion that exploitability should completely rule out a feature, but this is highly exploitable and you'd need a really good use case to back up getting this included in vanilla.
     
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    Why do you need this? What would you use it for?

    Your ability to keep your factories running is a good use case, but most of the time your factories aren't even running. It doesn't really justify the memory usage associated with this. If I slapped one of these blocks on a planet, or worse - if I made a bunch of alts and slapped a whole bunch of these on a whole bunch of planets - I could cause a lot of problems. I'm not of the opinion that exploitability should completely rule out a feature, but this is highly exploitable and you'd need a really good use case to back up getting this included in vanilla.
    Hence the limitations, also, my factories are permanently running 24/7, unless the sector unloads or I shut the server down.
     
    Joined
    Mar 10, 2015
    Messages
    122
    Reaction score
    50
    • Community Content - Bronze 1
    • Purchased!
    • Legacy Citizen 5
    Those limitations don't help, though. On whitelisted servers, they'd stop trolling but be a limiter to legitimate players; on an open server, perma loading dozens of planets is obscenely easy.

    If your factories are running 24/7, you either aren't using enough enhancers, or you're spending all of your time mining (or both). It's pretty quick to process the raw materials from hours of mining if you're using enough enhancers, and you should never be producing blocks that you aren't going to use in the near future.
     
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    If your factories are running 24/7, you either aren't using enough enhancers, or you're spending all of your time mining (or both). It's pretty quick to process the raw materials from hours of mining if you're using enough enhancers, and you should never be producing blocks that you aren't going to use in the near future.
    I am constantly feeding the factories, yes, BUT what if one doesn't have enough resources to spare on enhancers, because in the end he'll have to scrap the entire factory anyways(station-construction project on a nonbuildserver e.g.)
     
    Joined
    Mar 10, 2015
    Messages
    122
    Reaction score
    50
    • Community Content - Bronze 1
    • Purchased!
    • Legacy Citizen 5
    You can narrow down the scenario as much as you want, but one very specific use case just doesn't justify the feature when the potential for exploitation is large. That's why it's a mod in minecraft - because you choose what your mods are.
     
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    You can narrow down the scenario as much as you want, but one very specific use case just doesn't justify the feature when the potential for exploitation is large. That's why it's a mod in minecraft - because you choose what your mods are.
    In this game, you can disable many features in the config files.
     

    CyberTao

    鬼佬
    Joined
    Nov 10, 2013
    Messages
    2,564
    Reaction score
    641
    • Legacy Citizen 4
    • Railman Gold
    • Thinking Positive
    I know there are a few ways to lag a server (logic, beams, collisions), so my biggest concern is someone purposely using this to constantly lag a server when admins aren't around.

    World anchors have always caused considerable lag in numbers as well, when on servers with 1 man factions it'd be more abundant as well. Probably just me being paranoid though.
     
    Joined
    Mar 10, 2015
    Messages
    122
    Reaction score
    50
    • Community Content - Bronze 1
    • Purchased!
    • Legacy Citizen 5
    I know there are a few ways to lag a server (logic, beams, collisions), so my biggest concern is someone purposely using this to constantly lag a server when admins aren't around.

    World anchors have always caused considerable lag in numbers as well, when on servers with 1 man factions it'd be more abundant as well. Probably just me being paranoid though.
    This is exactly it. Think about the lag problems we already have, and then factor in all the new lag problems we'd get.

    In this game, you can disable many features in the config files.
    This is true, but I shouldn't have to go into my config files to prevent lag trolling.
     
    Joined
    Jul 21, 2013
    Messages
    2,932
    Reaction score
    460
    • Hardware Store
    This is exactly it. Think about the lag problems we already have, and then factor in all the new lag problems we'd get.



    This is true, but I shouldn't have to go into my config files to prevent lag trolling.
    Then what about having them disabled by default in the releases, so one'd have to turn them on for one's server.
     
    Joined
    Mar 11, 2015
    Messages
    141
    Reaction score
    39
    • Community Content - Bronze 1
    • Purchased!
    So why would i need this.

    I have enough factories, that is not my problem. My problem ist, that my logic gets unloaded when I leave the base and after i come back, the logic takes some time until it is loaded again.
    It will not be loaded at once but in the order i placed the templates and that is causing malfunction.

    Also, cause the clock ticks only with one Hertz, complex calculations need realy long. So the probability of malfunction is very high.

    This prevents me from doing what i would like with my station. A complex central System for ressource-counting, hangar-management, on station navigation, ...
    And all paired with accesscontrol.


    Other Ideas for such a block clould be to build an autonomous ship, flying through the galaxy, collecting ressources, ... (present functionality combined with future features like AI Update)
     

    CyberTao

    鬼佬
    Joined
    Nov 10, 2013
    Messages
    2,564
    Reaction score
    641
    • Legacy Citizen 4
    • Railman Gold
    • Thinking Positive
    So why would i need this.

    I have enough factories, that is not my problem. My problem ist, that my logic gets unloaded when I leave the base and after i come back, the logic takes some time until it is loaded again.
    It will not be loaded at once but in the order i placed the templates and that is causing malfunction.

    Also, cause the clock ticks only with one Hertz, complex calculations need realy long. So the probability of malfunction is very high.

    This prevents me from doing what i would like with my station. A complex central System for ressource-counting, hangar-management, on station navigation, ...
    And all paired with accesscontrol.


    Other Ideas for such a block clould be to build an autonomous ship, flying through the galaxy, collecting ressources, ... (present functionality combined with future features like AI Update)
    Could always try to build in a backup reboot system that would shut down the system and restart it. Logic is weak to lag, I assumed failsafes and backup systems were more common.

    Logic itself causes lag, just trying to keep that loaded may result in a few wannabe trolls attempting to overload the server with masses of it. I doubt it'd happen due to optimizations, but there is always possible consequences.
     
    Joined
    Mar 11, 2015
    Messages
    141
    Reaction score
    39
    • Community Content - Bronze 1
    • Purchased!
    Logic itself causes lag, just trying to keep that loaded may result in a few wannabe trolls attempting to overload the server with masses of it.
    Little question: What are masses? And how big is the lag it causes?
    Because in Singleplayer i haven't any problems with the logic, once it has been loaded.
    And i want to migrate some parts of it to a server when I'm finished.
     
    Joined
    Aug 21, 2013
    Messages
    237
    Reaction score
    76
    • Legacy Citizen 2
    • Purchased!
    • Community Content - Bronze 1
    Hm. I can understand the idea, but I don't find it that useful. However, if Astronaut mode expands to include agriculture and alien animals breeding then I could see more of a use.

    My logic systems don't need to run while I'm not on, and my factories have so many enhancers that I only need to run them a few minutes at a time.

    I wonder if it might be more practical to simulate persistence rather than actually keep it persistent. For instance:

    I have a factory that can produce 1 hull every second.
    I turn it on.
    I log out. --> The game saves the logic blocks/factories and stores the time at which that chunk de-spawned.
    I log in 1 hour later. ..> The game reads the time from when the chunk was despawned, and calculates how much time has passed (up to a server-configured maximum). It then multiplies the production of the factory by the number of ticks it was 'active but not loaded' and puts the output in the appropriate slot.
    I check my output, and see 3600 hull (1 for every second I was gone).

    I don't know how the variables for logic work, but I assume that the server can save the state of the block plus which block in the logic system is currently acting so that it can pick up from this spot to simulate persistent logic.

    This means that there is practically no change in the RAM used by 'persistent but empty' sectors, and instead adds a single string per sector of data to read when a chunk loads and a single function onload to update logic/factory/position. The same method can be used to calculate the position of orbiting entities when a system loads/unloads.
     
    Joined
    Mar 11, 2015
    Messages
    141
    Reaction score
    39
    • Community Content - Bronze 1
    • Purchased!
    That would maybe cause extrem long chunkload times, if it is calculated by the server.
    But the calculation of Logic could be outsourced to the local computer. I see no manipulation purpose in logic.
    Factories schould always be calculated by the server. (Storages can output to logic, one should not forget this, that could make trouble)

    Saving all logic-information with timestamps and let a local PC doing the work on chunkload/sectorload would be the perfect solution.
    No Server-lags and always-on logic :)
     
    • Like
    Reactions: Tobie
    Joined
    Mar 10, 2015
    Messages
    122
    Reaction score
    50
    • Community Content - Bronze 1
    • Purchased!
    • Legacy Citizen 5
    Neither persistent-but-unloaded sectors nor client-side logic evaluation are viable options.

    I have a factory that can produce 1 hull every second.
    I turn it on.
    I log out. --> The game saves the logic blocks/factories and stores the time at which that chunk de-spawned.
    I log in 1 hour later. ..> The game reads the time from when the chunk was despawned, and calculates how much time has passed (up to a server-configured maximum). It then multiplies the production of the factory by the number of ticks it was 'active but not loaded' and puts the output in the appropriate slot.
    I check my output, and see 3600 hull (1 for every second I was gone).
    This would only work for a single factory in a closed system. Several problems are raised by this system; of note:
    • What happens if I build a factory system that doesn't have enough power?
    • What happens if my factory is pulling raw materials from a storage container, and I run out of materials?
    • How does the game simulate my factories in the correct order so that factories can correctly pull intermediate materials from other factories?
    There's only one way to solve all of these problems, and it's the laggiest solution available: iteratively simulate each individual factory tick when the sector is re-loaded. The real kicker is that the game currently does these simulations all within one frame, which would mean that even with a relatively reasonable server maximum of an hour, it's pretty easy to build a factory system complicated enough to give everyone a "server not responding: 30 seconds" message every single time you log on. This would lead to most server admins disabling the feature or requiring players to turn off their factories before logging off, which would make the entire feature moot.

    Persistent logic causes the same problem, except worse; imagine you build a 7 segment digital clock and leave it running when you log off. When you log back on, the game has to simulate the entire clock, second by second, for an entire hour of operation, from the 1 second clock pulse to the incrementor to whatever sort of base 2 to base 10 conversion is involved, and finally to the actual number display - 3600 times. "server not responding: 30 seconds" would be what you wish you had.

    Client-side logic evaluation is both impractical and insecure. First of all, the server needs to know the final state of all the logic blocks on your station (and all your ships!) in order to properly function. If you outsource this insane computation to the client, you're effectively letting the client set any logic state they want when they log in. If you throw in factories, then the client can just say that all of the factories are filled with infinite amounts of every material. You can't let that happen, because that's a massive security flaw, so the only reasonable solution is to make the server validate the client's result - and the only way to do that is to make the server do all the work itself, so there's no point in making the client do anything in the first place.

    This is sometimes called "client controls inventory", and it's among the worst ways you can build a multiplayer game. Other favorites are "client controls position", where the client can teleport anywhere it wants.

    The short version of all of this is that the only way to do this properly is to keep the sectors loaded all the time, and as already discussed, that's a giant invitation for lag trolling on a massive scale.
     
    Joined
    Jan 24, 2015
    Messages
    57
    Reaction score
    46
    • Purchased!
    What happens if I build a factory system that doesn't have enough power?
    Add a check for an entities power stability. If the entity is not stable with all it's factories running, do not process any of them.

    What happens if my factory is pulling raw materials from a storage container, and I run out of materials?
    The same thing that happens normally in this situation should suffice.

    How does the game simulate my factories in the correct order so that factories can correctly pull intermediate materials from other factories?
    Iterate through and process each factory connected to the same storage system one at a time, until all of them are out of ingredients. This way the factories that are creating any prerequisite ingredients for other factories will be processed before their dependents are.

    If you throw in factories, then the client can just say that all of the factories are filled with infinite amounts of every material. You can't let that happen, because that's a massive security flaw, so the only reasonable solution is to make the server validate the client's result - and the only way to do that is to make the server do all the work itself, so there's no point in making the client do anything in the first place.

    This is sometimes called "client controls inventory", and it's among the worst ways you can build a multiplayer game. Other favorites are "client controls position", where the client can teleport anywhere it wants.
    I do agree that having the client side handling of cheat sensitive data is less than ideal from a security standpoint but the potential performance benefits for a game that runs its multiplayer on small private servers, like StarMade does, greatly eclipse this issue in my eyes.

    There are also several methods that can be used to validate client handled data that allow for the mitigation of any security risks, while also saving server resources.
    For example:
    Randomly and intermittently auditing clients by comparing their results against the server's, and banning or flagging for admin attention any clients that fail any of these audits.
    There are a number of multiplayer games that have client side handling of cheat sensitive data that very rarely have trouble with hackers. Off the top of my head, and if I recall correctly; Some of the recent BattleField games had client-side hit detection and Planetside 2 has client-side hit detection and maybe position tracking as well.

    Although I am not certain that calculating the output of factories over time is a complicated enough equation that it would need client-side processing to be practical.
     
    Joined
    Aug 21, 2013
    Messages
    237
    Reaction score
    76
    • Legacy Citizen 2
    • Purchased!
    • Community Content - Bronze 1
    • What happens if I build a factory system that doesn't have enough power?
    • What happens if my factory is pulling raw materials from a storage container, and I run out of materials?
    • How does the game simulate my factories in the correct order so that factories can correctly pull intermediate materials from other factories?
    I think you were imagining this as a for-loop that tries to do 3600 calculations in 1 frame. That is not what I was suggesting. Just imagine that the factory systems have a 1-time calculation as if they all had more enhancers than they actually do - when loaded they run 1 tick and pull a % more resources and produce a % more item, then run normally. The % is determined by the amount of time they were on but not loaded.


    1) I should have said that instead of production per tick, the factory output is output per tick while powered. Every 5 seconds the factory uses 1k energy, but it takes 10 seconds to generate 1k energy, so the output of the factory is 5/10 (or 50%) what my above suggestion stated. Thank you for pointing that out.

    2) The time the chunk is unloaded simply adds a multiplier to the way the factory normally runs. It is a one-time pulling of 3600x as many resources (up to a server cap), and if that is more than the available resources we get the same result as if there were more enhancers on a factory than resources available - it caps out at resources it can pull.

    3) The factory is calculated as normal. I don't know how the order of factories is determined, but if the system normally works in a factory tick then it would also work in the OnLoad which is just a normal factory tick calculation with a % multiplier.


    I do have to concede that this would not work for a logic system that loops infinitely - like the clock you mentioned.