(UPDATE 2 - 2/14/2016) Coding an external ship editor

[Cheapskate]

So, I've been thinking of coding my own external blueprint editor. I think I have a pretty good concept for the program's design. It's basically a multi-layer paint program with every pixel representing a block. There's also a 3D view so you can preview your ship before it's built, a speed/weapon strength/power/shield calculator, and a ship-segment copy/paste and any-angle rotator. There's a lot of other concepts I have that are still in the planning phase, these are just the things I'm sure I can do. Sounds great, right? Well, there is just one problem.
...
I have NO idea how to export blueprints. I've looked at the official wiki documentation and even tried to dissect blueprint files myself, but no dice. If someone could help me make heads or tails of the blueprint file format, I would be eternally grateful and reward them richly with advanced, easy-to-use blueprint editors... And also my eternal love.


UPDATE 2 - 2/14/2016

Good news! Apparently all of the extra data in the header I was worried about... doesn't actually do anything. I was just able to copy the extra data from another blueprint and have my program append it to the end of the files it creates. So far, it works. I've been able to inject an "artificial" header [one generated entirely by my program] into a variety of blueprints and so far they've worked a dream. My editor has also been able to log into the header the appropriate blocks and their numbers. So far, everything is looking up! Let's hope it stays this way. Right now, the next most important thing to do is make my program write the SMD2 files. This involves adding in a basic image converter and ship editor. This will by no means be ready for public release, but it's a start. If this works... that's pretty much all the hard stuff out of the way. The rest is just GUI, adding editor features, etc. So far, everything is looking pretty optimistic!

Spoiler: UPDATE 1 - 2/10/2016

Hello! I'm just going to get right to the point here and say that this project is in a bit of a pickle. Where we stand right now, I have the basic import/export functions of the editor finished. I can, basically, manipulate .SMEDIT and blueprint file. This for the foreseeable future is the hardest part. However, there is a snag. For reasons I don't know, the StarMade blueprint files are WAY bigger than they're supposed to be. More than twice as large. A blueprint with just it's core should be 42 bytes big. My test file I got from StarMade for study, however, is upwards of 100 bytes large. By and large this means there's over 50 bytes of unknown data I can't understand or account for. Now, I'm not sure if I'm just being an idiot or this is a simple problem, but as it stands I can't continue the project without understanding what that data is. Does anyone have any suggestions or insights?

[PS, for file sizes I'm going off File format - StarMade Wiki and SMTools/Blueprint at master · StarMade/SMTools · GitHub . The latter of which is three years out of date.Obviously, things have PROBABLY changed. I'll try to contact the guy who added those files and see if he can help me out.]

 

[Rixxan]

All I know is that 7-zip can open exported blueprings (.sment files) with ease.

 

[nightrune]

So, I've been thinking of coding my own external blueprint editor. I think I have a pretty good concept for the program's design. It's basically a multi-layer paint program with every pixel representing a block. There's also a 3D view so you can preview your ship before it's built, a speed/weapon strength/power/shield calculator, and a ship-segment copy/paste and any-angle rotator. There's a lot of other concepts I have that are still in the planning phase, these are just the things I'm sure I can do. Sounds great, right? Well, there is just one problem.
...
I have NO idea how to export blueprints. I've looked at the official wiki documentation and even tried to dissect blueprint files myself, but no dice. If someone could help me make heads or tails of the blueprint file format, I would be eternally grateful and reward them richly with advanced, easy-to-use blueprint editors... And also my eternal love.

That code is around, any chance you'll be using Unity? You should check with Megacrafter127

 

[Reilly Reese]

New SMEdit maybe?

 

[Rixxan]

New SMEdit maybe?

Oh I hope so... We need a new smedit.

 

[ResaRestart]

.

wait so you want to know how to Export (Blueprint) .sment file from the game or the way to open that file?

 

[Cheapskate]

That code is around, any chance you'll be using Unity? You should check with Megacrafter127

Nah, but I want to learn Unity anyways. Besides, all I need is the format of the Blueprint. Even if I'm unfamiliar with the coding structure used in Unity, I could figure it out with a little bit of research.

Oh I hope so... We need a new smedit.

That's the idea! My program initially won't be able to convert 3D models to ship hulls... but this program will have a LOT more functionality than SMedit. It will be versatile enough to add gigantic hull segments and also perform fine-tuned edits. I also hope to implement functionality that will allow the user to add connected logic [and also simulate logic before adding it in], add weapon systems connected to the appropriate barrels/cannons, all that good stuff. I also want to add a template mode, so complex logic doors, turrets, and weapon systems can easily be copy-pasted. Oh yeah, turrets. Did I mention I also have easy mass-docking planned? If you have a ton of places for turrets, you can dock a turret-template to as many rails as you please with the press of a button. I have BIG PLANS.... but I can't do any of them without knowing the format of blueprints.
[DOUBLEPOST=1454384300,1454383817][/DOUBLEPOST]

wait so you want to know how to Export (Blueprint) .sment file from the game or the way to open that file?

Sorry for the double post, but basically I need know how the game understands the an .sment file or how it understands the files in a blueprint folder [they're basically the same, just so you know. .sment is basically just a zipped blueprint folder]. I need to know how the file is set up, and what parts of each file mean what. Honestly, if someone could write some kind of DLL that would convert a string [such as "Grey Armor 3:x y:5 z:7 orientation:0 slab:1"] into blueprint file data, that would be amazing... but I would still need to know how to understand the data in the files.

 

[NeonSturm]

Honestly, if someone could write some kind of DLL that would convert a string [such as "Grey Armor 3:x y:5 z:7 orientation:0 slab:1"] into blueprint file data, that would be amazing... but I would still need to know how to understand the data in the files.

That would be nice to have – plugins!

Are you going to use the same format as SM uses? 10 bit ID, 1 bit logic, … + meta-data for specific blocks?

I think smEdit uses an orthogonal view to show the 3D model, which is OK.
But if you get logic, any renderer, rails, docking, display-blocks, etc all working, do you not have an "open-source StarMade"?
Ofcourse not a good real perspective optimisation and no planets but still a large part of it. Especially when some others get ideas to add combat simulators, etc

 

[Megacrafter127]

Honestly, if someone could write some kind of DLL that would convert a string [such as "Grey Armor 3:x y:5 z:7 orientation:0 slab:1"] into blueprint file data, that would be amazing...

Well, since I'm learning C/C++ now anyways, maybe I will try to write a shared library for that, including organizing the data from the files themselves.

 

[Cheapskate]

That would be nice to have – plugins!

Are you going to use the same format as SM uses? 10 bit ID, 1 bit logic, … + meta-data for specific blocks?

I think smEdit uses an orthogonal view to show the 3D model, which is OK.
But if you get logic, any renderer, rails, docking, display-blocks, etc all working, do you not have an "open-source StarMade"?
Ofcourse not a good real perspective optimisation and no planets but still a large part of it. Especially when some others get ideas to add combat simulators, etc

For the template and save files, I'll use whatever format provides the most versatility and functionality!

EDIT: sorry, misread your post. Thought you said the same format SMedit uses. This entire thread is actually asking for help in figuring out the StarMade format XD If you know how the file is structured, could you help me out and tell me about it?

On the second note, not really. You see, my program will not support any kind of AI for ships and such. You also won't be able to "fly" anywhere with it, it will just provide the math and statistics to figure out if your weapon systems won't be able to fire due to not having enough power, etc. It'll also do the math and tell you how long your shields last, how quickly they recharge, etc. Long and short, there is a GIANT chunk of code missing that would make this a "StarMade clone"... and to be honest, that's the last thing I would want to make. There's already a talented team making StarMade, why would I waste my time trying to make a game that's already being made better? xD
Though, for this reason, I'm probably not going to release the source code of this editor. The last thing I want to do is damage the game I'm writing a program to support.

Well, since I'm learning C/C++ now anyways, maybe I will try to write a shared library for that, including organizing the data from the files themselves.

Do you know how Blueprint files are structured? If you could let me in on what is apparently the biggest secret in all of StarMade, I would appreciate it.

 

[Megacrafter127]

Do you know how Blueprint files are structured? If you could let me in on what is apparently the biggest secret in all of StarMade, I would appreciate it.

Well, the structure of the files is layed out very well in a wiki, the only downside is, that said wiki is not the official wiki, and the official wiki ignores byte offsets, byte order and unknown bytes, which effectively negates any use for a programmer.
http://www.starmadewiki.com/wiki/File_format
The rest of that wiki is pretty outdated, but this file format description is still very accurate, at least regarding the smd2 format(which contains the data you are mostly looking for).

 

[Cheapskate]

Well, the structure of the files is layed out very well in a wiki, the only downside is, that said wiki is not the official wiki, and the official wiki ignores byte offsets, byte order and unknown bytes, which effectively negates any use for a programmer.
File format - StarMade Wiki
The rest of that wiki is pretty outdated, but this file format description is still very accurate, at least regarding the smd2 format(which contains the data you are mostly looking for).

Aah, thank you! This is what I was looking for. I found the file formats on another wiki but had no luck getting them to work. I'll take a look at this wiki and see what I can do.

PS. I'll keep y'all updated on program progress!

 

[Megacrafter127]

Aah, thank you! This is what I was looking for. I found the file formats on another wiki but had no luck getting them to work. I'll take a look at this wiki and see what I can do.

PS. I'll keep y'all updated on program progress!

If they don't work, try omitting or adding a few unknown bytes here and there. Unless the mechanic which's data is stored was completely changed in an update, the general formatting should be very similar.

 

[Ghent96]

A new SMEDIT might help some, but isn't really needed at this point. The original was useful for importing .obj, coloring them, copy-pasting large parts of the ship.... but now we can do all that very easily in-game. There's some other tools out there now for converting .obj or other model files to voxel format, and then into .smd. The only function that remains for SMEDIT would be a better import algorithm... or, if you just wanted to make or edit ships outside the actual game, but that's part of the point of playing.

Don't get me wrong, I've used and enjoyed the old SMEDIT a lot, and still use it for some things. If you truly want to do a new version of it, then be bold and bring some new things into it. Also, please be dedicated and stick with it. Block IDs may change and totally break your tool, and you have to change it. Stick with it.

 

[Cheapskate]

A new SMEDIT might help some, but isn't really needed at this point. The original was useful for importing .obj, coloring them, copy-pasting large parts of the ship.... but now we can do all that very easily in-game. There's some other tools out there now for converting .obj or other model files to voxel format, and then into .smd. The only function that remains for SMEDIT would be a better import algorithm... or, if you just wanted to make or edit ships outside the actual game, but that's part of the point of playing.

Don't get me wrong, I've used and enjoyed the old SMEDIT a lot, and still use it for some things. If you truly want to do a new version of it, then be bold and bring some new things into it. Also, please be dedicated and stick with it. Block IDs may change and totally break your tool, and you have to change it. Stick with it.

Ohhhh trust me. What I have planned is basically ALL new features... Come to think of it, since SMedit was primarily a 3D model converter, I guess this isn't a new version of SMedit. This program will do a heck of a lot more than SMedit EXCEPT import models. That feature be something quite far down the pipeline.

Though, as a singleplayer for life, ship building has never really been the most important part of the game for me. FLYING giant ships and murdering the poor defenseless pirates have always been more appealing to me. It has always taken me days, weeks, or in some cases months to build ships with the size, interiors, and details that I want... and there's always the fear of a pirate invasion coming in and blowing away my had work, my own ships I spawned around me damaging my new baby, etc. By the time I'm approaching something close to completion of one of my big projects, I have 5-8 others on the drawing board that I'm anxious to get to... I'm impatient. v. v

This program will basically provide an environment to build highly detailed ships very quickly, very effectively, and make it very easy to perform large-scale edits, fixes, etc... among other things. I will keep up with this program as best as I can with work and whatnot. At the current time I need to write the code to save ship files. The success or failure of this project kind of hinges on that... and as the coder I use has no built in hex functionality, this'll be fun :'D

You mention that there are now some other tools for converting model files to voxel format and saving them into .SMD. If there are any that provide a source code, can you link them to me? If I can see how they do it, it might speed the coding process along for me.

 

[alterintel]

Ohhhh trust me. What I have planned is basically ALL new features... Come to think of it, since SMedit was primarily a 3D model converter, I guess this isn't a new version of SMedit. This program will do a heck of a lot more than SMedit EXCEPT import models. That feature be something quite far down the pipeline.

Though, as a singleplayer for life, ship building has never really been the most important part of the game for me. FLYING giant ships and murdering the poor defenseless pirates have always been more appealing to me. It has always taken me days, weeks, or in some cases months to build ships with the size, interiors, and details that I want... and there's always the fear of a pirate invasion coming in and blowing away my had work, my own ships I spawned around me damaging my new baby, etc. By the time I'm approaching something close to completion of one of my big projects, I have 5-8 others on the drawing board that I'm anxious to get to... I'm impatient. v. v

This program will basically provide an environment to build highly detailed ships very quickly, very effectively, and make it very easy to perform large-scale edits, fixes, etc... among other things. I will keep up with this program as best as I can with work and whatnot. At the current time I need to write the code to save ship files. The success or failure of this project kind of hinges on that... and as the coder I use has no built in hex functionality, this'll be fun :'D

You mention that there are now some other tools for converting model files to voxel format and saving them into .SMD. If there are any that provide a source code, can you link them to me? If I can see how they do it, it might speed the coding process along for me.

I litterally can't wait for this to happen. I've tried putting something together myself, but the binary nature of the sment files throws me off. If there was a sment to txt converter that could make a text file that any basic program could read, that would be freakin awesome.

 

[Cheapskate]

I litterally can't wait for this to happen. I've tried putting something together myself, but the binary nature of the sment files throws me off. If there was a sment to txt converter that could make a text file that any basic program could read, that would be freakin awesome.

I KNOW, RIGHT? Earlier in this thread I had wished for something like this. I, too, am having trouble with the exporting of .smd files. That's basically what this entire project hinges on. If I can export, the rest of the code is amazingly simple [mostly because I've already written this code dozens of times in the past. Doing it again isn't a big deal]. I also realized while writing this post that my editor would also allow players to very easily move the cores of pre-existing ships [again, if I can get a handle on the hex format of ship files], rotate cores, etc.

 

[Nauvran]

If you're really going to make this things then please make sure to make it better optimized than SMedit.
That program was horrible at loading anything past the typical 500 long ships.

And good luck with it. Hope we can get to move our cores someday soon again :p

 

[Cheapskate]

If you're really going to make this things then please make sure to make it better optimized than SMedit.
That program was horrible at loading anything past the typical 500 long ships.

And good luck with it. Hope we can get to move our cores someday soon again :p

Of course! This program's main purpose is the easy and detailed construction of mega-ships! It's hard to make any kind of estimate right now, but I want to make this editor able to support ships over 10,000 blocks long [ambitious, am I right?]. I guess the only upper-limit is really just what your computer can handle, but by and large it should be optimized to load and edit pretty sizable crafts.

...Also in case you didn't already guess, this will be able to edit space stations and asteroids as well, but not on the initial release.

 

[navy3000]

will i beable to get a image of a ship and past it into the editor? then turn it into what i want?