Let me remake it here.
Download the build from http://files.star-made.org/build/
extract to a folder on your desktop called starmade_server.
Double click on the file called StarMade-dedicated-server-macosx.command that should start the server.
Find the connecting ip by going to terminal, type in ifconfig and look for your lan ip like 10.0.0.xx or 192.168.xx.xx then your brother should connect using something like 10.0.0.15:4242
Edit: Tutorial was made http://www.youtube.com/watch?v=2mDAynOgTiY