What do you mean by "radius of invulnerbility?" If it means ships in that area are invincible, that means meaningful combat can't take place there.
Radius of invulnerability would only affect a home base and entities docked to it. (what I meant was that if your home base extends past the radius, the part extending beyond the invulnerability radius would be subject to attack. If a docked ship extends past this radius, the part extending would also be subject to attack as well. So if you docked your ship, but half of it was sticking out past the radius, you could very well return only to find half of your ship has been blown off.
Sectors instead of km's could work as well. I'm not sure I like the idea of counting mass though. Anybody could just start building with asteroids parts to make their base massive.
I do like this idea of claimed sectors instead of a km radius though. Each sector claimed would have a players name assigned to it. So you could only have as many sectors as you have players? So when you join a faction, your own personal sector is assigned somewhere along the perimeter of the faction territory? or maybe each person chooses their own sector, and faction territory doesn't necessarily need to be adjacent to each other.
If you're in a faction you get faction bonuses, such as tax, salvage and mining bonuses, and invulnerability at the faction home base. If you're a loner, you only get invulnerability at your own home base in your sector and no other bonuses. (invulnerability only applies to home bases, and ships docked to them.)
Previously I suggested that a faction bank and linked storage block would have to be set up to receive tax salvage and mining bonuses. Perhaps instead the bonuses should go to each play individually? So if you join a faction you get bonuses from all faction territory.