I've thought about the flag idea as well but not so recently. I'll barf the thoughts here.
"You cannot attack this entity." warnings are ridiculous IMO.
Three types of ship; military, civilian and 'pirate'. Distinguishable by different radar pings, military and civilian transponder can be set/ switched at a shipyard (delay dependent on the change). You can attack whatever you want. The 'pirate' flag is set by attacking (or damaging?) a civilian ship. 'Pirate' ships show up on the galactic map, ship name, pilot name, faction and all, visible to everyone even unloaded like a fleet and might agro their own bases or temp loose faction docking access to discourage running home and docking. I like the 'pirate ships cannot dock' now. Ok, going further, the pirate flag cannot ever be removed from a ship. That ship cannot use faction facilities anymore ever and must be deconstructed which without a shipyard or a cargo ship would be near impossible for large ships. But maybe the game's Pirate faction will not attack you. Or, maybe there's a real long cool-down to switch back from pirate somehow.
Military ships need to have differences from civilian ships, bonuses to encourage their use and limits as well. Pirate is just a flag.
Possibilities;
Military ships have superior thrust-per-block (and/or other system bonuses) but due to the strain have a maintenance clock (based on total blocks or mass or allocated space) that winds down in service and back up (maybe at a different rate) while docked (to base, carrier or supply ship). If the clock runs out while the ships is deployed the bonuses expire and/or the ship could suffer various % system nerfs similar to taking systems damage but never such that it can't get home. Needs fleet AI to actually be able to do stuff to prevent micro-management overflow though. AI crew could be used to modify the clock's rate (perform maintenance). A military ship that turns pirate won't loose it's buffs right away but can no longer re-set it's maintenance clock due to inability to use it's faction facilities. Perhaps being aligned with the game's Pirate faction would allow docking at in-game Pirate stations for a similar function though. Another option could be that you could sell the ship to a pirate station and it just becomes an AI controlled pirate ship.
A New Block (yeah I know), maintenance locker; link it to cargo space, this acts as your visual maintenance clock. It 'fills' with nothing when docked and it's volume determines the max military deployment time for the ship. Could be transferrable to/from other ships through docking like regular cargo?
Docking a military ship to a civilian one; the civilian ship is temp-flagged as military but not given military bonuses, must leave the same sector as the military ship for it's flag to revert to civilian.
No warheads on civilian ships. Adding a warhead to a civilian ship sets it to pirate automatically (this needs a warning prompt I'd guess). Anything else goes as self-defense.
Civilian ships as spies or asteroid stealing; If your faction has claim to a sector can you attack a civilian in your own sector without being flagged as a pirate? Maybe a sector-owning civilian would have to get switched to military but not having a maintenance storage would prevent them from gaining any buffs? Not so great since as most core sectors get claimed being a civilian ship offers less and less security.
The first issue I see so far is accidental civilian casualties during military fleet engagement. My first thought was to have the pirate flag set on actual damage instead of shield hit but not all ships run shields and this would force civilian ships under attack from other civilian ships to wait until their shields are breached before counter-attacking to prevent being flagged themselves unless it could be coded some other way.
Second issue; civilian cloaked ship used to force-flag a victim as a pirate. Perhaps no cloakers on civilian ships but I don't really like that idea.
Third issue would be how something like this would interact with NPC factions if it needs to at all.
Certainly not a polished idea but some points here may inspire a better one.
Another problem with this would be a fleet with civilian shield supply beam support ships. They won't flag themselves by using support beams and cannot be attacked by the opposing force without causing pirate flags.