Depends on the context.
a captain has a responsibility to their crew and passengers, so if theres still survivors on board they should be doing everything in their power to save them before evacuating. if everyone is evacuated I dont see an issue with the captain following
in the military I think theres a culture where even if the crew has evacuated the captain will remain, Im not 100% sure on this but I did read of some cases of this on reddit