Not scary if you plan well, I am not scared, because I did not foolishly buy "lottery tickets" that required 20 years pay out, I have no loans, the only loan I have is car loan, which I can use to generate money by driving Uber or just sell it.
If really need to live in Johor, just rent out...