You gotta do what you gotta do....
And, you don't have to give them the 2 weeks. It is considered the polite an professional thing to do, but it'* not required. And don't worry about "Well if anyone ever calls...." If you put them as a job reference, you could be in trouble. If they're just part of your job history, all they can legally say is "Yes, he worked here from xxxx to xxxx". Nothing about how or why you left can legally be mentioned.