I'll answer any call if my phone doesn't say Scam Likely. If its something I don't care about, I just hang up.
3 seconds of time is a fair trade to me, to not risk missing what may be an important call. What if someone I know is needs help, and had to call from a different phone? What if my wife was in an accident or something? Ect.
http://tinyurl.com/b95dwtg