I was disappointed to be excluded from the process after a phone screen that never happened. It's okay to say no without a conversation, but pretending we spoke is poor form and makes it look like you're doing more "work" than you're actually doing.