Do you think he hates me?

Unfortunately with work there isn't always personal issues. Some people are so caught up with what they are doing and don't necessarily see their underlings as people who matter so they have little to do with them and that is par for the course - you're not on his level, why should he be your friend? What he sees is someone who is employed to do a job that involves assisting him as needed, not someone to share jokes with. Someone senior to you probably won't apologise to you for similar reasons, they are senior to you and don't have to (at least in their opinion). Since he has been irritable since that point I would guess that he doesn't hate you and probably even thinks that you have done a good job, and now he will have to get someone new to do that job. Since you're unhappy then leaving was probably a good idea, as your only other option would have been to talk with him about it. Having a stressed secretary would not be helpful to him and he probably wasn't even aware that was the case.