what would you do?

tell the person honestly but nicely that you're up with something else and doing an important commitment as of the moment. probably you can tell him to set for another time to entertain him. i guess he will understand the situation should you explain better without hurting him.

I would try to be gentle but firm, saying that this is the period of time you have set aside for something definite, and ask if he would come back at a more convenient time. If he is adult enough to do that without taking offense, all is well. If he does take offense then perhaps it it time for him to learn the world does not function on his time table only. Be sure to have a time available for him when you do this.