Suppose you want to give away a sum of money as charity, and your family will suffer because of it, can you call this charity? I would only call it foolishness. What is your duty as a householder? When you married and accepted the responsibilities of a family existence, you accepted to fulfill certain duties by the family. These duties are totally obligatory. Therefore, if your gift is going to make the family suffer, then it is not a gift at all, it is not charity. You are really robbing your own family.