I am going to recap the passport rules because I think things got a little fuzzy here...and maybe some tibits were accidentally incorrect.
If you have sole custody:
You can get a passport without the other parent's permission, and you can travel with the child internationally without the other parent's permission. However, you need to travel with a copy of your custody orders.
If you do not have sole custody (by court order):
1) The child's birth certificate does not list another parent...you are good to go, but MUST carry the BC with you while traveling.
2) The other parent does not have to be present at the passport office, they can sign an affidavit agreeing that a passport be issued to the child.
3) If the other parent is awol or refusing to sign, you can submit an affidavit asking for an exception to the rule requiring the other parent to agree, giving a full explanation of why you cannot get the other parent's signature/agreement and presenting any evidence that you have. You may or may not be approved for the exception. One good reason that works: Other parent is no where to be found. However, the state department does check outside sources to determine if the parent is truly likely to be MIA.....social security records, and things like that.
4) If the other parent is not MIA, but refuses, you can take it to court to ask a judge to give permission for the travel and to order that you are allowed to get a passport for the child without the other parent's signature. You either get the judge to force the parent to sign the affidavit giving permission while in court, or you attach the court order to your affidavit asking for an exception. You also keep a copy of that order with you when you travel.
Most of the time, unless the other parent presents credible evidence that you are at risk of not returning to the US with the child, the judge will make orders allowing the travel and allowing a passport to be obtained.