Am I being unreasonable here?

Sorry to hear it. Only one thing to do. Lawyer up. It will cost you big time, but it you want custody or any type of visitation and/or not get stripped of everything paying child support you're going to need someone to do the work and give advise. More and better advice than you will get on the Internet.