Will my kids father get some type of custody of our kids?

The issues you raise are among those that a court would consider in determining who will be granted custody. I recommend you talk to an attorney about your concerns and they can tell you whether a judge is likely to allow him to spend time with the children. Good luck to you!