Do we have to go to meditation 1st before court?mediator said both party dont have to follow the schedule agreement is it true?

in the divorce agreement it states that "the father will have the kids at 5pm every tuesday and put them on the bus and 5pm to noon every other weekend" but they want us to take the kids more. so it was finaly verbaly agreed that we will take the kids half the time which is every wednesday-saturday. and this often end up more specially before" we have been doing this for almost 3 years now. Now we want to go back to the original agreement which is every tuesday and every other weekend.but they wont do it.weve been trying this for couple of months. in the divorce agreement it also says any dis agreement will have to go to mediation first before court.do we have to follow that? since the original agreement is been agreed legaly during the time of their divorce. but their just not following