Is it our responsibility to remove the fences built prior to our ownership that fall into the neighbor's property line?

We purchased our house in 2012. Our backyard, fully fenced in, is adjacent to the uphill neighbor. The fences were built prior to our ownership. The uphill neighbor has no access to the land at the lower level where our yard is. They have their own fences built at a higher ground and ours are at the lower level. We both had a survey done and the results agreed. My questions are: 1. Are we required to remove the fences that were built before our ownership? We agree with the survey and do not have problem if they want to remove the fences themselves. Can they sue us if we don't remove the fences/move the shed for 1 ft.? 2. Can we claim adverse possession on the land (it's very small compared to the whole yard) that the neighbor has no access to? It's been at least ten years that the owners of our house used the yard openly and exclusively.