Is our fence our neighbors fence too?

If the fence is three feet inside your property line, it is probably solely owned by you. In that case, you are free to deny your neighbor permission to hang ladders on the fence, paint it, or do anything else with it. In fact, you are within your rights to deny him the right to come onto any portion of your property, including the three feet and land on "his side" of the fence. But (there's always a "but"), what is the basis for your knowledge of the location of the property line? Do you have a survey, and are there survey monuments in the ground? If so, you're probably operating on pretty solid knowledge. If there is no survey, however, there may be some disagreement between you and your neighbor as to the location of the property line. Even if the property line is located where you believe it to be, is it possible that your neighbor has gained title through adverse possession - that's a legal term referring to someone obtaining title to another person's property by possessing it himself over a long period of time (typically 10 years). A third issue to consider would be whether you and your neighbor have any agreements for the shared use of the fences. If the situation is as simple and clear cut as described in the question you posted, it would appear that you are the sole owner of the fence and your neighbor would have no rights to it. Regardless, your safest course would be to meet with an attorney who has experience with property boundary line disputes. He or she can help you to understand exactly what your property rights are, and how you can best protect them.