Our bamboo in neighbor's yard.

We have bamboo that has been growing for over 20 years. We attempted to remove the bamboo two years ago. Since our neighbor did nothing to stop it from growing in their yard, it was more invasive for them than for us. We received no monetary compensation for the amount we did remove, nor did they help with clearing it out. We spent about $2000 for equipment rental and laborers to cut it back... in both yards with our neighbor's approval. However, the bamboo is growing back. Now we have a voice message from our neighbor telling us we have to remove it from their yard, or they will sue us. We are retired on fixed income and don't feel an obligation to remove again in their yard. Are we legally responsible?