I am a permanent resident through derivative asylum. Change of name after marriage.

I've been married to a principal asylee, then we got divorced. Its been 3 years since the card was issued. How long should I wait before I can apply for naturalization? Did divorce effect anything? Or is it 5 years since card was issued regardless of divorce? I am getting married again, and I would like to take my future husbands last name. He is a green card holder too, but through the marriage. If we get married, will it effect my status? If I decide to take his last name, will it hurt my status since I am not marrying an asylee? What should I do? Thank you)