Can I stay in the US while swiching from H1B cap exempt to regular H1B? If not, when do I have to leave the US?

You are legally required to depart the USA the very next day when cease working on H-1B status. You are not allowed to "remain" in the US until start new H-1B employment on Oct. 1st. You are required to depart within a "reasonable time", needed to "liquidate" your affairs, i.e. no later than 10 days after cessation of work.

No, if you are in H-1B cap-exempt status, and the cap-exempt job ends in May, then even if a cap-subject petition by a new employer is approved, it will have an October 1 start date, and even under the new grace period established by the High-Skilled Worker rule, you will not be able to remain in the US between last day of the 10-day grace period after the end of the validity of your cap-exempt job and the October 1 start of your cap-subject job. See https://www.kwvisalaw.com/2016/11/18/over-a-decade-a-half-later-ac21-regulations/

No. You must depart from the US once the current job ends.