My brother applied for student visa while his i-130 still pending. will there be any problem with f-1 visa approval?

Yes, the I-130 can be an issue.

Generally F1 students need to prove non-immigrant intent. This means if a green card is pending, the F1 should not be issued. In reality though, the consulate will issue the F1 is there is a compelling reason- admission to a top notch university, applicant's academic record, other family reasons being a few.

Yes. If he has an I-130 pending he may be denied the student visa. The I-130 shows that he has immigrant intent. The F visa is supposed to be for persons with non immigrant intent. This is called duel intent.