I'm actually scared of my brother after what just happened.....?

Tell your parents , they can help you . This is a very dangerous possition you are in because maybe your brother wants to do something with you ( sex ) . i suggest tell your dad or prefarably mother first and they will suggest what you should do.