I'm a freshman ? why are you saying it to me ?

The word "freshman" dates from the mid-1500s. It originally meant novice or newcomer and by the end of the 1500s it was used to mean someone in his/her first year at a university. Sometimes words are so old, they make more sense if you look at their etymology (history) to learn more about them.