You will not get more speed and lose 30 to 35 lb. of payload. Your C of G will move forward which could be an issue if you have 2 heavy people occupying the front seats.
My 182 does the same thing. It appears that it is a normal situation. It is only critical when the remaining fuel is low. One has to balance the fuel like land by selecting the fullest tank.
