Just an opinion, but I would opt to change the pulley, as it has a small wrap @ that diameter and might slip under heavy grass conditions.
With that said, your smaller pulley @ 3.5" mathematically should use a 1-1/2" shorter belt than a 4-1/2" pulley would use. ( approx)