let me explain it this way:
The friction that keeps internal wire from moving in it's sheath is insufficient.
the pull from the governor spring and the vibration is returning it toward the idle position.
This is very common, just increase the friction by tightening the bolt at the base of the throttle lever on the dash just a bit, you will notice after you do that the hand pressure required to move the throttle, will increase slightly.
You just need a little more tightness as no need to over do it to a point the throttle is difficult to move.
Clear as mud?