When two objects are gravitationally attracted, the forces
on both of them are equal. As the objects move straight
toward each other, then, just as we'd expect, the object
with the greater mass has the smaller acceleration and
moves a smaller distance by the time they collide, while
the object with a smaller mass has the greater acceleration
and moves a greater distance by the time they collide.
The Earth's mass is roughly approximately something like 70,000,000,000,000,000,000,000 times as much as YOUR mass,
so the difference in acceleration and distance moved before
the collision is really substantial.