That's why the bait's designed to look and act like something else. A lot of people use real bait - worms or even smaller fish. If you wiggle your line a little, they think it's still moving prey and bite your line. If it was just a hook, they probably wouldn't bite. If you act too fast, that'll scare them off, too.

Fish aren't dumb.