About halfway through the movie, Quint comments that the shark is "either very smart or very dumb." I think it's the former. After Hooper is submerged in the cage, the shark is clever enough to lure his attention in one direction before circling back and surprising him from behind. Because he's startled, Hooper drops the spear with which he had planned to give the shark a lethal injection.
Rather than outsmarting Quint, the shark merely overpowers him. The shark's strength is greater than what Quint has seen before, so he's able to withstand Quint's strategy of harpooning barrels into him and using their buoyancy to force him to the surface. Near the end of the movie, the shark essentially leaps onto the boat, causing it to sink and Quint to slide straight into his mouth.
The shark is able to counter and evade the plans that Hooper and Quint have, and it's almost because of Brody's lack of qualifications that he's able to succeed. Throughout the movie, he's ill at ease around and on the water, and his last-minute plan involves frantically shooting at the compressed air tank in the shark's mouth. There's even an added degree of randomness in his shots because his glasses were knocked off earlier and he doesn't see clearly.
If Brody were more comfortable on the water or had a thought-out plan, the shark would have had some idea of what to expect from him and how to avoid it, just as he did with Hooper and Quint. It's Brody's unpredictability, spurred by his discomfort and desperation, that leads to his success in killing the shark.