We have a neighborhood cat around here, a stray that no one wants to turn in, but no one claims. He's an older cat, not above begging, but not dependent on anyone's handouts. (He sometimes hangs out around my bird feeders, does that tell you anything?).
He's doesn't mind a good petting session now and then, but he won't come around unless he's been called. What a cat, huh?
The one thing he does which is dismaying, is that he uses any dug up area for his personal litter box. That means one has to be careful with vegetable gardening, as cat feces contain dangerous pathogens which can be transferred to the vegetables.
One way we've discovered to keep him out of our gardens is to put down a piece of chicken wire or mesh fencing after preparing the soil, but before planting. If we put seeds or plants in the holes of the mesh, they do fine.
Cats won't be able to dig a hole without hitting a wire, and they won't be able to cover their scat without hitting one either, so they'll quit trying to use the area.
The one drawback to this method is that the wire makes it almost impossible to hoe, so you'll have to weed it all by hand. For small gardens, it's not so much of a problem, but if your garden is large or you have time or physical restraints, weeds can get out of control in a hurry.
In that case, I'd recommend a commercial product created for the purpose of keeping cats and other animals away. Our neighborhood cat responds well to the commercial products, but I don't, because of the cost. They are cheaper than buying all of your produce at the grocery, though!