Warning: This is just a guess.
I don't think you want to animate the ball but I'm also not sure making it into a physics object is the right approach either.
Instead you probably want the ball to be moved in code. Sort of a very simplified physics that you have entire control over that determines the speed and velocity and direction of the ball and then applies them manually.
You would probably want to choose what strength and speed and angle based on where the ball hits the racket and what kind of shot the player does. Find a good "gravity" feel by decreasing your vertical momentum each frame.
I'm not sure if this is the right approach, but watching videos of games like Mario Tennis it appears that they don't use real physics but something that does its best to imitate them in a predictable way.
↧