When an in/out field has both interp and grad, especially AtPoints, we can do the interp part for both interp and grad at the same time.