I had to grind off the ball joint castle nut and was a little messy. Slightly gouged the castle nut mating surface in the front, but 95% of the contact area is still there. Attaching some photos to give you an idea. I know some may insist on replacement but is it really a huge deal if the nut can stay torqued? The interior of the hole where the ball joint stud is inserted is untouched.
It should be just fine. The ball joint is held by tensioning the bolt when the nut is torqued. A slightly gouged surface on the seating of the not will not affect its holding power. Just make sure that you replace the cotter pin.
You'll be fine shouldn't be a problem done a few although with the prii. I have extra front hub assemblies just for this reason because they are aluminum and they will easily get damaged by cutting and using tools on them but that shouldn't hurt a thing that shouldn't be able to come undone or any of that should be pretty uneventful only one who will know is you
