Should be able to buff down any ridges using emery(sp?) cloth. I wouldn't be overly concerned about something like that unless it was subject to considerable friction, like a bearing race, or was in contact with a seal (not gasket).
That'll need to be smoothed up. A flapper wheel on a drill will work well, or experiment with the tools you have on hand. Just smooth it up without creating flat spots or other profile changes it. I'd initially shy away from your grinder as that'll take some finesse you wouldn't want to find out you didn't have, after the fact.