I am still fairly new to video editing and have only done a couple of things using the polygon mask, but I have run into this same problem.
It is not immediately obvious that if you delete a point on the mask, that point is deleted for all keyframes not just the one you are currently working on. I wasted a lot of time figuring out what was happening until I worked it out. The simple way around this is to never delete a point from a mask. Instead move a point that you don't want so that it is at the same position as another point.
It can be a real pain if you have a complex mask shape that you need to animate over a period of time.
I am now much more cautious about adding extra points when not necessary. I found it is very easy to accidentally add points without meaning to, when trying to click and drag an existing point for example.
I haven't tried this but I have just thought that another way around the problem might be to split your clip at the keyframe position. Then you can safely delete points from the second clip without it affecting the first. This might not help if you have key frames with a different shape at every frame, but might be useful if you have some time between key frames and interpolating.