In this case you need to make sure what is the FPS of your original video. If the frame rate is 5.64 as you mentioned, Debut is going to recognize it, on the Encoder try to use the H264 and change the Bitrate for a better quality. In this case using better Frame Rate you would have better image, remember that usually the basic is 24FPS, getting better FPS the image is going to be better and not choppy. Verify this information with the original video, also try to change the Encoder Format that you are using so you will get better results.