Delete form responses

I have a form posted to a shared location. I want to be able to delete responses once they are no longer relevant. When I try to delete them from the response file, and re-open it, they show up again. I know I can export the relevant responses to another response archive, but I still have to wade through all the old responses to find the new ones? Or I am doing something wrong:)