Watch out, there may be a few problems here. PLEASE backup everything
first.

Your code  is actually deleting two unnamed sheets as well as Sheet1 and
Sheet2 - look closely and you'll see why.

Here is code which would achieve it for you:

For Each sh in Sheets
If sh.Name <> "Summary" Then sh.Delete
Next sh

But another caution. Is your summary  sheet actually lifting values from
the other sheets. If so, delete  them and you'll get all sorts of errors
when the source sheets go missing. You might want to do a copy-paste
special-values on Summary before running the code.