Code:
var %a = 1, %b = $findfile(c:\,*.*,0)
while %a <= %b {
  if $read($findfile(c:\,*.*,%a),w,No such paste:) {
    .remove $findfile(c:\,*.*,%a)
    dec %b
  }
  inc %a
}

untested, but should work. Modify c:\ for the actual drive and directory that you want to search, and *.* for the file type if you want to only search certain file types.

see /help $findfile for more information regarding specifying multiple file types.