When I'm coding at my peak, I'm removing more code than I'm adding. No, I'm not talking about refactoring your code to be more terse or efficient. I'm talking about writing code to play around with ideas or flush out an example API, then getting rid of all of it if it doesn't pan out.
As I eluded to earlier, on the last application I worked on I found it much easier to start over from scratch multiple times rather than try to go back and fix the code. Sometimes the code was this bad because it had been written usually under bad assumptions or a bad understanding of the problem, and thus was fundamentally wrong. Other times I just found that even though the code was right and worked correctly, it just didn't feel right or was not as easy to work with as I would like. Regardless of how or why the code was wrong, it had to go, and I didn't think twice about deleting it.
I didn't necessarily scrap the entire application every time, but I would say between 10 and 15 times I scrapped large parts of the application, sometimes the entire application. Writing code is the easy part, but solving the problem in the right way is hard. You don't need solve every problem on your first try, and many times you won't, so feel free to start over and keep iterating until you solve the problem in a way that is acceptable to you.
Often, I would sketch out ideas on paper or try to figure things out in my head before trying to do the actual code. While in a lot of cases this was helpful, sometimes the solution I came up just didn't work out as well in code as it had on paper. The only way to truly know how something will translate to code is by writing the code itself. This is similar to one of the primary benefits of test driven development, though I find this process useful even before writing tests.
Ultimately, the best reason to delete your code isn't about how pretty it is, how efficient it is, or how well commented it is. The best reason to delete your code is that you may not get another chance to rewrite it. You may find yourself stuck in a position where you are confined by the rules of your original code. As long as you are satisfied that you can live with those constraints, then sign off on that code and move on. Your code is your scratch pad, tear off sheets liberally until you are ready to paint.