A lot of immigrants in America end up developing massive folders of documents related to their visas, green card, and finally citizenship. It is important to understand that till the day you take the oath of citizenship and have the certificate of naturalization in your hands, you can never have too many documents. In other words, preserve every single document till then. In fact, many immigrants have found that preserving boarding passes and airline tickets for overseas trips came quite handy. You have to be the best judge to decide what to save, but if in doubt, just save it. Keep all your paperwork organized either by type of visa or by year and you will be prepared for the next stage.
What should you do once you are a US citizen? For almost every citizen, there is no reason to save any document other than the certificate of naturalization, which is the only document you will ever need again to prove that you are an American. Needless to add that if you have changed your name legally at the time of naturalization, this is another important document. With these two documents, if you need a passport, you can get it. No authority should ask you for any other document. The old documents can be shredded, though, many immigrants scan digitally or preserve important documents like approval of green card and other visas, just in case (some of these documents might come handy in case of issues related to property ownership, divorce, etc.). In the worst case scenario, you can get a copy of every single paper submitted to the USCIS by paying a small fee, though, in reality almost all naturalized citizens are not expected to interact with the agency, since it deals primarily with aliens.
What to do with alien documents? This should be decided on a case-by-case basis and there is no one good answer. If you are a dual citizen, you will need to decide what to do about the other citizenship. You must follow the laws and guidelines of the other country you are a national of. Even if your native country strips you of your citizenship, you should formally renounce the citizenship of that country and save the paperwork. Other countries insist that you will always be treated as their citizen no matter what. In that unfortunate situation that you cannot decide your own fate, better save all the relevant paperwork.