The answer to this question depends on the reason for which the application for naturalization is denied. The USCIS has a process that is followed once a petition for citizenship is denied. In the simplest scenario, if the application is denied because you were not eligible at that time or did not provide enough documentation (e.g. if you have failed to file taxes or owe money to the IRS), then, it is understood that you will be eligible at a later date, and in that case, your permanent resident status is still valid.
On the other hand, if the USCIS believes that you cannot be naturalized because you pose a security threat to the United States or its allies, or that you provided false information during the green card process, or you committed a deportable offense, or that your green card was granted in error, the USCIS will take the necessary legal steps to let you know about them (in extreme cases, it could be in the form of immediate arrest at the time of interview -- think terrorism and national security cases). With the help of an attorney, you maybe able to fight your removal from the United States.