Commas and periods always go inside the quotation marks. Exclamation points and question marks go inside the quotation marks when they are a part of the quoted matter. Otherwise, they go outside.
Right: Sergeant Parker gave the following order: “Peel potatoes!”
Right: Schaeffer’s book asks, “How Shall We Then Live?”
Right: What did King mean when he said, “I have a dream”?
Semicolons and colons should be placed outside quotation marks or parentheses. When a passage ending with one of these punctuation marks is quoted, the colon or semicolon is dropped. In running quotations, each new paragraph should begin with open quotation marks.