Sounds like water hammer which is a condition caused when there is no air space in the pipe to cushion the jolt of the water being turned off. If your house is more than 30 years old, it may or may not have hammer arrestors in place, which is just a length of pipe above the shut off valves at each fixture location.
If your home is less than 30 years old, then it likely has them and they've become water logged over time. Simply turn off the water main and hope a hose bib outside, then go in and turn on both the hot and cold water at each fixture for a couple seconds. This will drain the water out of the arrestor and all should be well again when you turn the water main back on.
If you have an older home and your plumbing doesn't have arrestors, there are aftermarket systems available. TOH has showcased these a couple times. I cannot offer any advice with them because I've neither seen or used one personally.
I suffer from CDO ... Its like OCD, but in alphabetical order, LIKE IT SHOULD BE!!!