Here is some additional information:
The temperature and pressure relief valve is a:
WATTS blue ribbon model No.100XL

The valve only leaks when we use hot water or the water needs to be heated. When the water heater is sitting idle (not making any noise) the leaking doesn't happen.

I am not sure if I have any type of backflow prevention device at my house, but it would seem to make sense if I did.