No good answers as lots of points mentioned are right. If I were you, pull off the boards on your side where the nest is, get the nest out, and put them back.

Maybe the handymen at ProperSee have a better idea? They came up with some out of the box solutions for me, and could look at it with me live.