I recently bought a 1865 Second Empire in need of a ton of work. As an easy task I decided to remove some wallpaper and paint a few rooms as an oasis away from bigger project, but then I discovered many of my plaster walls aren't plaster. One room has one beaver board wall, another room has two, a mantle has some panels of it, it seems to pop up everywhere.

What do I do with it? A plasterer told me he can't tape and blend it with the plaster walls because it will just move. Does this mean I need to remove it and replace it with drywall? I can't put this off for the future as we would like to get the house insulated soon, and I have a feeling it would be a bad idea to replace wallboard after having insulation blown in...