As long as the chimney has bee terminated properly, filling in the hole to the room is no problem.

Install some backer material. If you will be using plaster, I'd go with as large a piece of plywood as you can fit through the hole, follow that up with some wire lath, then layer in the plaster. If you're going to fill the hole with drywall, then a couple of sticks across the hole will suffice, cut your patch and shim stock to flush with wall surface, then tape and mud in with drywall compound.

Let me reiterate, as long as the chimney has been terminated properly, filling the hole is not a problem.