I have done exactly this in our house. I first called the chimney sweep and he said, just like you wrote, that the old walls were no problem, but where I was to build new ones, he said that I should start framing the wall at least 10 cm from the chimney stack and leave air between the chimney stack and the frame. However, you could connect with wood-based sheet material or other cladding against it as long as it was end grain. This also applied to the connection of any ceiling paneling.