What kind of building are you looking to simulate. Is it a skyscraper of metal and glass, a wooden house, a multi-story brick school? It's going to take a number of layers, and you're going to need to think of the different materials involved and how they'll fail to build character into the design. How does the collaps collapse affect the environment around it, is that even something you should consider? There are a lot of angles you can approach from.
The first Sound Design Challenge I ran was a video of a controlled explosive building demolition. Check out some of the entries to see how people approached it, and maybe it will give you some ideas. All of the entries can be found here.