What are the challenges in using 3D Visualisation to make these applications user-friendlier?

I am a firm believer that by using 3D visualisation one can display and depict more information on screen at any one time in a manner that is a lot easier to understand than 2D visualisation. Instead of looking for suitable metaphors to symbolise objects, why not create the object in 3D in a 3D environment that someone can understand and relate to the environment the program is supposed to represent and control.

“Interfaces that adhere strictly to a metaphor based on the physical world are in danger of becoming simulations of reality instead of useful analogies”

Instead of creating a symbolic application to try and metaphorically depict a situation or environment, why not create the environment that accurately represents itself. By using architectural drawings and the like to build a 3D environment of a said building and then, using the applications I mentioned earlier in conjunction to maintain the site. The applications I mentioned earlier normally give some sort of numerical or alphabetical reference to locate problems that occur which have then to be compared to architectural drawings to pin point, create the site in a 3D with a virtual representation of the network running through it that works with the program. Instead of having to cross reference grid points and reference points, a problem will be visible on screen, in 3D, in the exact virtually represented point of a structure.

I think this application will reduce time needed in locating problems in a network.