[wire] stone has been developing highly interactive graphical applications using markup
languages like XAML and HTML for some time. However, due to performance needs, we sometimes
develop using DirectX. We developed the proof of concept depicted in this video under
Windows 8 as a DirectX app to test the limits of DirectX.
What we discovered in the process was enlightening. And we've got a few insights
that can help developers skill up on DirectX intricacies a bit faster.
Insight #1: Forget everything you know about graphics
That's the first, important thing we learned. Concepts like cameras,
lights, and materials are not abstracted as well as you will find in WPF, for example.
Also, keep in mind that the fundamental performance strategy in hardware
acceleration is multiprocessing—that will help you accept and
embrace
all of the DirectX craziness. Modern graphics chips have dozens, even hundreds, of processing cores. That is true even for
laptop graphics controllers. Your mind needs to switch from linear to massive parallel
processing thinking. This is vital to remember. DirectX relies heavily on shaders.
And shaders are tiny programs that are meant to run in parallel.
Insight #2: It's a 3D world
Everything from the video card perspective is done in 3D. EVERYTHING! The smallest 3D model element is a 3D triangle.
A 3D triangle is defined by supplying three x/y/z vertices. Objects in the 3D environment are created by combining
3D triangles. For example, a flat rectangular surface is made of two triangles. A sphere can be modeled using
triangles and the more triangles you use, the closer it will look like a sphere.
Insight #3: There are no pixels without pixel shaders
So, you've defined all the triangles. You still won't see any pixels.
You need pixel shaders to put pixels on the screen. A pixel shader is a tiny
program that gets parameters (you can design and specify) and returns a
pixel. The pixel shader defines what pixel to display at different locations.
The GPU takes care of managing its hundreds of cores that run your pixel shader.
So, the simplest way to draw in DirectX is to write a pixel shader that will
run in a massive parallel environment for each pixel in a 3D triangle.
Going beyond the basics
Fabulous insights you say. Thanks for those. But what about how to draw images,
rotate 3D objects, define diffuse/specular/emissive materials, and manage
camera positions? What about that stuff? Well let's get into that.
Drawing images
In order to draw images (Jpeg, PNG, etc.), the pixel shader needs to
return the image pixel for a given coordinate in a 3D triangle. A common
technique is to associate each triangle vertex 3D coordinate to a 2D
coordinate in the image. Then you interpolate pixels in between. For those who
worked with the WPF XAML 3D, that’s where texture coordinates came in. Somewhere in
WPF there is a pixel shader that receives the XAML texture coordinate data and defines
the pixels on the screen.
Lights, materials, action!
WPF and other frameworks provide an abstraction to lights and materials. You can have one or
more lights like ambient, spotlight, etc. Surfaces could react differently to light
depending on their material types (diffusive, emissive, specular, etc.).
Well, DirectX does not offer this level of abstraction. Pixel shaders are used to handle
lights and materials. The shader receives information about light positions and material
characteristics at a given 3D point. Using very sophisticated formulas it generates the
corresponding pixel for every portion of the surface. The good news is that there is
tons of code out there that implements lights and materials for you.
Where are the cameras?
One of the things that many markup developers look for when doing DirectX is camera position control.
Well, the truth is THERE ARE NO CAMERAS! The way you accomplish the illusion of moving a camera in
DirectX is by moving the 3D objects instead. But how you move thousands of triangles and vertices? That's
where vertex shaders come in. Just like pixel shaders, they are small programs designed to run in a
massive parallel computing environment. Luckily shaders have support for matrix operations
that are essential mathematical tools for 3D operations.
Windows 8 apps: mixing DirectX and XAML
Now that we have scared you with this "simple" explanation of DirectX complexities,
here is the good news: Under Windows 8 there is a way to mix DirectX and XAML.
That way you can have maximum graphics performance with DirectX and leverage the
layout and abstraction of WinRT and XAML. That's content for an entirely different
post, but to get an understanding of how it works, take a look at this DirectX 3D
shooting game sample over at the Windows Dev Center:
http://code.msdn.microsoft.com/windowsapps/Metro-style-DirectX-18f98448
Potential applications
- Large visual experiences that require the highest level of graphics performance
- Complex 3D visualizations