Designing an effective and performant HDR implementation for my game's engine was another step that was complicated by the Xbox 360.  As a quick refresher for those who aren't experts on the subject, HDR is most commonly implemented by rendering the scene to a floating-point buffer and then performing a tone-mapping pass to bring the colors back into he visible range.  Floating-point formats (like A16B16G16R16F, AKA HalfVector4) are used because their added precision and floating-point nature allows them to comfortbly store linear RGB values in ranges beyond the [0,1] typically used for shader output to the backbuffer, which is crucial as HDR requires having data with a wide dynamic range.  They're also convenient, as this it allows values to be stored in the same format they're manipulated in the shaders.  Newer GPU's also support full texture filtering and alpha-blending with fp surfaces, which prevents the need for special-case handling of things like non-opaque geometry.  However as with most things, what's convient is not always the best option.  As with my shadows, I once again came up with a list of possible techniques and enumerated their pros and cons:

• Standard HDR, fp16 buffer
  +Very easy to integrate (no special work needed for the shaders)
  +Good precision
  +Support for blending on SM3.0+ PC GPU's
  +Allows for HDR bloom effects
  -Double the bandwidth and storage requirements of R8G8B8A8
  -Weak support for multi-sampling on SM3.0 GPU's (Nvidia NV40 and G70/G71 can't do it)
  -Hardware filtering not available on ATI SM2.0 and SM3.0 GPU's
  -No blending on the Xbox 360
  -Requires double space in framebuffer on the 360, which increases the number of tiles needed
• HDR with tone-mapping applied directly in the pixel shader (Valve-style)
  +Doesn't require output to an HDR format, no floating-point or encoding required
  +Multi-sampling and blending is supported, even on old hardware
  -Can't do HDR bloom, since only an LDR image is availble for post-processing
  -Luminance can't be calculated directly, need to use fancy techniques to estimate it
  -Increases shader complexity and combinations
• HDR using an encoded format
  +Allows for a standard tone-mapping chain
  +Allows for HDR bloom effects
  +Most formats offer a very wide dynamic range
  +Same bandwidth and storage as LDR rendering
  +Certain formats allow for multi-sampling and/or linear filtering with reasonable quality
  -Alpha-blending usually isn't an option, since the alpha-channel is used by most formats
  -Linear filtering and multisampling usually isn't mathmatically correct, although often the results are "good enough"
  -Additional shader math needed for format conversions
  -Adds complexity to shaders

 

My early prototyping used a standard tone-mapping chain and I didn't want to ditch that, nor did I want to move away from what I was comfortable with.  This pretty much eliminated the second option for me off the bat...although I was unlikely to choose it anyway due its other drawbacks (having nice HDR bloom was something I felt was an important part of the look I wanted for my game, and in my opinion Valve's method doesn't do a great job of determining average luminance).  When I tried out the first method I found that it worked as well as it always did on the PC (I've used it before), but on the 360 it was another story.  I'm not sure why exactly, but for some reason it simply does not like the HalfVector4 format.  Performance was terrible, I couldn't blend, I got all kinds of strange rendering artifacts (entire lines of pixels missing), and I'd get bizarre exceptions if I enabled multi-sampling. Loads of fun, let me tell you.

This left me with option #3.  I wasn't a fan of this approach initially, as my original design plan called for things to be simple and straightforward whenever possible.  I didn't really want to have two versions of my material shaders to support encoding, nor did I want to integrate decoding into the other parts of the pipeline that needed.  But unfortunately, I wasn't really left with any other options after I found there were no plans to bring the support for the 360's special fp10 backbuffer format to XNA (which would have conveniently solved my problems on the 360).  So, I started doing my research.  Naturally the first place I looked was to actual released commercial game.  Why?  Because usually when a technique is used in a shipped game, it means it's gone trhough the paces and has been determined to actually be feasible and practical in game environment.  Which of course naturally led me to consider NAO32.

NAO32 is a format that gained some fame in the dev community when ex-Ninja Theory programmer Marco Salvi shared some details on the technique over on the beyond3D forums.  Used in the game Heavenly Sword, it allowed for multi-sampling to be used in conjuction with HDR on a platform (PS3) whose GPU didn't support multi-sampling of floating-point surfaces (The RSX is heavily based on Nvidia G70).  In this technique, color is stored in the LogLuv format usinga standard R8G8B8A8 surface.  Two components are used to store X and Y at 8-bit precision, and the other two are used to store the log of luminance at 16-bit precision.  Having 16 bits for luminance allows for a wide dynamic range to be stored in this format, and storing the log of the luminance allows for linear filtering in multi-sampling or texture sampling.  Since he first explained it other games have also used it, such as Naughty Dog's Uncharted.  It's likely that it's been used in many other PS3 games, as well.

My actual shader implementation was helped along quite a bit by Christer Ericson's blog post, which described how to derive optimized shader code for encoding RGB into the LogLuv format.  Using his code as a starting point, I came up with the following HLSL code for encoding and decoding:

 // M matrix, for encoding
const static float3x3 M = float3x3(
    0.2209, 0.3390, 0.4184,
    0.1138, 0.6780, 0.7319,
    0.0102, 0.1130, 0.2969);

// Inverse M matrix, for decoding
const static float3x3 InverseM = float3x3(
    6.0013,    -2.700,    -1.7995,
    -1.332,    3.1029,    -5.7720,
    .3007,    -1.088,    5.6268);    

float4 LogLuvEncode(in float3 vRGB)
{        
    float4 vResult;
    float3 Xp_Y_XYZp = mul(vRGB, M);
    Xp_Y_XYZp = max(Xp_Y_XYZp, float3(1e-6, 1e-6, 1e-6));
    vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z;
    float Le = 2 * log2(Xp_Y_XYZp.y) + 127;
    vResult.w = frac(Le);
    vResult.z = (Le - (floor(vResult.w*255.0f))/255.0f)/255.0f;
    return vResult;
}

float3 LogLuvDecode(in float4 vLogLuv)
{    
    float Le = vLogLuv.z * 255 + vLogLuv.w;
    float3 Xp_Y_XYZp;
    Xp_Y_XYZp.y = exp2((Le - 127) / 2);
    Xp_Y_XYZp.z = Xp_Y_XYZp.y / vLogLuv.y;
    Xp_Y_XYZp.x = vLogLuv.x * Xp_Y_XYZp.z;
    float3 vRGB = mul(Xp_Y_XYZp, InverseM);
    return max(vRGB, 0);
}

 
Once I had this implemented and worked through a few small glitches, results were much improved in the 360 version.   Performance was much much better, I could multi-sample again, and the results looked great.  So once again things didn't exactly work out in an ideal way, but I'm pleased with the results.

As get deeper into my current project, which happens to be 3D vehicle-based platformer/racer loosely based on the old DOS game Skyroads, I find myself spending more and more time grappling with various headaches that come up on the 360 version.  So I've decided I'm going to share some of trials and tribulations, so that maybe some people learn from them (and also so I can just plain vent about them!). 

One of the early goals I set for myself when starting the project was that I wanted to make some use of modern graphics techniques, and really try to squeeze some performance out of both the 360 and PC GPU's.  This meant some decent shaders, HDR, 4xMSAA 1280 x 720 @ 60fps, and of a good shadowing implementation.  Shadows are a topic that I keep near and dear to my heart, so I'm going to talk about them in my first entry.

When first planning out my renderer, I came up with the following choices for my shadow implementation:

• Stencil shadows
  +Pixel-perfect shadows, no resolution issues
  -Requires multi-pass for any lights needing shadows
  -Hard edges are ugly!
  -Softening is very expensive
  -Tied to geometric complexity
  -Not really any active research on the subject...even Carmack ditched them
• Shadow maps, with standard PCF
  +Very commonly used, lot's of known optimizations
  +Can soften edges and give good filtering when enough samples are used
  +Can be used with an R32F texture, which gives good precision
  -No hardware filtering
  -PCF samples are incredibly expensive on the 360, which has very limited bandwidth
  -Biasing artifacts are very common
• Variance Shadow Maps
  +Can use hardware filtering
  +Can be used with multi-sampling, which is "free" on the 360
  +Can be pre-filtered, for example with seperable gaussian blur
  -Very precision-hungry, in most cases R32G32F is needed (which doubles bandwidth per read)
  -Light bleeding

Ultimately I decide to go with VSM's first, and then try other approaches later.  VSM is capable of producing some tremendously good-looking shadows thanks when anisotropic filtering and a gaussian blur are used.  Plus being able to enable MSAA very cheaply on the 360 made it seem like a very good fit for the hardware, especially considering how expensive PCF can be.  I remember seeing Wolf mention that he was limited to 6 or so PCF taps on the consoles...IMO that's not enough to produce high-quality shadows in many situations.  Later on I confirmed this:  using higher numbers of samples (16, 25) was enough to bring my framerate crashing down on the 360.  VSM has not been without it's headaches though...the anti-aliasing may be cheap but the blurring sure isn't.  Even using a blur with just 9 taps is enough to cause quite a noticable framerate drop, which unfortunate considering the improvement in quality it brings. 

One trick I've found to be quite effective on the 360 is deferred shadow maps.  The technique is one I first saw mentioned by nAo over on the beyond3d forums, and it involves rendering the results of a shadow-map comparison to a texture in a seperate pass before doing the full lighting pass.  It basically works like this:

-Have all geometry render linear depth to a screen-sized texture
-Render the shadow-map
-In a full-screen pass: sample depth from the depth texture, reconstruct view space, convert to light-space, and compare depth with the shadow-map
-Use the shadow occlusion texture to attenuate lighting in the actual main pass

The reason it's a win performance-wise, is because when you perform a full-screen pass you ensure that every quad of pixels is fully utilized.  This isn't the case when you're rendering a bunch of 3D geometry, since the triangles often won't overlap all 4 pixels of a quad.  It's all very nice from a pure design standpoint:  your shadowing is completely decoupled from your lighting.  Your fancy normal-mapping or sub-surface scattering don't need to be aware at all of what shadowing method you're using, it just needs to sample the final attenuation value from a texture.  Plus you don't need to have two versions of those shaders:  if shadows are disabled, you can just feed the lighting shaders a 1x1 white texture.  Having a depth texture available also usually isn't a big deal, since pretty much every modern engine needs depth for just about every friggin' effect. 

As of now with this shadowing system in place, I have everything running at a little over 60 fps with a 1024 x 1024 shadow map being rendering with 4xMSAA.  In order to keep my framerate high enough to remain at a steady 60fps with vsync, I'm defintely going to need to find some more optimizations and the shadowing system is certainly somewhere I'm going to come back to.  However at the moment, I'm quite satisfied with the results. 

 

Since the stuff I'm working on for XNAInfo is taking forever to finish, I thought I'd post a link to this little gem here in the meantime. On my forum rounds I find myself linking to Tom's Excellent DirectX Faq at least once a week. Obviously it's DirectX specific, but that easily translates to tons of useful information on XNA development on Windows. It's a great resource that should prove useful for just about anyone.

 

(Sorry if this pops up twice in your RSS reader, something was off with the publishing) 

Some time ago a topic popped up on GameDev on how to stick a WebBrowser control on a D3D surface, or rather an XNA texture level. After some tinkering we got it to work (obviously Windows only), so it can be used to render any webpage to an XNA texture. The next step was to try and make it interactive, paving the way for HTML and Flash based GUIs. Unfortunately we ran into a strange bug here, which I haven't been able to solve. So I figured I'd post this out here, hoping anyone comes across this who can help out.

Here are the (messy) demo projects:

• Original browser-to-texture code (useful if you're just looking to draw HTML in XNA)
   
• Threaded interactive browser on texture
  

The bug surfaces in the 2nd project. Basically it works fine until the user left-clicks anywhere on the control (doesn't have to be a link), after which WebBrowser.DrawToBitmap fails silently and only an empty white bitmap gets rendered. The strange thing is that the WebBrowser control does load the new webpage. By uncommenting line 241, mouse moves are posted to the control and the window title will show the HREF of links on the (loaded but invisible) page as you hover over them.

Anyway, perhaps someone better versed in window messages can check this out, see if the code for posting mouse presses/releases is correct. While researching this problem, I also came across this MSDN page which states DrawToBitmap isn't supported for the WebBrowser control anyway, so it seems a miracle it works in the first place. If anyone cares to comment on that, please let's hear it.

Having just been badly upstaged by MJP's post, I reckon this might not be too interesting except for a few RTS/AI enthusiasts. Anyway...

In a recent discussion on RTS AI, Matias Goldberg pointed out the quite ancient text The Art of War by Sun Tzu. Since I've been fostering an RTS pet project for years now, I decided to get a copy and it turned out the best $4.95 I probably ever spent (ISBN10 0-486-42557-6).

Obviously it's not a cookbook for writing RTS strategies, but after a first read I can definitely see how various stratagems and axioms could be used to construct a clever AI. In fact, I'm amazed how many clear cut rules of thumb Sun Tzu puts forward that could be readily applied. These however may only serve to highlight the limited scale on which popular RTS games play out.

Food for thought at any rate.

After completing my first game programming tutorial for XNAinfo.com, I've realized something:  I like writing these things!  So I've been spending some idle time musing about future subjects for which a new article or tutorial would be of use to the XNA community.  Here's a few possibilities I've come up with so far:

• Debugging shaders and profiling with PIX.  I can't tell you how many times I've replied to a thread in the gamedev.net forums by saying "just step through your shader in PIX!".  PIX is undoubtedly a huguely useful tool (even when it's occasionally crashing), and I think it's important that beginners get comfortable with it early on.  There's documentation in the SDK, but I think a lot of beginner XNA programmers don't think to look through there and therefore don't understand how powerful PIX can be. 
  
  
• HDR and tonemapping.  It's probably pretty advanced for your average XNA project, but thesedays everybody wants to know how to do it.  There's some great samples in the SDK but once again I'm not sure all XNA programmers want to go poking around in there, especially if they don't know C++.  Plus there's always cool things I could throw in, like the LogLuv encoding scheme Rim and I worked out.
  
  
• Skyboxes.  Another topic I see brought up all the time on the gamedev forums.  It's a very very simple technique once you understand how it works, but unfortunately I don't know of any tutorials devoted to that topic.  Plus there's always a few advanced things I could throw in there...like RGBE (or LogLuv) encoding for HDR, or comparing DXT compression vs. uncompressed.
  
  
• An article or maybe even a series about map editors.  I've been working on a map editor for my own project, and I'm constantly amazed at all the cool stuff you can do with .NET when your engine is made with managed code.
  
  
• A follow-up article for my ModTool tutorial that discusses instancing.  There's an excellent Instancing sample on the creator's club website, however integregating the the technique with the content authoring system I described can be tricky (it requires modifying the shaders quite a bit, and also requires making some changes to the content processor). 
  

Well I guess that's all I have for now.  I'm not sure which I'll start with...maybe the skybox one because it's easy. 

On my favorite GameDev forum a lot of threads seem to be popping up from beginners worrying that getting started in game development, even as a hobby, is Hard. Lacking anything really useful to talk about at the moment, I thought I'd write up some points about this.

Get your feet wet

When you're just starting out everything seems hard and no amount of reading is going to do you much good until you sit down and just take a shot at it. Don't worry if you don't create an AAA game on your first try or even when you hit a wall and don't know how to continue (I'll get back on that). The key is, you've at the very least got some basics down and tried out various things that did or did not work. Congratulations, you've now learned something and are no longer a n00b.
 

The best language   

Please stop worrying about what would be the best programming language, please! If you even vaguely know how to code in some language, just stick with that unless you feel you absolutely need to move to another language since it offers something you really can't live without. Programming languages may all have their pros and cons (in no small part due to the libraries they'll allow you to use), but getting bogged down learning hard language X isn't going to help you create your game. Most coding skills you aquire carry over pretty well and should make it a lot easier to learn other languages anyway. And though it may be a fact that language X is the current industry standard,  should you really care about that now?
 

Getting into the industry

Or rather, do you really want to be in the industry right now or in the near future? It's a nice dream to create games for a living and certainly worth hanging on to, but when you're just getting started you really shouldn't be worrying about the industry too much. Just have a go at creating some games or at least coding some fancy things to get a feel for it. Then if coding games really turns out to be the carreer what you want, you will still have plenty of time to brush up on industry standards and how to actually get in there. In the meantime, you'll have a bit of a portfolio layed down which just might show off your experience and passion better than stating you know language X. Here's a little anecdote.

Getting stuck

So you went and started coding your game, but you've hit a wall. That's ok, it means you have something new to learn. Research the problem, ask around on forums and try to get it working. Even if it turns out you've created a monolithic unwieldly behemoth of a program or that something's deeply flawed in your approach, you'll at least know to avoid it in the future and you can start fresh with your newly gained knowledge. But chances are things aren't this bad and you'll have learned how to solve the (perhaps common) issue.
 

Well, that about covers it for now. I'm sure more ramblings will be forthcoming

I stumbled upon Tim Schafers development diary for Grim Fandango today on GameSpot. Obviously it's quite dated, but it's both interesting and fun to read that there are just some things that never seem to change.