BlurFilter

Sep 7, 2010 at 9:08 AM

Hi,

I recently integrated xen in my xna project and I'm quite happy regarding the "programmer friendly" aspect of the framework as well as its efficiency (performance wise). However, i'm a bit disapointed by the lack of documentation (personnaly i would like to have more comments/explainations in the API doc ...).

I'm new here so that was just for a global feedback :D

Today my question is about applying blur on objects in a scene. What I want to do is to blur the displayed object, according to the distance between them and the camera. The further they are, blurryer there are (and vice versa). Initialy i wanted to use a shader but I saw the BlurFilter class in Xen.Ex.Filters.

Can I reach my goal using the BlurFilter and if yes, how to do it ?

Thanks !

Coordinator
Sep 7, 2010 at 12:28 PM

Hi.

Typically, most games implement DOF is fairly simple, they render the scene to a texture, create a blurred copy (BlurFilter) then display both images on screen using the depth buffer to interpolate (so distant objects show the blurred texture, close ones show the source texture).

The trouble is, XNA doesn't give you access to the depth buffer as a texture. This means you will need to write depth out yourself, this is the tricky bit. For example, you could write depth to alpha - however this brings up some potential issues with transparent objects (you'd want to mask out writing alpha in such a case). You probably don't want to render the scene a second time just for depth. Although it would depend a lot on your game.

Does that make sense?

Cheers

Sep 7, 2010 at 1:26 PM

Yes,

that make sense and maybe I could try the solution you suggest.

However, that make sense about the concept but I'm a bit confused regarding the implementation.

I think I will need an example :D

So,

Considering I have my source Texture2D (basic rendering of the game using a DrawTargetTexture2D, instead of using a DrawTargetScreen), the blured Texture2D (obtained by applying the Blurfilter on the source Texture2D), and of course a DrawTargetScreen to render the result on the screen.

Using your suggestion (and if you have time), could you give me a kind of step by step example of code with xen ? Particularly how do you write depth to alpha and then how do you compute the interpolation between the textures ?

I'm not very comfortable with per pixel stuff and depthbuffer manipulation :/

Otherwise, rather than writing depth to alpha, maybe i could use a 2D array of floats (or something) to write depth inside and then use it to interpolate ?

Sep 9, 2010 at 12:28 PM

Hi,

I found a tutorial about implementing depth of field with XNA.

So I think I will adapt the code to make it run with Xen.

The problem is that when i want to compile the shaders using XenFX, I get empty .fx.cs files with this comment (undefined object reference):

// Unhandled exception in XenFX:
//System.NullReferenceException: La r?f?rence d'objet n'est pas d?finie ? une instance d'un objet.
//   ? Xen.Graphics.ShaderSystem.CustomTool.Dom.ShaderRegisters..ctor(SourceShader source, String techniqueName, Platform platform) dans c:\Users\Clauclaudr\Downloads\xen_1.8.2\xen\src\Xen.Graphics.ShaderSystem\Xen.Graphics.ShaderSystem.CustomTool\Dom\Registers.cs:ligne 21
//   ? Xen.Graphics.ShaderSystem.CustomTool.Dom.ShaderDom..ctor(SourceShader source, String techniqueName, Platform platform, CompileDirectives directives) dans c:\Users\Clauclaudr\Downloads\xen_1.8.2\xen\src\Xen.Graphics.ShaderSystem\Xen.Graphics.ShaderSystem.CustomTool\Dom\ShaderDom.cs:ligne 70
//   ? Xen.Graphics.ShaderSystem.CustomTool.Dom.SourceDom..ctor(SourceShader shader, String baseNamespace, CodeDomProvider provider) dans c:\Users\Clauclaudr\Downloads\xen_1.8.2\xen\src\Xen.Graphics.ShaderSystem\Xen.Graphics.ShaderSystem.CustomTool\Dom\Dom.cs:ligne 76
//   ? Xen.Graphics.ShaderSystem.CustomTool.ShaderCodeGenerator.GenerateCode(String inputFileName, String inputFileContent, String fileNameSpace, CodeDomProvider codeProvider) dans c:\Users\Clauclaudr\Downloads\xen_1.8.2\xen\src\Xen.Graphics.ShaderSystem\Xen.Graphics.ShaderSystem.CustomTool\PluginEntry.cs:ligne 40

The error is the same for both shaders.

As an example, here is the code of one of the shaders, PostProcessBlur.fx :

// The blur amount( how far away from our texel will we look up neighbour texels? )
float BlurDistance = 0.003f;

// This will use the texture bound to the object( like from the sprite batch ).
sampler ColorMapSampler : register(s0);

float4 PixelShader(float2 Tex: TEXCOORD0) : COLOR
{
    float4 Color;
   
    // Get the texel from ColorMapSampler using a modified texture coordinate. This
    // gets the texels at the neighbour texels and adds it to Color.
    Color  = tex2D( ColorMapSampler, float2(Tex.x+BlurDistance, Tex.y+BlurDistance));
    Color += tex2D( ColorMapSampler, float2(Tex.x-BlurDistance, Tex.y-BlurDistance));
    Color += tex2D( ColorMapSampler, float2(Tex.x+BlurDistance, Tex.y-BlurDistance));
    Color += tex2D( ColorMapSampler, float2(Tex.x-BlurDistance, Tex.y+BlurDistance));

    // We need to devide the color with the amount of times we added
    // a color to it, in this case 4, to get the avg. color
    Color = Color / 4;   
   
    // returned the blured color
    return Color;
}

technique PostProcessBlur
{
    pass P0
    {
        // A post process shader only needs a pixel shader.
        PixelShader = compile ps_2_0 PixelShader();
    }
}

 

Any idea on how to fix this ?

Coordinator
Sep 26, 2010 at 11:54 AM

Hi

Sorry I havn't replied quickly. I've been on holiday for a while :-)

 

Looks like a bug in the shader system there. It should be reporting that the technique must have a vertex shader defined as well as a pixel shader - and it should also be complaining about the pass being named. I'll look into it and see if the problem still exists.

In any case, the BlurFilter already does the blur very efficiently - so I would highly recommend you use that for the actual blur step. The tricky bit is how to interpolate to it, it you would need to create a shader that interpolates between the original rendererd image, and the blurred version. This would be done based on depth (which would need to be generated - eg, using the linear depth output shaders in Ex.Shaders).

This would require the extra step of rendering the scene a second time to an extra DrawTargetTexture2D in order to generate the depth(urgh!) - unless you can fit it in somewhere else in your rendering pipeline.

The output shader would then be fairly simple... It would take the three textures as input (orignal rendererd texture, blurred version of that texture, and the depth texture) - sample all three, then do a lerp() between the first two, based on the depth. Used with a ShaderElement to display it to the screen.

Does that make sense? Unfortunately I really don't think I have the time to whip up a sample right now :-(

Sep 27, 2010 at 10:04 AM
Edited Sep 27, 2010 at 10:27 AM

Hi

Yes that make sense and it is exactly the way it's done in the tutorial I found.

So, what i want to do is to use the tutorial's DOF computing shader :

 

// This will use the texture bound to the object( like from the sprite batch ).
sampler SceneSampler : register(s0);

float Distance;
float Range;
float Near;
float Far;
                       
texture D1M;  
sampler D1MSampler = sampler_state { Texture = <D1M>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; AddressU = Clamp; AddressV = Clamp; }; texture BlurScene; sampler BlurSceneSampler = sampler_state { Texture = <BlurScene>; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; AddressU = Clamp; AddressV = Clamp; }; float4 PixelShader(float2 Tex: TEXCOORD0) : COLOR { // Get the scene texel float4 NormalScene = tex2D(SceneSampler, Tex); // Get the blurred scene texel float4 BlurScene = tex2D(BlurSceneSampler, Tex); // Get the depth texel float fDepth = tex2D(D1MSampler, Tex).r; // Invert the depth texel so the background is white and the nearest objects are black fDepth = 1 - fDepth; // Calculate the distance from the selected distance and range on our DoF effect, set from the application float fSceneZ = ( -Near * Far ) / ( fDepth - Far); float blurFactor = saturate(abs(fSceneZ-Distance)/Range); // Based on how far the texel is from "distance" in Distance, stored in blurFactor, mix the scene return lerp(NormalScene,BlurScene,blurFactor); } technique PostProcess { pass P0 { PixelShader = compile ps_2_0 PixelShader(); } }

 

The problem is that i can't compile this shader with xenFx :/ (like you saw in my previous post. I get the same error with this one ...)

About the blur, as you recommend, i'll use the blurfilter in xen.ex.filters.

In the tutorial, to generate the depth texture, they use a shader as well but it seems to be a bit tricky to make it run with xen (they directly deal with the GraphicsDevice, using a DepthStencilBuffer and a xna RenderTarget2D).

So, if i want to use Xen's depth output shaders int order to generate my depth texture, i have to instanciate a DepthOutRg (or another one ?), bind it to a DrawTargetTexture2D, render the scene, and then get the texture of this DrawTargetTexture2D ?

Sep 27, 2010 at 3:18 PM

According to what you said previously, i modified a bit the PostProcessDOF shader to make it compile with XenFX (now it compiles well) :

float Distance;
float Range;
float Near;
float Far;

texture BaseScene;
sampler SceneSampler = sampler_state
{
   Texture = <BaseScene>;
   MinFilter = Linear;
   MagFilter = Linear;
   MipFilter = Linear;  
   AddressU  = Clamp;
   AddressV  = Clamp;
};

texture BlurScene;
sampler BlurSceneSampler = sampler_state
{
   Texture = <BlurScene>;
   MinFilter = Linear;
   MagFilter = Linear;
   MipFilter = Linear;  
   AddressU  = Clamp;
   AddressV  = Clamp;
};
           
texture DepthMap;
sampler DepthMapSampler = sampler_state
{
   Texture = <DepthMap>;
   MinFilter = Linear;
   MagFilter = Linear;
   MipFilter = Linear;  
   AddressU  = Clamp;
   AddressV  = Clamp;
};


float4 DOF_PixelShader(float2 Tex: TEXCOORD0) : COLOR
{
    // Get the scene texel
    float4 NormalScene = tex2D(SceneSampler, Tex);
   
    // Get the blurred scene texel
    float4 BlurScene = tex2D(BlurSceneSampler, Tex);
   
    // Get the depth texel
    float  fDepth = tex2D(DepthMapSampler, Tex).r;
   
    // Invert the depth texel so the background is white and the nearest objects are black
    fDepth = 1 - fDepth;
   
    // Calculate the distance from the selected distance and range on our DoF effect, set from the application
    float fSceneZ = ( -Near * Far ) / ( fDepth - Far);
    float blurFactor = saturate(abs(fSceneZ-Distance)/Range);
   
    // Based on how far the texel is from "distance" in Distance, stored in blurFactor, mix the scene
    return lerp(NormalScene,BlurScene,blurFactor);
}

void DOF_VertexShader(
    float4 pos : POSITION, float2 tex : TEXCOORD0,
    out float4 o_pos : POSITION, out float2 o_tex : TEXCOORD0)
{
    o_pos = pos;
    o_tex = tex;
}

technique PostProcessDOF
{
    pass
    {
        PixelShader = compile ps_2_0 DOF_PixelShader();
        VertexShader = compile vs_2_0 DOF_VertexShader();
    }
}

For the vertex shader i just put very simple stuff because I guess there is nothing special to do with the vertices (this shader only deals with the pixels of the 3 textures) ?

Coordinator
Sep 27, 2010 at 7:57 PM

Yup. I've fixed that up in my local build,

To get that shader working, just multiply the position by the world*view*projection matrix, like most other shaders do.

So just add:

float4x4 worldViewProjection    : WORLDVIEWPROJECTION;

 

and change the line:

o_pos = mul(pos, worldViewProjection);


Then it should work fine with 2D element code. Otherwise that looks fine, just a matter of generating the depth texture.
As you guessed, just bind one of the depth output shaders (probably the linear one) and draw your scene again.

Sep 28, 2010 at 8:20 PM
Edited Sep 28, 2010 at 8:29 PM

Hi !

I managed to get Depth Of Field working well in my application :)

You can check the result by watching this short video I made in order to quickly illustrate that.

 

So, to sum up the process :

(all the fields are members of the class which manages the process)

1/ The scene is drawn into a DrawTargetTexture2D (let's say target1)

2/ The BaseScene field of the DofShader is set with the texture of target1

3/ A TexturedElement linked to target1 (passed in the constructor to automatically update the texture) is drawn into a second DrawTargetTexture2D (let's say target2)

4/ The BlurFilter object, initially added to target2 is applied (by calling Draw)

5/ The BlurScene field of the DofShader is set with the texture of target2

6/ The base scene is drawn into a third DrawTargetTexture2D (let's say target3), with a DepthOutRg shader instance binded to the DrawState

7/ The DepthMap field of the DofShader is set with the texture of target3

8/ Finally, a ShaderElement linked to the DofShader is drawn into the DrawTargetScreen of the application

 

Do not hesitate to ask me questions or to tell me if something looks wrong.

Maybe there is a more efficient way to do this ...

 

By the way I modified again the Pixel Shader to make it a little bit simpler and get the result I wished :

(The shader linearly compute the blur amount according to a focus distance and a far clip)

 

float4 DOF_PixelShader(float2 Tex: TEXCOORD0) : COLOR
{
    // Get the scene texel
    float4 NormalScene = tex2D(SceneSampler, Tex);
   
    // Get the blurred scene texel
    float4 BlurScene = tex2D(BlurSceneSampler, Tex);
   
    // Get the depth texel
    float  fDepth = tex2D(DepthMapSampler, Tex).r;
   
    float distRatio = Distance/Far;
   
    float blurFactor = clamp(abs((fDepth - distRatio)/(1 - distRatio)), 0, 1);
   
    // Based on how far the texel is from "distance" in Distance, stored in blurFactor, mix the scene
    return lerp(NormalScene,BlurScene,blurFactor);
}

 

One last remark :

In order to avoid an important performance loss and as well get better results visually speaking, the Far Clip of the camera's projection is reduced when the depth map is drawn. This reduced far clip value is also the one set to the Far field of the DOF Shader.

For example, in my application, the Far Clip for Depth Map and Dof Shader is 10 time smaller than the base scene one. In consequences the rendering of the depth map is less expansive and the result is that all the visual stuff located between this reduced far clip and the normal far clip is simply displayed with the maximum blur.

That's it.

Thanks a lot :)

 

 

 

 

Coordinator
Sep 28, 2010 at 11:25 PM
Edited Sep 28, 2010 at 11:27 PM

Cool :-)

You can also do simple tricks, like draw the depth map at a lower resolution (certainly, you would stuggle to notice half resolution - or even less). The same is true of the blurred texture, it may look just as good copying it to a half or quater size RT, and blurring that - which would be faster given blurs can be very expensive.

You might notice that the technique isn't perfect. You get nasty halos around all your near objects - because the pixels around them technically are at a very far depth - but the DOF blur filter is in screenspace. Fixing this issue is typically very difficult, either requiring a 'smart' blur that is depth aware (either tricky or expensive! - at minimum it requires 2x the texture lookups and a lot more logic) or you do something sneaky, like 'inflate' the objects drawn into the depth buffer if they are close to the viewer (typically done by extruding the vertices along the normal).

Lots of games simply use an artist defined shell model, typically much lower detail and slightly enlarged, to be used when drawing depth, motion vectors, or whatever extra information they need.

Sep 29, 2010 at 9:42 AM

I have'nt thought to lower the resolution of the blur texture and the depth map. I'll do it ;)

As you said I noticed the halos problem. The scale solution you suggested looks interesting.

However it is probably something i'll fix later, in order to be sure that's ok Art wise.

Thanks !