This project is read-only.

Model Shader Modification

Jun 23, 2009 at 3:41 PM

Hi there!

I'm the leader of a team of three programmers and two artists. We've been using XEN for about two/three weeks, but I've run into a bit of trouble regarding shaders. What's the best way to apply a new shader to an animated, normal mapped, lit and textured model? Is there a way to access the shader that this model uses?

 

Hope to hear from you soon!


Dave

Jun 24, 2009 at 12:52 AM
Edited Jun 24, 2009 at 1:02 AM

As you might imagine that's a fairly tricky thing to do :-)
First, I'll say the possible alternative is that you draw the model in twice - once with the current shader and once with your extra shader. If this can achieve the effect you desire (ie, by using additive or alpha blending, etc) then that is probably the easiest way. However you still need to implement animation, etc, in the shader. (This is what is done in the shadow map example)

The shaders are buried deep within MaterialShader. MaterialShader is a complex beast and I've tried my best to hide that complexity.
Internally, how it works is it had a *large* set of existing shader techniques for the *large* number of combinations of vertex and pixel options available. Then, at runtime, it combines the vertex options VS and pixel options PS to produce a new shader to perform the exact lighting / material options desired. There are literally 100s of thousands of combinations that could be possible with this system, and being SM 2.0 (and due to early limits with the shader system) flow control wasn't an option most of the time.

So. With that out the way :-)
The case you describe will be using a vertex shader from BlendMaterial.fx, and a pixel shader from Material.fx.

If the model is animated, normal map lit (with specular?) and using a texture, then the shaders in use will look like this:

Vertex shader:

  float4x4 world : WORLD;
  float4x4 worldViewProj : WORLDVIEWPROJECTION;
  float4x4 viewProj : VIEWPROJECTION;

  float4 viewPoint : VIEWPOINT;
  float4 ambient;

  //format is position,specular,diffuse,attenuation
  float4 v_lights[24]; //max 6

  float4 blendMatrices[72*3];


  void VS_norm_blend(
        float4 pos_in : POSITION,
        float3 normal : NORMAL, // z
        float3 binorm : BINORMAL, // y
        float3 tangent : TANGENT, // x
        
        out float4 pos_out : POSITION,
        
        float4 vcolour : COLOR,
        float2 tex0 : TEXCOORD0,
        
        out float4 colour : TEXCOORD0,
        out float2 tex : TEXCOORD2,
        out float4 wpos : TEXCOORD3,
        out float4 vcolour_out : TEXCOORD1,
        
        out float4 onormal : TEXCOORD4,
        out float4 obinormal : TEXCOORD5,
        out float4 otangent : TEXCOORD6,
        
        float4 weights : BLENDWEIGHT,
        float4 indices : BLENDINDICES,

        out float4 viewPointOut : TEXCOORD7,
        
        uniform int vsLightCount,
        uniform bool useVertexColours)
  {

    //don't worry about the transpose, it gets optimised out and this is actually very efficient.
    float4x3 blendMatrix
           = transpose(float3x4(
            blendMatrices[indices.x*3+0] * weights.x + blendMatrices[indices.y*3+0] * weights.y + 
              blendMatrices[indices.z*3+0] * weights.z + blendMatrices[indices.w*3+0] * weights.w,
            blendMatrices[indices.x*3+1] * weights.x + blendMatrices[indices.y*3+1] * weights.y + 
              blendMatrices[indices.z*3+1] * weights.z + blendMatrices[indices.w*3+1] * weights.w,
            blendMatrices[indices.x*3+2] * weights.x + blendMatrices[indices.y*3+2] * weights.y + 
              blendMatrices[indices.z*3+2] * weights.z + blendMatrices[indices.w*3+2] * weights.w
             ));
                   
    float3 blendPosition =	mul(pos_in,blendMatrix);  
    wpos = float4(mul(float4(blendPosition,1),world).xyz,1);
    
    pos_out = mul(wpos,viewProj);
    
    tex = tex0;
    colour = ambient;
    normal = normalize(mul(mul(normal,(float3x3)blendMatrix).xyz,(float3x3)world).xyz);

    onormal = float4(normal,0);
    obinormal = float4(normalize(mul(mul(binorm,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
    otangent = float4(normalize(mul(mul(tangent,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
    
    
    viewPointOut = viewPoint;
    float3 viewNorm = normalize(viewPoint.xyz - wpos);
    
    float3 vertexColours = 0;
    for(int n=0; n<vsLightCount; n++)
    {
      float4 lightPosition = v_lights[n*4];
      float4 specularColour = v_lights[n*4+1];
      float3 diffuseColour = v_lights[n*4+2].xyz;
      float4 attenuation = v_lights[n*4+3];
      
      float3 lightDif = lightPosition.xyz - wpos*lightPosition.w;
      float len2 = dot(lightDif,lightDif);
      float len = sqrt(len2);
      
      lightDif /= len;
      
      float diffuse = saturate(dot(lightDif,normal));
      float specular = saturate(dot(normalize(lightDif+viewNorm),normal));
      
      float falloff = 1.0 / (attenuation.x + len * attenuation.y + len2 * attenuation.z);
      
      specular = pow(specular,specularColour.w);
      
      diffuse *= falloff;
      specular *= falloff;
      
      vertexColours += specularColour.xyz * specular + diffuseColour * diffuse;
      
    }
    colour.xyz += vertexColours;
    
    vcolour_out = 1;
    if (useVertexColours)
    {
      vcolour_out = vcolour;
    }
  }

Pixel shader:

  float4 p_lights[16] : register(c0); //max 4


  float4 PS_norm_specular(
        float4 vcolour : TEXCOORD1,
        float4 colour : TEXCOORD0,
        float2 tex : TEXCOORD2,
        float3 wpos : TEXCOORD3,
        float3 normal : TEXCOORD4,
        float3 binorm : TEXCOORD5,
        float3 tangent : TEXCOORD6,
        float3 viewPoint : TEXCOORD7,
        const uniform int psLightCount) : COLOR
  {
    float4 normalMap = tex2D(CustomNormalMapSampler,tex);
    float3x3 normalSpace = float3x3(tangent,binorm,normal);
    
    normal = normalize(mul(normalMap.xyz-0.5,normalSpace));
    float3 viewNorm = normalize(viewPoint - wpos);
    
    float3 pixelColour = 0;
    
    for(int n=0; n<psLightCount; n++)
    {
      float4 lightPosition = p_lights[n*4];
      float4 attenuation = p_lights[n*4+3];
      float3 lightDif = lightPosition.xyz - wpos*lightPosition.w;
      float len2 = dot(lightDif,lightDif);
      float len = sqrt(len2);
    
      lightDif /= len;
      
      float diffuse = saturate(dot(lightDif,normal));
      float specular = saturate(dot(normalize(lightDif+viewNorm),normal));
      
      float falloff = 1.0 / (attenuation.x + len * attenuation.y + len2 * attenuation.z);
      
      specular = pow(specular,p_lights[n*4+1].w);
      
      diffuse *= falloff;
      specular *= falloff * normalMap.w;
      
      pixelColour += p_lights[n*4+1].xyz * specular + p_lights[n*4+2].xyz * diffuse;
    }
    float4 textureSample = tex2D(CustomTextureSampler,tex);
    colour.xyz += pixelColour;
    
    return textureSample * colour * vcolour;
  }

Note the VS may be reworked a bit in a future version (but it won't look any different). The PS without specular is much less complex as well (it's in Material.fx)

Jun 24, 2009 at 11:17 AM
Edited Jun 24, 2009 at 11:23 AM

OK, I've implemented the two methods into a technique, but when I run it applied to our model, I get an unhandled InvalidOperationException  - "Cannot use shader of type 'QAT.Shaders.Shiney.Shine' with Vertices<System.Byte> object as it does not include element 'Tangent0'".

Here is our override class

 

class ShaderOverride : ModelInstanceShaderProvider
     {
         public override bool ProviderModifiesWorldMatrixInBeginDraw
         {
             get {return false; }
         }

         public override bool BeginGeometryShaderOverride(DrawState state, GeometryData geometry, MaterialLightCollection lights)
         {
             QAT.Shaders.Shiney.Shine shader = null;

             shader = state.GetShader<QAT.Shaders.Shiney.Shine>();

             shader.Ambient = new Vector4(1.0f, 1.0f, 1.0f, 1.0f);
             shader.CustomTexture = geometry.MaterialShader.TextureMap;
             shader.CustomNormalMap = geometry.MaterialShader.NormalMap;

             shader.Bind(state);

             return true;

             //return false;

         }
     }

 

When I try this with different shaders from Material.fx, I can manage to get a bind-pose model, with a texture and the amibient light applied - is all the information the shader requires being passed in automatically?

Also, how do I pass in lighting information into the shader?

Man, this got long fast!

Thanks for your help so far!

 

Dave

Jun 24, 2009 at 11:21 AM

Might help if I actually post the shader too

//Lol, shader?
float4x4 world : WORLD;
float4x4 worldViewProj : WORLDVIEWPROJECTION;
float4x4 viewProj : VIEWPROJECTION;

float4 viewPoint : VIEWPOINT;
float4 ambient;

//format is position,specular,diffuse,attenuation
float4 v_lights[24]; //max 6

float4 blendMatrices[72*3];

float4 p_lights[16]; //max 4

texture2D CustomTexture;
sampler2D CustomTextureSampler = sampler_state
{
	Texture = (CustomTexture);
};
texture2D CustomNormalMap;
sampler2D CustomNormalMapSampler = sampler_state
{
	Texture = (CustomNormalMap);
};

void VS_norm_blend(
      float4 pos_in : POSITION,
      float3 normal : NORMAL, // z
      float3 binorm : BINORMAL, // y
      float3 tangent : TANGENT, // x
      
      out float4 pos_out : POSITION,
      
      float4 vcolour : COLOR,
      float2 tex0 : TEXCOORD0,
      
      out float4 colour : TEXCOORD0,
      out float2 tex : TEXCOORD2,
      out float4 wpos : TEXCOORD3,
      out float4 vcolour_out : TEXCOORD1,
      
      out float4 onormal : TEXCOORD4,
      out float4 obinormal : TEXCOORD5,
      out float4 otangent : TEXCOORD6,
      
      float4 weights : BLENDWEIGHT,
      float4 indices : BLENDINDICES,

      out float4 viewPointOut : TEXCOORD7,
      
      uniform int vsLightCount,
      uniform bool useVertexColours)
{

  //don't worry about the transpose, it gets optimised out and this is actually very efficient.
  float4x3 blendMatrix
         = transpose(float3x4(
          blendMatrices[indices.x*3+0] * weights.x + blendMatrices[indices.y*3+0] * weights.y + 
            blendMatrices[indices.z*3+0] * weights.z + blendMatrices[indices.w*3+0] * weights.w,
          blendMatrices[indices.x*3+1] * weights.x + blendMatrices[indices.y*3+1] * weights.y + 
            blendMatrices[indices.z*3+1] * weights.z + blendMatrices[indices.w*3+1] * weights.w,
          blendMatrices[indices.x*3+2] * weights.x + blendMatrices[indices.y*3+2] * weights.y + 
            blendMatrices[indices.z*3+2] * weights.z + blendMatrices[indices.w*3+2] * weights.w
          ));
                 
  float3 blendPosition =	mul(pos_in,blendMatrix);  
  wpos = float4(mul(float4(blendPosition,1),world).xyz,1);
  
  pos_out = mul(wpos,viewProj);
  
  tex = tex0;
  colour = ambient;
  normal = normalize(mul(mul(normal,(float3x3)blendMatrix).xyz,(float3x3)world).xyz);

  onormal = float4(normal,0);
  obinormal = float4(normalize(mul(mul(binorm,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
  otangent = float4(normalize(mul(mul(tangent,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
  
  
  viewPointOut = viewPoint;
  float3 viewNorm = normalize(viewPoint.xyz - wpos);
  
  float3 vertexColours = 0;
  for(int n=0; n

Thanks again!

Jun 24, 2009 at 11:23 AM
Edited Jun 24, 2009 at 11:26 AM

Sounds like the model doesn't have a normal map, in which case it won't be generating tangent frames. (or, at least, parts of it doesn't have a normal map).
You can get around it in two ways, first is to check the GeometryData passed in to the override method - and check it's material to see if there is a normal map (and use a different shader in that case)
OR, the much easier way :) you can force the model to always generate tangent frames by setting the property in the model files content properties. Set 'Generate tangent frames' to true.

The error you were getting was a bit cryptic because the only way to get vertex data from a content pipeline model is a raw byte[] array. But as you see, in debug mode I do a sanity check to make sure what you are trying to access in the shader actually exists in the geometry data. :-) Saves some really nasty bugs

Jun 24, 2009 at 11:40 AM
Edited Jun 24, 2009 at 12:03 PM

As you probably noticed, the shaders both take in a constant integer to say how many lights there are. This internally is how MaterialShader works, creating a whole load of variations for each number of lights.
So supporting a variable number of lights would mean implementing a similar (and very complex) system. I'll have some stuff in the next update that would make this easier (better flow control support for SM3).

As for the actual array values set (for the light data) you can see the format is 4x4x3x3 in position (w indicates directional), specular RGB+power, diffuse RGB and finally attenuation CLQ (constant, linear, quadratic). It's all setup in the material shader code (as you might guess).

Jun 24, 2009 at 12:02 PM

OK! Sorry to keep bothering you, but now the model is not rendered at all. I pilfered some vertex and pixel shaders that don't have normal mapping- here it is-

//Lol, shader?
float4x4 world : WORLD;
float4x4 worldViewProj : WORLDVIEWPROJECTION;
float4x4 viewProj : VIEWPROJECTION;

float4 viewPoint : VIEWPOINT;
float4 ambient;

//format is position,specular,diffuse,attenuation
float4 v_lights[24]; //max 6

float4 blendMatrices[72*3];

float4 p_lights[16]; //max 4

texture2D CustomTexture;
sampler2D CustomTextureSampler = sampler_state
{
	Texture = (CustomTexture);
};

//***************************************************
//  Pilfered Shader Code Goes Here
//***************************************************

technique Shine
{
   pass
   {
		VertexShader = compile vs_3_0 VS_blend(6,true, false, true);
		PixelShader = compile ps_3_0 PS_specular(2, true);
   }
}

The override function is the same as the one above except with the normal mapping stuff removed.

I'm concerned that the issue might be hard to diagnose short of sending you everything we've got, but any help is greatly appreciated!

Thanks again for the help!

Jun 24, 2009 at 12:05 PM

did you pull the normal mapping code out, or did you copy the non-normal map versions from MaterialShader?

Jun 24, 2009 at 1:13 PM

It uses VS_blend from blendmaterial.fx and PS_specular from material.fx, so they should be the non-normal map versions. Should I have pulled the normal stuff out of the normal mapping shaders?

Jun 24, 2009 at 1:21 PM
Edited Jun 25, 2009 at 12:44 AM

No, that should be correct.

*EDIT*

Doh. In the shader provider class, if you want animation support you need to override the two BeginDraw() methods.  The method that is called depends if the model is animated - in which case the overload with animation bone data is called.
Depending on this, you will then use an animated or non-animated shader - and in the animated case, you will need to copy in the animation bone data to the blendMatrices array (this would be why nothing is showing up).
The only point I'll make is there is a small issue in that example, where it'll copy the animation bones redundantly if a mesh has more than one piece of geometry.

The shadow mapping example does something very similar to this.

Jun 24, 2009 at 4:50 PM
Edited Jun 25, 2009 at 10:08 AM

Success!

Thanks for your help, we can now add in our own effects! I'll post a screenshot a bit later...

Here it is!

http://daretobedigital.com/AdminPanel/team_diary/team_diary_images/1875_1big.jpg

 

Thanks again for the help- hopefully everything else will go more smoothly.

Jun 25, 2009 at 1:23 PM

Hey that's looking good :-)
I hope (otherwise) things have have been working out well. Let me know if anything is tricky or proves troublesome.

The glow is the effect you have been creating? If so, that's the perfect sort of effect to render the model a second time (with alpha blending). That way you can still keep the complex lighting intact without having to recreate it.

Good luck! And feel free to post more pics if you make something cool :)