
Hi there!
I'm the leader of a team of three programmers and two artists. We've been using XEN for about two/three weeks, but I've run into a bit of trouble regarding shaders. What's the best way to apply a new shader to an animated, normal mapped, lit and textured
model? Is there a way to access the shader that this model uses?
Hope to hear from you soon!
Dave


Jun 24, 2009 at 12:52 AM
Edited Jun 24, 2009 at 1:02 AM

As you might imagine that's a fairly tricky thing to do :)
First, I'll say the possible alternative is that you draw the model in twice  once with the current shader and once with your extra shader. If this can achieve the effect you desire (ie, by using additive or alpha blending, etc) then that is probably the easiest
way. However you still need to implement animation, etc, in the shader. (This is what is done in the shadow map example)
The shaders are buried deep within MaterialShader. MaterialShader is a complex beast and I've tried my best to hide that complexity.
Internally, how it works is it had a *large* set of existing shader techniques for the *large* number of combinations of vertex and pixel options available. Then, at runtime, it combines the vertex options VS and pixel options PS to produce a new shader to
perform the exact lighting / material options desired. There are literally 100s of thousands of combinations that could be possible with this system, and being SM 2.0 (and due to early limits with the shader system) flow control wasn't an option most of the
time.
So. With that out the way :)
The case you describe will be using a vertex shader from BlendMaterial.fx, and a pixel shader from Material.fx.
If the model is animated, normal map lit (with specular?) and using a texture, then the shaders in use will look like this:
Vertex shader:
float4x4 world : WORLD;
float4x4 worldViewProj : WORLDVIEWPROJECTION;
float4x4 viewProj : VIEWPROJECTION;
float4 viewPoint : VIEWPOINT;
float4 ambient;
//format is position,specular,diffuse,attenuation
float4 v_lights[24]; //max 6
float4 blendMatrices[72*3];
void VS_norm_blend(
float4 pos_in : POSITION,
float3 normal : NORMAL, // z
float3 binorm : BINORMAL, // y
float3 tangent : TANGENT, // x
out float4 pos_out : POSITION,
float4 vcolour : COLOR,
float2 tex0 : TEXCOORD0,
out float4 colour : TEXCOORD0,
out float2 tex : TEXCOORD2,
out float4 wpos : TEXCOORD3,
out float4 vcolour_out : TEXCOORD1,
out float4 onormal : TEXCOORD4,
out float4 obinormal : TEXCOORD5,
out float4 otangent : TEXCOORD6,
float4 weights : BLENDWEIGHT,
float4 indices : BLENDINDICES,
out float4 viewPointOut : TEXCOORD7,
uniform int vsLightCount,
uniform bool useVertexColours)
{
//don't worry about the transpose, it gets optimised out and this is actually very efficient.
float4x3 blendMatrix
= transpose(float3x4(
blendMatrices[indices.x*3+0] * weights.x + blendMatrices[indices.y*3+0] * weights.y +
blendMatrices[indices.z*3+0] * weights.z + blendMatrices[indices.w*3+0] * weights.w,
blendMatrices[indices.x*3+1] * weights.x + blendMatrices[indices.y*3+1] * weights.y +
blendMatrices[indices.z*3+1] * weights.z + blendMatrices[indices.w*3+1] * weights.w,
blendMatrices[indices.x*3+2] * weights.x + blendMatrices[indices.y*3+2] * weights.y +
blendMatrices[indices.z*3+2] * weights.z + blendMatrices[indices.w*3+2] * weights.w
));
float3 blendPosition = mul(pos_in,blendMatrix);
wpos = float4(mul(float4(blendPosition,1),world).xyz,1);
pos_out = mul(wpos,viewProj);
tex = tex0;
colour = ambient;
normal = normalize(mul(mul(normal,(float3x3)blendMatrix).xyz,(float3x3)world).xyz);
onormal = float4(normal,0);
obinormal = float4(normalize(mul(mul(binorm,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
otangent = float4(normalize(mul(mul(tangent,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
viewPointOut = viewPoint;
float3 viewNorm = normalize(viewPoint.xyz  wpos);
float3 vertexColours = 0;
for(int n=0; n<vsLightCount; n++)
{
float4 lightPosition = v_lights[n*4];
float4 specularColour = v_lights[n*4+1];
float3 diffuseColour = v_lights[n*4+2].xyz;
float4 attenuation = v_lights[n*4+3];
float3 lightDif = lightPosition.xyz  wpos*lightPosition.w;
float len2 = dot(lightDif,lightDif);
float len = sqrt(len2);
lightDif /= len;
float diffuse = saturate(dot(lightDif,normal));
float specular = saturate(dot(normalize(lightDif+viewNorm),normal));
float falloff = 1.0 / (attenuation.x + len * attenuation.y + len2 * attenuation.z);
specular = pow(specular,specularColour.w);
diffuse *= falloff;
specular *= falloff;
vertexColours += specularColour.xyz * specular + diffuseColour * diffuse;
}
colour.xyz += vertexColours;
vcolour_out = 1;
if (useVertexColours)
{
vcolour_out = vcolour;
}
}
Pixel shader:
float4 p_lights[16] : register(c0); //max 4
float4 PS_norm_specular(
float4 vcolour : TEXCOORD1,
float4 colour : TEXCOORD0,
float2 tex : TEXCOORD2,
float3 wpos : TEXCOORD3,
float3 normal : TEXCOORD4,
float3 binorm : TEXCOORD5,
float3 tangent : TEXCOORD6,
float3 viewPoint : TEXCOORD7,
const uniform int psLightCount) : COLOR
{
float4 normalMap = tex2D(CustomNormalMapSampler,tex);
float3x3 normalSpace = float3x3(tangent,binorm,normal);
normal = normalize(mul(normalMap.xyz0.5,normalSpace));
float3 viewNorm = normalize(viewPoint  wpos);
float3 pixelColour = 0;
for(int n=0; n<psLightCount; n++)
{
float4 lightPosition = p_lights[n*4];
float4 attenuation = p_lights[n*4+3];
float3 lightDif = lightPosition.xyz  wpos*lightPosition.w;
float len2 = dot(lightDif,lightDif);
float len = sqrt(len2);
lightDif /= len;
float diffuse = saturate(dot(lightDif,normal));
float specular = saturate(dot(normalize(lightDif+viewNorm),normal));
float falloff = 1.0 / (attenuation.x + len * attenuation.y + len2 * attenuation.z);
specular = pow(specular,p_lights[n*4+1].w);
diffuse *= falloff;
specular *= falloff * normalMap.w;
pixelColour += p_lights[n*4+1].xyz * specular + p_lights[n*4+2].xyz * diffuse;
}
float4 textureSample = tex2D(CustomTextureSampler,tex);
colour.xyz += pixelColour;
return textureSample * colour * vcolour;
}
Note the VS may be reworked a bit in a future version (but it won't look any different). The PS without specular is much less complex as well (it's in Material.fx)


Jun 24, 2009 at 11:17 AM
Edited Jun 24, 2009 at 11:23 AM

OK, I've implemented the two methods into a technique, but when I run it applied to our model, I get an unhandled InvalidOperationException  "Cannot use shader of type 'QAT.Shaders.Shiney.Shine' with Vertices<System.Byte> object as it does
not include element 'Tangent0'".
Here is our override class
class ShaderOverride : ModelInstanceShaderProvider
{
public override bool ProviderModifiesWorldMatrixInBeginDraw
{
get {return false; }
}
public override bool BeginGeometryShaderOverride(DrawState state, GeometryData geometry, MaterialLightCollection lights)
{
QAT.Shaders.Shiney.Shine shader = null;
shader = state.GetShader<QAT.Shaders.Shiney.Shine>();
shader.Ambient = new Vector4(1.0f, 1.0f, 1.0f, 1.0f);
shader.CustomTexture = geometry.MaterialShader.TextureMap;
shader.CustomNormalMap = geometry.MaterialShader.NormalMap;
shader.Bind(state);
return true;
//return false;
}
}
When I try this with different shaders from Material.fx, I can manage to get a bindpose model, with a texture and the amibient light applied  is all the information the shader requires being passed in automatically?
Also, how do I pass in lighting information into the shader?
Man, this got long fast!
Thanks for your help so far!
Dave



Might help if I actually post the shader too
//Lol, shader?
float4x4 world : WORLD;
float4x4 worldViewProj : WORLDVIEWPROJECTION;
float4x4 viewProj : VIEWPROJECTION;
float4 viewPoint : VIEWPOINT;
float4 ambient;
//format is position,specular,diffuse,attenuation
float4 v_lights[24]; //max 6
float4 blendMatrices[72*3];
float4 p_lights[16]; //max 4
texture2D CustomTexture;
sampler2D CustomTextureSampler = sampler_state
{
Texture = (CustomTexture);
};
texture2D CustomNormalMap;
sampler2D CustomNormalMapSampler = sampler_state
{
Texture = (CustomNormalMap);
};
void VS_norm_blend(
float4 pos_in : POSITION,
float3 normal : NORMAL, // z
float3 binorm : BINORMAL, // y
float3 tangent : TANGENT, // x
out float4 pos_out : POSITION,
float4 vcolour : COLOR,
float2 tex0 : TEXCOORD0,
out float4 colour : TEXCOORD0,
out float2 tex : TEXCOORD2,
out float4 wpos : TEXCOORD3,
out float4 vcolour_out : TEXCOORD1,
out float4 onormal : TEXCOORD4,
out float4 obinormal : TEXCOORD5,
out float4 otangent : TEXCOORD6,
float4 weights : BLENDWEIGHT,
float4 indices : BLENDINDICES,
out float4 viewPointOut : TEXCOORD7,
uniform int vsLightCount,
uniform bool useVertexColours)
{
//don't worry about the transpose, it gets optimised out and this is actually very efficient.
float4x3 blendMatrix
= transpose(float3x4(
blendMatrices[indices.x*3+0] * weights.x + blendMatrices[indices.y*3+0] * weights.y +
blendMatrices[indices.z*3+0] * weights.z + blendMatrices[indices.w*3+0] * weights.w,
blendMatrices[indices.x*3+1] * weights.x + blendMatrices[indices.y*3+1] * weights.y +
blendMatrices[indices.z*3+1] * weights.z + blendMatrices[indices.w*3+1] * weights.w,
blendMatrices[indices.x*3+2] * weights.x + blendMatrices[indices.y*3+2] * weights.y +
blendMatrices[indices.z*3+2] * weights.z + blendMatrices[indices.w*3+2] * weights.w
));
float3 blendPosition = mul(pos_in,blendMatrix);
wpos = float4(mul(float4(blendPosition,1),world).xyz,1);
pos_out = mul(wpos,viewProj);
tex = tex0;
colour = ambient;
normal = normalize(mul(mul(normal,(float3x3)blendMatrix).xyz,(float3x3)world).xyz);
onormal = float4(normal,0);
obinormal = float4(normalize(mul(mul(binorm,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
otangent = float4(normalize(mul(mul(tangent,(float3x3)blendMatrix).xyz,(float3x3)world).xyz),0);
viewPointOut = viewPoint;
float3 viewNorm = normalize(viewPoint.xyz  wpos);
float3 vertexColours = 0;
for(int n=0; n
Thanks again!


Jun 24, 2009 at 11:23 AM
Edited Jun 24, 2009 at 11:26 AM

Sounds like the model doesn't have a normal map, in which case it won't be generating tangent frames. (or, at least, parts of it doesn't have a normal map).
You can get around it in two ways, first is to check the GeometryData passed in to the override method  and check it's material to see if there is a normal map (and use a different shader in that case)
OR, the much easier way :) you can force the model to always generate tangent frames by setting the property in the model files content properties. Set 'Generate tangent frames' to true.
The error you were getting was a bit cryptic because the only way to get vertex data from a content pipeline model is a raw byte[] array. But as you see, in debug mode I do a sanity check to make sure what you are trying to access in the shader actually
exists in the geometry data. :) Saves some really nasty bugs


Jun 24, 2009 at 11:40 AM
Edited Jun 24, 2009 at 12:03 PM

As you probably noticed, the shaders both take in a constant integer to say how many lights there are. This internally is how MaterialShader works, creating a whole load of variations for each number of lights.
So supporting a variable number of lights would mean implementing a similar (and very complex) system. I'll have some stuff in the next update that would make this easier (better flow control support for SM3).
As for the actual array values set (for the light data) you can see the format is 4x4x3x3 in position (w indicates directional), specular RGB+power, diffuse RGB and finally attenuation CLQ (constant, linear, quadratic). It's all setup in the material shader
code (as you might guess).



OK! Sorry to keep bothering you, but now the model is not rendered at all. I pilfered some vertex and pixel shaders that don't have normal mapping here it is
//Lol, shader?
float4x4 world : WORLD;
float4x4 worldViewProj : WORLDVIEWPROJECTION;
float4x4 viewProj : VIEWPROJECTION;
float4 viewPoint : VIEWPOINT;
float4 ambient;
//format is position,specular,diffuse,attenuation
float4 v_lights[24]; //max 6
float4 blendMatrices[72*3];
float4 p_lights[16]; //max 4
texture2D CustomTexture;
sampler2D CustomTextureSampler = sampler_state
{
Texture = (CustomTexture);
};
//***************************************************
// Pilfered Shader Code Goes Here
//***************************************************
technique Shine
{
pass
{
VertexShader = compile vs_3_0 VS_blend(6,true, false, true);
PixelShader = compile ps_3_0 PS_specular(2, true);
}
}
The override function is the same as the one above except with the normal mapping stuff removed.
I'm concerned that the issue might be hard to diagnose short of sending you everything we've got, but any help is greatly appreciated!
Thanks again for the help!



did you pull the normal mapping code out, or did you copy the nonnormal map versions from MaterialShader?



It uses VS_blend from blendmaterial.fx and PS_specular from material.fx, so they should be the nonnormal map versions. Should I have pulled the normal stuff out of the normal mapping shaders?


Jun 24, 2009 at 1:21 PM
Edited Jun 25, 2009 at 12:44 AM

No, that should be correct.
*EDIT*
Doh. In the shader provider class, if you want animation support you need to override the two BeginDraw() methods. The method that is called depends if the model is animated  in which case the overload with animation bone data is called.
Depending on this, you will then use an animated or nonanimated shader  and in the animated case, you will need to copy in the animation bone data to the blendMatrices array (this would be why nothing is showing up).
The only point I'll make is there is a small issue in that example, where it'll copy the animation bones redundantly if a mesh has more than one piece of geometry.
The shadow mapping example does something very similar to this.


Jun 24, 2009 at 4:50 PM
Edited Jun 25, 2009 at 10:08 AM

Success!
Thanks for your help, we can now add in our own effects! I'll post a screenshot a bit later...
Here it is!
http://daretobedigital.com/AdminPanel/team_diary/team_diary_images/1875_1big.jpg
Thanks again for the help hopefully everything else will go more smoothly.



Hey that's looking good :)
I hope (otherwise) things have have been working out well. Let me know if anything is tricky or proves troublesome.
The glow is the effect you have been creating? If so, that's the perfect sort of effect to render the model a second time (with alpha blending). That way you can still keep the complex lighting intact without having to recreate it.
Good luck! And feel free to post more pics if you make something cool :)

