This project is read-only.

Interesting ShaderSystem bug

Oct 1, 2011 at 6:12 AM


I recently ran into what seems like a bug in the ShaderSystem component. I've narrowed the issue down to a very simple test case to point out the issue.

For a shader like the following:

//CompilerOptions = DisableGenerateExtensions
void BadFunc() { } void A() { BadFunc(); } float4 VS() : POSITION { BadFunc(); A(); return 1; } float4 PS() : COLOR0 { return 0; } technique tec0 { pass Pass2 { VertexShader = compile vs_3_0 VS(); PixelShader = compile ps_3_0 PS(); } }

The ShaderSystem compiler internally generates:
void /*8*/ _A__VS() /*8*/ { _BadFunc__VS(); /*8*/ } void /*4*/ _BadFunc__VS() /*4*/ { } float4 /*10*/ _VS__VS():POSITION /*10*/ { _BadFunc__VS(); /*12*/ _A__VS(); /*13*/ return /*15*/ 1; /*15*/ } float4 /*18*/ _PS__PS():COLOR0 /*18*/ { return /*20*/ 0; /*20*/ } technique Shader { pass { VertexShader = compile vs_3_0 _VS__VS(); PixelShader = compile ps_3_0 _PS__PS(); } }

As you can see, the function declaration ordering is modified due to the function invoke order, which then in turn causes the compiler errors out with the following message:
// XenFX
// Assembly = Xen.Graphics.ShaderSystem.CustomTool, Version=, Culture=neutral, PublicKeyToken=null
// SourceFile = test.fx
// Namespace = Shaders
// Error generating shader:
//XenFX Platform Technique Extraction Failed: (line: 0 col: 0)
// Error generating shader:
//Errors compiling test.fx: (line: 0 col: 0)
// Error generating shader:
//(9,): error X3004: undeclared identifier '_BadFunc__VS' (line: 8 col: 0)

My understanding is that the problem stems from the fact that the ShaderSystem compiler is tracking the hlsl function dependencies from the root (VS/PS) level down as it recognizes that a function is used and unfortunately, in doing so, it accidentally shuffles the order of the functions as the original file ordering requires that a function be defined before it's used in the proceeding hlsl functions.
My initial guess is that, in order to fix this, it'd be best to break apart the function dependency tracking and internal "string building of the internal hlsl code" into separate steps, instead of doing them interleaved.
1) Parse the function dependecies and figure out which functions are being used for a given VS or PS. Results in the method list without actually populating the "StringBuilder".
2) Sort the methods to follow the same order they had in the original hlsl file.
2) Populate the StringBuilder that will generate the final internal shader using the extracted dependent methods.
Anyway. This is what I'm thinking. Any input would be appreciated because there might be a way easier way to fix this issue, and I might be just overlooking it.