Hi,
I do the following computation in a pixel shader:
const float texel = 1.0 / 128.0;
result = floor(In.texcoord.w / 16.0) / 64.0 + texel
In.texcoord.w containes an integer value between 0 and 299, stored as a float, interpolated over the triangle, where each vertex stores the same value, so in practice there shouldn't be any interpolation happening.
When I output result as a color, I notice distortions over the triangle and a flickering when I move the mouse, for some values of In.texcoord.w. But actually the result should be constant for each of the triangles pixels because the computation uses the same values for each pixel - at least it should.
Any idea of what causes this Precision issues Is there any remedy
Nico
pixel shader computation: precision problem?
The Omnipotent
Thanks. I just didn't know. That was probably to simple to pop into my mind... lol ;-)
By the way, do you know why the round function is so slow
SQL2K5
As you have this problem with RefRast too you can try to use the shader debugger.
虫豸
Cerberuss
Well, yes, but I have to do some updates of my programming environment for that. I'm still using the VS2005 beta... lol
I plan to do this, but probably in a few weeks, so up to then I have to find another solution. I implemented a convenient CPU environment for executing hlsl pixel shaders. I could use that for debugging, but it will not reproduce the GPU precision issues.
Joao Ferreirinha
Your debugging project sounds interesting!
I just found the cause to the distortion. As I suspected it was due to imprecision during interpolation. When I round the values of In.texcoord.w before I use them the problem disappears.
Unfortunately rounding is a costly operation. I do it using this code:
float tmp = frac( In.texcoord.w ) ;
if ( tmp > 0.0 )
{
if (tmp < 0.5) tmp = floor(In.texcoord.w);
else tmp = ceil(In.texcoord.w);
}
else tmp = In.texcoord.w;
Then I use tmp in my computation. When I use the hlsl round fuction my frame count drops even more...
Anyway, thanks for your ideas Ralf. ;-)
Latika
Yes, equal inputs should get equal outputs but GPUs are nor CPU and care less about floating point accuracy.
Maybe some additional information about the shader profiles and hardware you are using could be helpful.
Have you try this with RefRast
You say that your mouse moves caused flickering. Do you change anything depending on the current mouse position
Maybe you should try to output your W coordinate to a F32 render target to check if the input is already wrong.
Shreyas Patel
jpkuzma
The debugger use Refrast and can’t show you GPU precision issues too because it runs on the CPU. I was playing around with a shader debugger that runs on the GPU. It splits the shader in parts and stores the intermediate results in FP32 render targets that are transferred to the CPU to show them for every pixel. But unfortunately this is only one of my playtime projects and not ready to use.