HLSL pixel shader: problem with tex coord computation/texture lookup

I'm sitting on a strange problem for half a day now and can't figure out why things are happening as they happen. :)

What I'm trying to do is to look up a texture with entries of size 3*3 texels each (g_samCurvatureTex: this is a magnified part of it, the whole tex is repeated like this - I am only interested in the blue channel right now). At first I need to compute the centre index of the entry:

#define CURV_TEX_ENTRIES 42

// p_TS.xy are texcoords in the range [0,1] that vary smoothly over the surface

int xCM = (int)(p_TS.x * CURV_TEX_ENTRIES);  

int yCM = (int)(p_TS.y * CURV_TEX_ENTRIES);

xCM = xCM * 3 + 1;

yCM = yCM * 3 + 1;

Then I compute its texture coordinate, adding a direction vector of length one texel to finally get the actual (filtered) texture value:

float2 tcCurv;

tcCurv.x = (xCM + 0.5) / (float)(CURV_TEX_SIZE) + viewDirXY_CurvTex.x;   //viewDirXY_CurvTex is a vector of one texel length in g_samCurvatureTex

tcCurv.y = (yCM + 0.5) / (float)(CURV_TEX_SIZE) + viewDirXY_CurvTex.y;

float curvTex = tex2D( g_samCurvatureTex, tcCurv ).b;

Unfortunately this does not exactly work as I expect it. Cutouts of the results are shown here:

This I want. (blue is drawn read here)

This I get. (blue is drawn read here)

So there is a kind of raster that corresponds with the number of texture entries, 42 in this case. The raster lines are each located in the beginning of a space covered by one 3x3 entry in the tex. Does anybody have an idea what causes them to appear I'm quite helpless right now.

Thanks a lot in advance.

Nico

 

 

 

 



Answer this question

HLSL pixel shader: problem with tex coord computation/texture lookup

  • P. RICHARDOT

    >Why do you add 0.5 to xCM and yCM

    I found out that this is necessary to get exactly the centre of the texel. For instance the first texel begins at 0.0 and ends at 1/texSize.

    >Can you post an image of texture, too

    The first link in the post is a link to a tiny part of the texture. Currently what you can see there gets repeated over the whole texture. And as I mentioned, I'm only interested in the blue channel.

    This is a magnified part of the texture with the blue channel only: Tex.b

    Here is the complete texture.

     


  • Shaun Mulligan

    Well, yes, I read that article before. But it starts with:

    "When rendering 2D output using pre-transformed vertices..."

    But I'm rendering in 3D and I am not using pre-transformed vertices. So I don't think it applies in my case. But anyway, I will read it again...

     

    P.S.: Hab grad zufallig Deinen Artikel zu Grafikartentreibern in der PC Games Hardware gelesen - falls Du der selbe Ralf Kornmann bist. :)

     

     


  • C.T

    >You need to subtract (1.0/render target width) and(1.0/render target height).

    To subtract from what I mean, I guess the cause for the raster lines should be somewhere in the index computation or texture look up. Actually the indices seem ok, that I see when I render them as pixel output colors. And finally I don't use the values I receive from the texture for mapping them onto the object, they are used in a ray tracing algorithm so anything happening during rasterization shouldn't be the cause of the problem. The 0.5 offset I only use to get the indices into the texture right. That seems very logical to me, but I must have missed something... 

    >Maybe you should add a floor or ceil to your index calculation because current pixel shader hardware does not support int as a native type.

    I already tried that, no success.

    >Can you post the assembler code of the whole shader

    I have only the HLSL code. How can I get the assembler code Btw the shader is a 3.0 shader and fairly long.


  • ansic

    >Does this work the same with the reference device

    Yes.

    >Any filtering in your sampler

    I'm using bilinear filtering:

    sampler g_samCurvatureTex =

    sampler_state

    {

    Texture = <g_CurvatureTex>;

    MinFilter = Linear;

    MagFilter = Linear;

    MipFilter = Point;

    };


  • Lucho_1981

    No, it doesn't.
  • JP&amp;#35;

    Why do you add 0.5 to xCM and yCM

    Can you post an image of texture, too



  • RvD

    Two questions:

    Does this work the same with the reference device

    Any filtering in your sampler


  • Ross Grayum MSFT

    Sorry I was believed that you try to map texel to pixel because your two images show no 3d object. Anyway the raster rules are always valid. It is more complex in the case you don’t have pretransformed coordinates.  You need to subtract (1.0/render target width) and (1.0/render target height).< xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

    Maybe you should add a floor or ceil to your index calculation because current pixel shader hardware does not support int as a native type.

    PS: Can you post the assembler code of the whole shader

    PSS: Ja ich bin der selbe Ralf Kornmann aka Demirug in einigen anderen Foren.



  • Tarey Wolf

    Does it look better with point sampling

  • Rob123qwe123

    The offset need to be subtract from the final position in the vertex shader. But in the case of real 3d objects a solution that doesn’t need this will be better.

    You can generate the ASM Code with FXC but HLSL is fine too. You write that your code is fairly long. In this case we should first try to find the lines that caused this problem and build a smaller shader with them. As you only post some lines in your first post I am think that you already have limited it to this part. In this case you should build a shader with only this lines.

    May I am ask what kind of effect you are try to build with this shader (part)



  • tyrus

    Hm, strange, it says in my browser the last post is by Ralf Kornmann, but there is no new post displayed... anyway, unfortunately I still haven't found the cause of the raster lines. Any ideas are welcome. :)
  • srkvellanki

    This half pixel is the default offset of the raster process that Direct3D use. You should not add it to the texture coordinate. The right way is to correct the pixel coordinates. I have try it in the past the same way as you and get some ugly results, too.

    Maybe this will help you: http://msdn.microsoft.com/library/en-us/directx9_c/Directly_Mapping_Texels_to_Pixels.asp frame=true



  • Tom Gibson

    My last post is above the post from Eyal Teler. I am not sure why the forum software has inserted it at this position.

  • HLSL pixel shader: problem with tex coord computation/texture lookup