Skip to content

Type Coercion Exception while performing a MAC operation with signed values over unsigned buffers #8644

Closed

Description

@crmartin781

Hi there -

I've re-worked my RGB -> YUV420 conversion into a Generator class defined as follows:

using conv_mat_row = std::array<int, 3>;

// Define a Generator to convert RGB to planar YUV420
class RGBtoYUV420 : public Halide::Generator<RGBtoYUV420> {
  Var x,y,c;

  /* note:  if you're wondering why the 'excessive' use of curly braces, 
   * look here:  https://stackoverflow.com/questions/12844475/why-cant-simple-initialize-with-braces-2d-stdarray 
   */
  std::array<conv_mat_row, 3> conv1 = { {
         { {  66,  129,  25 } },
	 { { -38, -74,  112 } },
	 { { 112, -94, -18   } } 
    } };

//etc 
  
public:
  Input<Buffer<uint8_t, 3>> input{"rgb_input"};
  Output<Buffer<uint8_t>> outY{"i420_Y_output"};
  Output<Buffer<uint8_t>> outU{"i420_U_output"};
  Output<Buffer<uint8_t>> outV{"i420_V_output"};


  void generate() {
    
    Func y_part("Luma"), u_part("Chroma-U"), v_part("Chroma-V");
    Func avg_rgb("avg_rgb"), avg_rgb_x("avg_rgb_x");

    Expr yval = input(x, y, c);
    Expr uval = input(x, y, c);
    Expr vval = input(x, y, c);
    
    //more code, that is not of issue to the discussion...

    //apply yval(r, g, b) transform, 1-1 sampled from image
    yval = conv1[0][0] * input(x, y, 0) + conv1[0][1] * input(x, y, 1) + conv1[0][2] * input(x, y, 2);

    //etc...

    //apply uval(r,g,b) transform, with averaged rgb samples, over 2x2 sub-areas
    uval = conv1[1][0] * avg_rgb(2 * x, 2 * y, 0) - conv1[1][1] * avg_rgb(2 * x, 2 * y, 1) + conv1[1][2] * avg_rgb(2 * x, 2 * y, 2);

   //similar code for vval

 // Output planar YUV420
    outY(x,y) = y_part(x,y);
    outU(x,y) = u_part(x,y);
    outV(x,y) = v_part(x,y);
  } //end generate() method

 void schedule() {
  }
};

HALIDE_REGISTER_GENERATOR(RGBtoYUV420, rgb2i420)

Where things start to go sideways, is during the evaluation of uval Expr. Due to the fact that conv1[1][0] is < 0, the generator throw the following exception message:

Unhandled exception: Error: Integer constant -38 will be implicitly coerced to type uint8, which changes its value to (uint8)218.

I tried Halide::cast<int8_t>(input(x,y,c)) the Expr's, but that didn't seem to work.
So, how do I get past this coercion problem, properly?

Thanks, Charles.

Activity

  1. crmartin781 commented on Jun 11, 2025

    @crmartin781
    Author

    So, I got past my immediate problems with type-coercion, with careful use of Halide::cast<>.
    However, the halide-generator that results from my build throws again.
    Upon isolating, I discovered this code fragment is the issue:

    Expr roundup_and_add_bias(Expr val, int8_t bias)
      {
        val = val + 127;
        //val = val / cast<int16_t>(256); //the generator likes this...
        val = val >> 8; //the generator throws...WHY?
        val = val + bias + 1;
        return val;
      }
    

    I get this exception:

    Unhandled exception: Error: Signed integer overflow occurred during constant-folding. Signed integer overflow for int32 and int64 is undefined behavior in Halide.
    

    To me, this smells like a bug, since the Expr class has trouble dealing with certain type-combinations of the right-shift operator (>>), and possibly left-shift (<<).

    Is there a document explaining these exception objects?

  2. mcourteaux commented on Jun 11, 2025

    @mcourteaux
    Contributor

    It's important to know the type of the val argument. When that is still uint8, then shifting 8 bits left or right, fully wipes the number. This is checked in Simplify_Call.cpp at line 148.

  3. locked and limited conversation to collaborators on Jun 11, 2025
  4. converted this issue into a discussion #8645 on Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions