You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Type Coercion Exception while performing a MAC operation with signed values over unsigned buffers #8644
I've re-worked my RGB -> YUV420 conversion into a Generator class defined as follows:
using conv_mat_row = std::array<int, 3>;
// Define a Generator to convert RGB to planar YUV420
class RGBtoYUV420 : public Halide::Generator<RGBtoYUV420> {
Var x,y,c;
/* note: if you're wondering why the 'excessive' use of curly braces,
* look here: https://stackoverflow.com/questions/12844475/why-cant-simple-initialize-with-braces-2d-stdarray
*/
std::array<conv_mat_row, 3> conv1 = { {
{ { 66, 129, 25 } },
{ { -38, -74, 112 } },
{ { 112, -94, -18 } }
} };
//etc
public:
Input<Buffer<uint8_t, 3>> input{"rgb_input"};
Output<Buffer<uint8_t>> outY{"i420_Y_output"};
Output<Buffer<uint8_t>> outU{"i420_U_output"};
Output<Buffer<uint8_t>> outV{"i420_V_output"};
void generate() {
Func y_part("Luma"), u_part("Chroma-U"), v_part("Chroma-V");
Func avg_rgb("avg_rgb"), avg_rgb_x("avg_rgb_x");
Expr yval = input(x, y, c);
Expr uval = input(x, y, c);
Expr vval = input(x, y, c);
//more code, that is not of issue to the discussion...
//apply yval(r, g, b) transform, 1-1 sampled from image
yval = conv1[0][0] * input(x, y, 0) + conv1[0][1] * input(x, y, 1) + conv1[0][2] * input(x, y, 2);
//etc...
//apply uval(r,g,b) transform, with averaged rgb samples, over 2x2 sub-areas
uval = conv1[1][0] * avg_rgb(2 * x, 2 * y, 0) - conv1[1][1] * avg_rgb(2 * x, 2 * y, 1) + conv1[1][2] * avg_rgb(2 * x, 2 * y, 2);
//similar code for vval
// Output planar YUV420
outY(x,y) = y_part(x,y);
outU(x,y) = u_part(x,y);
outV(x,y) = v_part(x,y);
} //end generate() method
void schedule() {
}
};
HALIDE_REGISTER_GENERATOR(RGBtoYUV420, rgb2i420)
Where things start to go sideways, is during the evaluation of uval Expr. Due to the fact that conv1[1][0] is < 0, the generator throw the following exception message:
Unhandled exception: Error: Integer constant -38 will be implicitly coerced to type uint8, which changes its value to (uint8)218.
I tried Halide::cast<int8_t>(input(x,y,c)) the Expr's, but that didn't seem to work.
So, how do I get past this coercion problem, properly?
So, I got past my immediate problems with type-coercion, with careful use of Halide::cast<>.
However, the halide-generator that results from my build throws again.
Upon isolating, I discovered this code fragment is the issue:
Expr roundup_and_add_bias(Expr val, int8_t bias)
{
val = val + 127;
//val = val / cast<int16_t>(256); //the generator likes this...
val = val >> 8; //the generator throws...WHY?
val = val + bias + 1;
return val;
}
I get this exception:
Unhandled exception: Error: Signed integer overflow occurred during constant-folding. Signed integer overflow for int32 and int64 is undefined behavior in Halide.
To me, this smells like a bug, since the Expr class has trouble dealing with certain type-combinations of the right-shift operator (>>), and possibly left-shift (<<).
Is there a document explaining these exception objects?
It's important to know the type of the val argument. When that is still uint8, then shifting 8 bits left or right, fully wipes the number. This is checked in Simplify_Call.cpp at line 148.
Activity
crmartin781 commentedon Jun 11, 2025
So, I got past my immediate problems with type-coercion, with careful use of Halide::cast<>.
However, the halide-generator that results from my build throws again.
Upon isolating, I discovered this code fragment is the issue:
I get this exception:
To me, this smells like a bug, since the Expr class has trouble dealing with certain type-combinations of the right-shift operator (>>), and possibly left-shift (<<).
Is there a document explaining these exception objects?
mcourteaux commentedon Jun 11, 2025
It's important to know the type of the
valargument. When that is still uint8, then shifting 8 bits left or right, fully wipes the number. This is checked inSimplify_Call.cppat line 148.