2D propagate_adjoints: 'Inverse Rendering'

[soufianekhiat]

Hello,

Currently propagate_adjoints support only scalar function. Which make sense in the context of differential programming when we only deal with a loss function.

/*
 *  Given a scalar Func with size 1, (back)propagate the gradient
 *  to all dependent Funcs, buffers, and parameters.
 *  For each Func the output depends on, and for the pure definition and
 *  each update of that Func, it generates a derivative Func stored in
 *  the Derivative.
 */

What about "Inverse Rendering"?
Current situation:

Func image;
ImageParam p;
image( x, y ) = // Do somethign with p
// Compute Derivative for each pixel
std::vector<Func> derivatives;
derivatives.reserve(width*height);
for (int i = 0; i < height; ++i)
{
    for (int j = 0; j < width; ++j)
    {
        Expr res = image(j, i);
        Func cur;
        cur() = res;
        Derivative adj_image_ij = propagate_adjoints(cur);
        derivatives.push_back( adj_image_ij( p ) );
    }
}
// Use derivative to generate the image = d image / dp.
std::vector<float> dimage_dp_result(width*height);
for (int i = 0; i < height; ++i)
{
    for (int j = 0; j < width; ++j)
    {
        Buffer<float> singlePixel = Buffer<float>::make_scalar(&dimage_dp_result[i*width + j], "single_pixel");
        dimage_dp[i*width + j].realize(singlePixel, target);
    }
}

'Nice to have':

Derivative adj_image_ij = propagate_adjoints(cur, { x, y });
Func dimage_dp = adj_image_ij(p); // Here dimage_dp is a 2D function
Buffer<float> wholeImage(&dimage_dp_result[0], {width, height}, "single_pixel");
dimage_dp.realize(wholeImage, target);

Which allow us to implement algorithm related to 'Inverse Rendering' (cf. Mitsuba and enoki) without needing to store one Func per pixel.