Skip to content

Possible persistent handle leak when producing with opaque and no delivery callback #1147

Open

Description

@OvOhao

Possible persistent handle leak when producing with opaque and no delivery callback

I found a possible native and JavaScript-object retention leak in produce() when an opaque value is passed without a delivery callback.

Files: lib/producer.js, src/producer.cc, src/callbacks.cc

Functions: Producer.prototype.produce, Producer::NodeProduce, Delivery::dr_cb, DeliveryReportDispatcher::Flush

Relevant public API:

Producer.prototype.produce = function(topic, partition, message, key, timestamp, opaque, headers) {
  // ...
  return this._errorWrap(
    this._client.produce(topic, partition, message, key, timestamp, opaque, headers));
};

Delivery callbacks are only registered when dr_cb or dr_msg_cb is configured:

if (dr_msg_cb || dr_cb) {
  this._cb_configs.event.delivery_cb = function(err, report) {
    this.emit('delivery-report', err, report);
  }.bind(this);
}

The native produce() implementation allocates a persistent handle for every
defined opaque argument:

void* opaque = NULL;
if (info.Length() > 5 && !info[5]->IsUndefined()) {
  opaque = new Nan::Persistent<v8::Value>(info[5]);
}

The synchronous enqueue-error path cleans it up:

if (error_code != 0 && opaque) {
  Nan::Persistent<v8::Value> *persistent =
    static_cast<Nan::Persistent<v8::Value> *>(opaque);
  persistent->Reset();
  delete persistent;
}

The successful delivery cleanup happens later in Flush():

if (event.opaque) {
  Nan::Persistent<v8::Value> *persistent =
    static_cast<Nan::Persistent<v8::Value> *>(event.opaque);
  v8::Local<v8::Value> object = Nan::New(*persistent);
  Nan::Set(jsobj, Nan::New("opaque").ToLocalChecked(), object);

  persistent->Reset();
  delete persistent;
}

But the delivery callback returns before creating a DeliveryReport when no JS
delivery callback is registered:

void Delivery::dr_cb(RdKafka::Message &message) {
  if (!dispatcher.HasCallbacks()) {
    return;
  }

  DeliveryReport msg(message, m_dr_msg_cb);
  // ...
}

So a successfully enqueued message with opaque can bypass both cleanup paths:
it was accepted by librdkafka, but no delivery report is queued to release the
Nan::Persistent.

Suggested fix: reclaim message.msg_opaque() even when there are no JS delivery
callbacks, preferably on the Node/V8 thread. Another option is to avoid
allocating the persistent when no delivery callback path can ever expose it.


Related: #953 (closed) reported the sibling case — opaque leaking when
produce() itself errors. That synchronous enqueue-error path is now cleaned up
(the if (error_code != 0 && opaque) { ...delete... } block above). This report
covers the remaining path: a message that is successfully enqueued while no JS
delivery callback is registered
, where Delivery::dr_cb early-returns before a
DeliveryReport is built, so Flush() never releases the persistent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions