The Z-Wave interface I’m working on is an inherently asynchronous beast. Callbacks abound, and the use of lambda functions makes that much easier to deal with. This fact led me to select C++11 as the language standard for the project.

And then I added automatic memory management with std::shared_ptr<>, and it all fell apart.

Why, you ask?

Background: Lambdas in C++

The naïve view of lambdas is that they’re little more than function pointers in a fancy package. Basically, in C++11, you can do something like this and it will work as expected:

#include <iostream>

typedef void (*func)();

int main(int argc, char **argv) {
  func myfunc = [](){ std::cout << "did something" << std::endl; };
  myfunc();
}

So long as those square brackets have nothing between them, this will work fine; the lambda is compatible with a standard function pointer. But if you want to do something more complicated, like capturing variables from the parent scope, things have to look a little different:

#include <iostream>
#include <functional>

typedef std::function<void()> func;

int main(int argc, char **argv) {
  int mynum = 42;

  func myfunc = [mynum](){ std::cout << "the answer is " << mynum << std::endl; };
  myfunc();
}

This one captures the value of mynum, and will use it when the lambda is called. Note the change in the typedef, however: capturing something changes the datatype of the lambda, and you can no longer assign it to a standard function pointer.

While it may not be immediately obvious, the implications of this are extremely important!

The Problem With Shared Pointers

The Z-Wave protocol is inherently asynchronous, and as a result I use a lot of callbacks in my code. A typical pattern might be something along these lines:

class my_class {
// ...
public:
  typedef std::function<void()> callback;
  void on_complete(callback cb) { complete_callback = cb; }
private:
  callback complete_callback;
// ...
};

  // ...
  std::shared_ptr<my_class> obj = std::make_shared<my_class>();
  obj->on_complete([obj]() {
    obj->clean_something_up();
  });
  executor->submit(obj);
  // ...

Note that the shared pointer is captured by the callback. I use a similar pattern in a number of places to avoid having to pass pointers around through the callback signature. Unfortunately, this pattern is effectively broken.

It leaks like a sieve.

The object allocated by that std::make_shared<> call? It’ll never get deleted.

Wait, what?

The Truth About Lambdas

Here’s where we come back to that change in type signature when we capture variables with a lambda.

A lambda that captures nothing is simply a function pointer; it doesn’t need to pass any data around. A lambda that captures variables, on the other hand, needs a place to put them. When you create a lambda function that captures variables, you are in essence creating an entirely new kind of object, and the function pointer is just one member of that object; the captured variables are the others.

This means that as long as something holds a reference to your lambda, it also holds indirect references to copies of each element of captured data.

Normally this is not a big deal. Captures are typically copies, so you get a copy of some object and don’t worry about it. Even if you’re not too careful, nothing bad happens outside a bit of extra memory use.

In matters of std::shared_ptr<>, however, it’s a little bit different.

If you capture a shared pointer with a lambda, that lambda will contain a shared pointer, pointing to the same object as the original. It’s just like manually creating two shared pointers. The copy gets passed around with the lambda, ensuring that there’s an extra reference to that object.

If you then store such a lambda on the object for which it has captured a shared pointer, you’ve just created a scenario where the object owns a reference to itself, and thus can never be deleted.

Fun times.

Fixing The Problem

To fix this, first establish a rule: never capture a std::shared_ptr<> in a lambda. It’s an accident waiting to happen, and you’ll be forever trying to chase down that errant reference. When it bites you, it will bite you hard.

There are two safe alternatives you can use.

The most obvious (and the one I do not recommend) is to capture the raw pointer for your lambda to use. This has a major downside, though, in that the object might be deleted before you try to reference it, and you have no way of knowing it. At least it won’t hold an extra reference, though.

The better way (IMO) is to use std::weak_ptr<>.

The std::weak_ptr<> object, by definition, does not hold a reference to the target. It does, however, track the target, so you can tell if the target was deleted. This is perfect for most scenarios where lambdas might be used.

A simple solution would look something like this:

class my_class {
// ...
public:
  typedef std::function<void()> callback;
  void on_complete(callback cb) { complete_callback = cb; }
private:
  callback complete_callback;
// ...
};

  // ...
  std::shared_ptr<my_class> obj = std::make_shared<my_class>();
  std::weak_ptr<my_class> weak_obj(obj);

  obj->on_complete([weak_obj]() {
    auto obj = weak_obj.lock();
    if (obj) {
      obj->clean_something_up();
    }
  });
  executor->submit(obj);
  // ...

Note that the only change is to the lambda itself. Instead of capturing the shared pointer, it captures a weak reference to the object it points at.

The use of the weak pointer eliminates the potential for self-referencing, and the only cost is that you need to verify the pointer before using it to avoid an accidental null pointer dereference. You don’t need to be concerned about adding an extra reference any longer; even if you store the lambda on the object the shared pointer references, everything will still work perfectly.

It’s an easy solution once you know the problem exists.

And with that, I’m off to do more work on my little toy…