stl.rst 9.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251
  1. STL containers
  2. ##############
  3. Automatic conversion
  4. ====================
  5. When including the additional header file :file:`pybind11/stl.h`, conversions
  6. between ``std::vector<>``/``std::deque<>``/``std::list<>``/``std::array<>``/``std::valarray<>``,
  7. ``std::set<>``/``std::unordered_set<>``, and
  8. ``std::map<>``/``std::unordered_map<>`` and the Python ``list``, ``set`` and
  9. ``dict`` data structures are automatically enabled. The types ``std::pair<>``
  10. and ``std::tuple<>`` are already supported out of the box with just the core
  11. :file:`pybind11/pybind11.h` header.
  12. The major downside of these implicit conversions is that containers must be
  13. converted (i.e. copied) on every Python->C++ and C++->Python transition, which
  14. can have implications on the program semantics and performance. Please read the
  15. next sections for more details and alternative approaches that avoid this.
  16. .. note::
  17. Arbitrary nesting of any of these types is possible.
  18. .. seealso::
  19. The file :file:`tests/test_stl.cpp` contains a complete
  20. example that demonstrates how to pass STL data types in more detail.
  21. .. _cpp17_container_casters:
  22. C++17 library containers
  23. ========================
  24. The :file:`pybind11/stl.h` header also includes support for ``std::optional<>``
  25. and ``std::variant<>``. These require a C++17 compiler and standard library.
  26. In C++14 mode, ``std::experimental::optional<>`` is supported if available.
  27. Various versions of these containers also exist for C++11 (e.g. in Boost).
  28. pybind11 provides an easy way to specialize the ``type_caster`` for such
  29. types:
  30. .. code-block:: cpp
  31. // `boost::optional` as an example -- can be any `std::optional`-like container
  32. namespace pybind11 { namespace detail {
  33. template <typename T>
  34. struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {};
  35. }}
  36. The above should be placed in a header file and included in all translation units
  37. where automatic conversion is needed. Similarly, a specialization can be provided
  38. for custom variant types:
  39. .. code-block:: cpp
  40. // `boost::variant` as an example -- can be any `std::variant`-like container
  41. namespace pybind11 { namespace detail {
  42. template <typename... Ts>
  43. struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {};
  44. // Specifies the function used to visit the variant -- `apply_visitor` instead of `visit`
  45. template <>
  46. struct visit_helper<boost::variant> {
  47. template <typename... Args>
  48. static auto call(Args &&...args) -> decltype(boost::apply_visitor(args...)) {
  49. return boost::apply_visitor(args...);
  50. }
  51. };
  52. }} // namespace pybind11::detail
  53. The ``visit_helper`` specialization is not required if your ``name::variant`` provides
  54. a ``name::visit()`` function. For any other function name, the specialization must be
  55. included to tell pybind11 how to visit the variant.
  56. .. warning::
  57. When converting a ``variant`` type, pybind11 follows the same rules as when
  58. determining which function overload to call (:ref:`overload_resolution`), and
  59. so the same caveats hold. In particular, the order in which the ``variant``'s
  60. alternatives are listed is important, since pybind11 will try conversions in
  61. this order. This means that, for example, when converting ``variant<int, bool>``,
  62. the ``bool`` variant will never be selected, as any Python ``bool`` is already
  63. an ``int`` and is convertible to a C++ ``int``. Changing the order of alternatives
  64. (and using ``variant<bool, int>``, in this example) provides a solution.
  65. .. note::
  66. pybind11 only supports the modern implementation of ``boost::variant``
  67. which makes use of variadic templates. This requires Boost 1.56 or newer.
  68. Additionally, on Windows, MSVC 2017 is required because ``boost::variant``
  69. falls back to the old non-variadic implementation on MSVC 2015.
  70. .. _opaque:
  71. Making opaque types
  72. ===================
  73. pybind11 heavily relies on a template matching mechanism to convert parameters
  74. and return values that are constructed from STL data types such as vectors,
  75. linked lists, hash tables, etc. This even works in a recursive manner, for
  76. instance to deal with lists of hash maps of pairs of elementary and custom
  77. types, etc.
  78. However, a fundamental limitation of this approach is that internal conversions
  79. between Python and C++ types involve a copy operation that prevents
  80. pass-by-reference semantics. What does this mean?
  81. Suppose we bind the following function
  82. .. code-block:: cpp
  83. void append_1(std::vector<int> &v) {
  84. v.push_back(1);
  85. }
  86. and call it from Python, the following happens:
  87. .. code-block:: pycon
  88. >>> v = [5, 6]
  89. >>> append_1(v)
  90. >>> print(v)
  91. [5, 6]
  92. As you can see, when passing STL data structures by reference, modifications
  93. are not propagated back the Python side. A similar situation arises when
  94. exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
  95. functions:
  96. .. code-block:: cpp
  97. /* ... definition ... */
  98. class MyClass {
  99. std::vector<int> contents;
  100. };
  101. /* ... binding code ... */
  102. py::class_<MyClass>(m, "MyClass")
  103. .def(py::init<>())
  104. .def_readwrite("contents", &MyClass::contents);
  105. In this case, properties can be read and written in their entirety. However, an
  106. ``append`` operation involving such a list type has no effect:
  107. .. code-block:: pycon
  108. >>> m = MyClass()
  109. >>> m.contents = [5, 6]
  110. >>> print(m.contents)
  111. [5, 6]
  112. >>> m.contents.append(7)
  113. >>> print(m.contents)
  114. [5, 6]
  115. Finally, the involved copy operations can be costly when dealing with very
  116. large lists. To deal with all of the above situations, pybind11 provides a
  117. macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
  118. conversion machinery of types, thus rendering them *opaque*. The contents of
  119. opaque objects are never inspected or extracted, hence they *can* be passed by
  120. reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
  121. the declaration
  122. .. code-block:: cpp
  123. PYBIND11_MAKE_OPAQUE(std::vector<int>);
  124. before any binding code (e.g. invocations to ``class_::def()``, etc.). This
  125. macro must be specified at the top level (and outside of any namespaces), since
  126. it adds a template instantiation of ``type_caster``. If your binding code consists of
  127. multiple compilation units, it must be present in every file (typically via a
  128. common header) preceding any usage of ``std::vector<int>``. Opaque types must
  129. also have a corresponding ``class_`` declaration to associate them with a name
  130. in Python, and to define a set of available operations, e.g.:
  131. .. code-block:: cpp
  132. py::class_<std::vector<int>>(m, "IntVector")
  133. .def(py::init<>())
  134. .def("clear", &std::vector<int>::clear)
  135. .def("pop_back", &std::vector<int>::pop_back)
  136. .def("__len__", [](const std::vector<int> &v) { return v.size(); })
  137. .def("__iter__", [](std::vector<int> &v) {
  138. return py::make_iterator(v.begin(), v.end());
  139. }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
  140. // ....
  141. .. seealso::
  142. The file :file:`tests/test_opaque_types.cpp` contains a complete
  143. example that demonstrates how to create and expose opaque types using
  144. pybind11 in more detail.
  145. .. _stl_bind:
  146. Binding STL containers
  147. ======================
  148. The ability to expose STL containers as native Python objects is a fairly
  149. common request, hence pybind11 also provides an optional header file named
  150. :file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
  151. to match the behavior of their native Python counterparts as much as possible.
  152. The following example showcases usage of :file:`pybind11/stl_bind.h`:
  153. .. code-block:: cpp
  154. // Don't forget this
  155. #include <pybind11/stl_bind.h>
  156. PYBIND11_MAKE_OPAQUE(std::vector<int>);
  157. PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
  158. // ...
  159. // later in binding code:
  160. py::bind_vector<std::vector<int>>(m, "VectorInt");
  161. py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
  162. When binding STL containers pybind11 considers the types of the container's
  163. elements to decide whether the container should be confined to the local module
  164. (via the :ref:`module_local` feature). If the container element types are
  165. anything other than already-bound custom types bound without
  166. ``py::module_local()`` the container binding will have ``py::module_local()``
  167. applied. This includes converting types such as numeric types, strings, Eigen
  168. types; and types that have not yet been bound at the time of the stl container
  169. binding. This module-local binding is designed to avoid potential conflicts
  170. between module bindings (for example, from two separate modules each attempting
  171. to bind ``std::vector<int>`` as a python type).
  172. It is possible to override this behavior to force a definition to be either
  173. module-local or global. To do so, you can pass the attributes
  174. ``py::module_local()`` (to make the binding module-local) or
  175. ``py::module_local(false)`` (to make the binding global) into the
  176. ``py::bind_vector`` or ``py::bind_map`` arguments:
  177. .. code-block:: cpp
  178. py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
  179. Note, however, that such a global binding would make it impossible to load this
  180. module at the same time as any other pybind module that also attempts to bind
  181. the same container type (``std::vector<int>`` in the above example).
  182. See :ref:`module_local` for more details on module-local bindings.
  183. .. seealso::
  184. The file :file:`tests/test_stl_binders.cpp` shows how to use the
  185. convenience STL container wrappers.