exceptions.rst 14 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306
  1. Exceptions
  2. ##########
  3. Built-in C++ to Python exception translation
  4. ============================================
  5. When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler
  6. that will trap C++ exceptions, translate them to the corresponding Python exception,
  7. and raise them so that Python code can handle them.
  8. pybind11 defines translations for ``std::exception`` and its standard
  9. subclasses, and several special exception classes that translate to specific
  10. Python exceptions. Note that these are not actually Python exceptions, so they
  11. cannot be examined using the Python C API. Instead, they are pure C++ objects
  12. that pybind11 will translate the corresponding Python exception when they arrive
  13. at its exception handler.
  14. .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
  15. +--------------------------------------+--------------------------------------+
  16. | Exception thrown by C++ | Translated to Python exception type |
  17. +======================================+======================================+
  18. | :class:`std::exception` | ``RuntimeError`` |
  19. +--------------------------------------+--------------------------------------+
  20. | :class:`std::bad_alloc` | ``MemoryError`` |
  21. +--------------------------------------+--------------------------------------+
  22. | :class:`std::domain_error` | ``ValueError`` |
  23. +--------------------------------------+--------------------------------------+
  24. | :class:`std::invalid_argument` | ``ValueError`` |
  25. +--------------------------------------+--------------------------------------+
  26. | :class:`std::length_error` | ``ValueError`` |
  27. +--------------------------------------+--------------------------------------+
  28. | :class:`std::out_of_range` | ``IndexError`` |
  29. +--------------------------------------+--------------------------------------+
  30. | :class:`std::range_error` | ``ValueError`` |
  31. +--------------------------------------+--------------------------------------+
  32. | :class:`std::overflow_error` | ``OverflowError`` |
  33. +--------------------------------------+--------------------------------------+
  34. | :class:`pybind11::stop_iteration` | ``StopIteration`` (used to implement |
  35. | | custom iterators) |
  36. +--------------------------------------+--------------------------------------+
  37. | :class:`pybind11::index_error` | ``IndexError`` (used to indicate out |
  38. | | of bounds access in ``__getitem__``, |
  39. | | ``__setitem__``, etc.) |
  40. +--------------------------------------+--------------------------------------+
  41. | :class:`pybind11::key_error` | ``KeyError`` (used to indicate out |
  42. | | of bounds access in ``__getitem__``, |
  43. | | ``__setitem__`` in dict-like |
  44. | | objects, etc.) |
  45. +--------------------------------------+--------------------------------------+
  46. | :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
  47. | | wrong value passed in |
  48. | | ``container.remove(...)``) |
  49. +--------------------------------------+--------------------------------------+
  50. | :class:`pybind11::type_error` | ``TypeError`` |
  51. +--------------------------------------+--------------------------------------+
  52. | :class:`pybind11::buffer_error` | ``BufferError`` |
  53. +--------------------------------------+--------------------------------------+
  54. | :class:`pybind11::import_error` | ``import_error`` |
  55. +--------------------------------------+--------------------------------------+
  56. | Any other exception | ``RuntimeError`` |
  57. +--------------------------------------+--------------------------------------+
  58. Exception translation is not bidirectional. That is, *catching* the C++
  59. exceptions defined above above will not trap exceptions that originate from
  60. Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
  61. <handling_python_exceptions_cpp>` for further details.
  62. There is also a special exception :class:`cast_error` that is thrown by
  63. :func:`handle::call` when the input arguments cannot be converted to Python
  64. objects.
  65. Registering custom translators
  66. ==============================
  67. If the default exception conversion policy described above is insufficient,
  68. pybind11 also provides support for registering custom exception translators.
  69. To register a simple exception conversion that translates a C++ exception into
  70. a new Python exception using the C++ exception's ``what()`` method, a helper
  71. function is available:
  72. .. code-block:: cpp
  73. py::register_exception<CppExp>(module, "PyExp");
  74. This call creates a Python exception class with the name ``PyExp`` in the given
  75. module and automatically converts any encountered exceptions of type ``CppExp``
  76. into Python exceptions of type ``PyExp``.
  77. It is possible to specify base class for the exception using the third
  78. parameter, a `handle`:
  79. .. code-block:: cpp
  80. py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
  81. Then `PyExp` can be caught both as `PyExp` and `RuntimeError`.
  82. The class objects of the built-in Python exceptions are listed in the Python
  83. documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_.
  84. The default base class is `PyExc_Exception`.
  85. When more advanced exception translation is needed, the function
  86. ``py::register_exception_translator(translator)`` can be used to register
  87. functions that can translate arbitrary exception types (and which may include
  88. additional logic to do so). The function takes a stateless callable (e.g. a
  89. function pointer or a lambda function without captured variables) with the call
  90. signature ``void(std::exception_ptr)``.
  91. When a C++ exception is thrown, the registered exception translators are tried
  92. in reverse order of registration (i.e. the last registered translator gets the
  93. first shot at handling the exception).
  94. Inside the translator, ``std::rethrow_exception`` should be used within
  95. a try block to re-throw the exception. One or more catch clauses to catch
  96. the appropriate exceptions should then be used with each clause using
  97. ``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set
  98. the python exception to a custom exception type (see below).
  99. To declare a custom Python exception type, declare a ``py::exception`` variable
  100. and use this in the associated exception translator (note: it is often useful
  101. to make this a static declaration when using it inside a lambda expression
  102. without requiring capturing).
  103. The following example demonstrates this for a hypothetical exception classes
  104. ``MyCustomException`` and ``OtherException``: the first is translated to a
  105. custom python exception ``MyCustomError``, while the second is translated to a
  106. standard python RuntimeError:
  107. .. code-block:: cpp
  108. static py::exception<MyCustomException> exc(m, "MyCustomError");
  109. py::register_exception_translator([](std::exception_ptr p) {
  110. try {
  111. if (p) std::rethrow_exception(p);
  112. } catch (const MyCustomException &e) {
  113. exc(e.what());
  114. } catch (const OtherException &e) {
  115. PyErr_SetString(PyExc_RuntimeError, e.what());
  116. }
  117. });
  118. Multiple exceptions can be handled by a single translator, as shown in the
  119. example above. If the exception is not caught by the current translator, the
  120. previously registered one gets a chance.
  121. If none of the registered exception translators is able to handle the
  122. exception, it is handled by the default converter as described in the previous
  123. section.
  124. .. seealso::
  125. The file :file:`tests/test_exceptions.cpp` contains examples
  126. of various custom exception translators and custom exception types.
  127. .. note::
  128. Call either ``PyErr_SetString`` or a custom exception's call
  129. operator (``exc(string)``) for every exception caught in a custom exception
  130. translator. Failure to do so will cause Python to crash with ``SystemError:
  131. error return without exception set``.
  132. Exceptions that you do not plan to handle should simply not be caught, or
  133. may be explicitly (re-)thrown to delegate it to the other,
  134. previously-declared existing exception translators.
  135. .. _handling_python_exceptions_cpp:
  136. Handling exceptions from Python in C++
  137. ======================================
  138. When C++ calls Python functions, such as in a callback function or when
  139. manipulating Python objects, and Python raises an ``Exception``, pybind11
  140. converts the Python exception into a C++ exception of type
  141. :class:`pybind11::error_already_set` whose payload contains a C++ string textual
  142. summary and the actual Python exception. ``error_already_set`` is used to
  143. propagate Python exception back to Python (or possibly, handle them in C++).
  144. .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
  145. +--------------------------------------+--------------------------------------+
  146. | Exception raised in Python | Thrown as C++ exception type |
  147. +======================================+======================================+
  148. | Any Python ``Exception`` | :class:`pybind11::error_already_set` |
  149. +--------------------------------------+--------------------------------------+
  150. For example:
  151. .. code-block:: cpp
  152. try {
  153. // open("missing.txt", "r")
  154. auto file = py::module_::import("io").attr("open")("missing.txt", "r");
  155. auto text = file.attr("read")();
  156. file.attr("close")();
  157. } catch (py::error_already_set &e) {
  158. if (e.matches(PyExc_FileNotFoundError)) {
  159. py::print("missing.txt not found");
  160. } else if (e.matches(PyExc_PermissionError)) {
  161. py::print("missing.txt found but not accessible");
  162. } else {
  163. throw;
  164. }
  165. }
  166. Note that C++ to Python exception translation does not apply here, since that is
  167. a method for translating C++ exceptions to Python, not vice versa. The error raised
  168. from Python is always ``error_already_set``.
  169. This example illustrates this behavior:
  170. .. code-block:: cpp
  171. try {
  172. py::eval("raise ValueError('The Ring')");
  173. } catch (py::value_error &boromir) {
  174. // Boromir never gets the ring
  175. assert(false);
  176. } catch (py::error_already_set &frodo) {
  177. // Frodo gets the ring
  178. py::print("I will take the ring");
  179. }
  180. try {
  181. // py::value_error is a request for pybind11 to raise a Python exception
  182. throw py::value_error("The ball");
  183. } catch (py::error_already_set &cat) {
  184. // cat won't catch the ball since
  185. // py::value_error is not a Python exception
  186. assert(false);
  187. } catch (py::value_error &dog) {
  188. // dog will catch the ball
  189. py::print("Run Spot run");
  190. throw; // Throw it again (pybind11 will raise ValueError)
  191. }
  192. Handling errors from the Python C API
  193. =====================================
  194. Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling
  195. the Python C API directly. When calling the Python C API directly, in
  196. addition to manually managing reference counts, one must follow the pybind11
  197. error protocol, which is outlined here.
  198. After calling the Python C API, if Python returns an error,
  199. ``throw py::error_already_set();``, which allows pybind11 to deal with the
  200. exception and pass it back to the Python interpreter. This includes calls to
  201. the error setting functions such as ``PyErr_SetString``.
  202. .. code-block:: cpp
  203. PyErr_SetString(PyExc_TypeError, "C API type error demo");
  204. throw py::error_already_set();
  205. // But it would be easier to simply...
  206. throw py::type_error("pybind11 wrapper type error");
  207. Alternately, to ignore the error, call `PyErr_Clear
  208. <https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_.
  209. Any Python error must be thrown or cleared, or Python/pybind11 will be left in
  210. an invalid state.
  211. .. _unraisable_exceptions:
  212. Handling unraisable exceptions
  213. ==============================
  214. If a Python function invoked from a C++ destructor or any function marked
  215. ``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there
  216. is no way to propagate the exception, as such functions may not throw.
  217. Should they throw or fail to catch any exceptions in their call graph,
  218. the C++ runtime calls ``std::terminate()`` to abort immediately.
  219. Similarly, Python exceptions raised in a class's ``__del__`` method do not
  220. propagate, but are logged by Python as an unraisable error. In Python 3.8+, a
  221. `system hook is triggered
  222. <https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_
  223. and an auditing event is logged.
  224. Any noexcept function should have a try-catch block that traps
  225. class:`error_already_set` (or any other exception that can occur). Note that
  226. pybind11 wrappers around Python exceptions such as
  227. :class:`pybind11::value_error` are *not* Python exceptions; they are C++
  228. exceptions that pybind11 catches and converts to Python exceptions. Noexcept
  229. functions cannot propagate these exceptions either. A useful approach is to
  230. convert them to Python exceptions and then ``discard_as_unraisable`` as shown
  231. below.
  232. .. code-block:: cpp
  233. void nonthrowing_func() noexcept(true) {
  234. try {
  235. // ...
  236. } catch (py::error_already_set &eas) {
  237. // Discard the Python error using Python APIs, using the C++ magic
  238. // variable __func__. Python already knows the type and value and of the
  239. // exception object.
  240. eas.discard_as_unraisable(__func__);
  241. } catch (const std::exception &e) {
  242. // Log and discard C++ exceptions.
  243. third_party::log(e);
  244. }
  245. }
  246. .. versionadded:: 2.6