Merge branch 'install_scripts' into persist
This commit is contained in:
commit
a722920d36
879 changed files with 183496 additions and 5657 deletions
8
.gitignore
vendored
8
.gitignore
vendored
|
@ -12,14 +12,6 @@ __pycache__/
|
|||
db/
|
||||
nohup.out
|
||||
|
||||
# VDF executables
|
||||
lib/chiavdf/fast_vdf/compile_asm
|
||||
lib/chiavdf/fast_vdf/vdf
|
||||
lib/chiavdf/fast_vdf/server
|
||||
lib/chiavdf/fast_vdf/vdf_server
|
||||
# Flint dependency
|
||||
lib/chiavdf/fast_vdf/flint
|
||||
|
||||
# Keys and plot files
|
||||
config/keys.yaml
|
||||
config/plots.yaml
|
||||
|
|
12
.gitmodules
vendored
12
.gitmodules
vendored
|
@ -1,12 +0,0 @@
|
|||
[submodule "lib/chiapos"]
|
||||
path = lib/chiapos
|
||||
url = git@github.com:Chia-Network/proof-of-space.git
|
||||
[submodule "lib/bip158/lib/pybind11"]
|
||||
path = lib/bip158/lib/pybind11
|
||||
url = https://github.com/pybind/pybind11.git
|
||||
[submodule "aiter"]
|
||||
path = lib/aiter
|
||||
url = https://github.com/richardkiss/aiter.git
|
||||
[submodule "lib/python-prompt-toolkit"]
|
||||
path = lib/python-prompt-toolkit
|
||||
url = https://github.com/prompt-toolkit/python-prompt-toolkit
|
17
README.md
17
README.md
|
@ -5,22 +5,13 @@ Python 3.7 is used for this project. Make sure your python version is >=3.7 by t
|
|||
|
||||
```bash
|
||||
# for Debian-based distros
|
||||
sudo apt-get install build-essential cmake python3-dev python3-venv --no-install-recommends
|
||||
sudo apt-get install build-essential cmake python3-dev python3-venv --no-install-recommends mongodb-org=4.2.1
|
||||
|
||||
# for MacOS
|
||||
brew install cmake
|
||||
brew tap mongodb/brew
|
||||
brew install cmake mongodb-community@4.2
|
||||
|
||||
git clone https://github.com/Chia-Network/chia-blockchain.git
|
||||
cd chia-blockchain
|
||||
git submodule update --init --recursive
|
||||
python3 -m venv .venv
|
||||
. .venv/bin/activate
|
||||
pip install wheel # For building blspy
|
||||
pip install -e .
|
||||
pip install -r requirements.txt
|
||||
|
||||
cd lib/chiavdf/fast_vdf
|
||||
# Install libgmp, libboost, and libflint, and then run the following
|
||||
git clone https://github.com/Chia-Network/chia-blockchain.git && cd chia-blockchain
|
||||
sh install.sh
|
||||
|
||||
# Install mongoDB from https://docs.mongodb.com/manual/administration/install-community/
|
||||
|
|
|
@ -66,10 +66,8 @@ full_node:
|
|||
host: "127.0.0.1"
|
||||
port: 8003
|
||||
introducer_peer:
|
||||
host: "127.0.0.1"
|
||||
host: "216.39.16.173"
|
||||
port: 8445
|
||||
# - host: "216.39.16.173" # Chia beast
|
||||
# port: 8004
|
||||
|
||||
introducer:
|
||||
host: "127.0.0.1"
|
||||
|
|
9
install.sh
Normal file
9
install.sh
Normal file
|
@ -0,0 +1,9 @@
|
|||
python3 -m venv .venv
|
||||
. .venv/bin/activate
|
||||
pip install wheel # For building blspy
|
||||
pip install -e .
|
||||
pip install -r requirements.txt
|
||||
|
||||
#cd lib/chiavdf/fast_vdf
|
||||
# Install libgmp, libboost, and libflint, and then run the following
|
||||
# sh install.sh
|
|
@ -1 +0,0 @@
|
|||
Subproject commit 2da160c58f848e2d67d57912eef59a8e8a62eacf
|
12
lib/aiter/.gitignore
vendored
Normal file
12
lib/aiter/.gitignore
vendored
Normal file
|
@ -0,0 +1,12 @@
|
|||
.coverage
|
||||
MANIFEST
|
||||
activate
|
||||
aiter.egg-info
|
||||
cover/
|
||||
build/
|
||||
dist
|
||||
docs/_build
|
||||
requests.egg-info/
|
||||
*.pyc
|
||||
*.swp
|
||||
*.egg
|
10
lib/aiter/.travis.yml
Normal file
10
lib/aiter/.travis.yml
Normal file
|
@ -0,0 +1,10 @@
|
|||
dist: xenial
|
||||
language: python
|
||||
python:
|
||||
- "3.6"
|
||||
- "3.7"
|
||||
- "nightly"
|
||||
install: pip install coverage
|
||||
script:
|
||||
- coverage run -m py.test tests
|
||||
- bash <(curl -s https://codecov.io/bash)
|
0
lib/aiter/HISTORY.md
Normal file
0
lib/aiter/HISTORY.md
Normal file
22
lib/aiter/LICENSE
Normal file
22
lib/aiter/LICENSE
Normal file
|
@ -0,0 +1,22 @@
|
|||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2019 by Richard Kiss
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
|
1
lib/aiter/MANIFEST.in
Normal file
1
lib/aiter/MANIFEST.in
Normal file
|
@ -0,0 +1 @@
|
|||
include README.md LICENSE NOTICE HISTORY.md requirements.txt
|
0
lib/aiter/NOTICE
Normal file
0
lib/aiter/NOTICE
Normal file
26
lib/aiter/README.md
Normal file
26
lib/aiter/README.md
Normal file
|
@ -0,0 +1,26 @@
|
|||
[![Build Status](https://travis-ci.org/richardkiss/aiter.png?branch=master)](https://travis-ci.org/richardkiss/aiter)
|
||||
[![codecov.io](https://codecov.io/github/richardkiss/aiter/coverage.svg?branch=master)](https://codecov.io/github/richardkiss/aiter)
|
||||
[![Documentation Status](https://readthedocs.org/projects/aiter/badge/?version=latest)](https://aiter.readthedocs.io/en/latest/?badge=latest)
|
||||
|
||||
|
||||
|
||||
aiter -- Asynchronous Iterator Patterns
|
||||
=======================================
|
||||
|
||||
|
||||
[PEP 525](https://www.python.org/dev/peps/pep-0525/) describes *asynchronous iterators*, a merging of iterators with async functionality. Python 3.6 makes legal constructs such as
|
||||
|
||||
```
|
||||
async for event in peer.event_iterator:
|
||||
await process_event(event)
|
||||
```
|
||||
|
||||
which is a huge improvement over using `async.Queue` objects which have no built-in way to determine "end-of-stream" conditions.
|
||||
|
||||
This module implements some patterns useful for python asynchronous iterators.
|
||||
|
||||
Documentation available on [readthedocs.io](https://aiter.readthedocs.io/).
|
||||
|
||||
A [tutorial](TUTORIAL.org) is available. [github version](https://github.com/richardkiss/aiter/blob/feature/tutorial/TUTORIAL.org)
|
||||
|
||||
*CAVEAT* This project is still in its infancy, and I reserve the right to rename things and cause other breaking changes.
|
131
lib/aiter/TUTORIAL.org
Normal file
131
lib/aiter/TUTORIAL.org
Normal file
|
@ -0,0 +1,131 @@
|
|||
|
||||
* Set up
|
||||
|
||||
Make sure you have ~ipython~, ~aiter~, and ~nc~ installed. In a pinch,
|
||||
you can use ~telnet~ instead of ~nc~.
|
||||
|
||||
|
||||
* An initial trivial server
|
||||
|
||||
** Set up the server
|
||||
|
||||
#+BEGIN_SRC python
|
||||
ipython
|
||||
|
||||
from aiter.server import start_server_aiter
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
async for _ in aiter: print(_)
|
||||
#+END_SRC
|
||||
|
||||
or
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
ipython -c "from aiter.server import start_server_aiter; s, a = await start_server_aiter(7777); [print(_) async for _ in a]"
|
||||
#+END_SRC
|
||||
|
||||
|
||||
** Connect to it
|
||||
|
||||
In another terminal, do the following:
|
||||
|
||||
nc localhost 7777
|
||||
|
||||
then hit control-C. Then do it a few more times.
|
||||
|
||||
On each connection, you'll see an ordered pair (r, w) where r is a StreamReader and w is a StreamWriter.
|
||||
|
||||
** Exit cleanly
|
||||
|
||||
Now try the following:
|
||||
|
||||
#+BEGIN_SRC python
|
||||
from aiter.server import start_server_aiter
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
count = 0
|
||||
async for _ in aiter:
|
||||
print(_)
|
||||
count += 1
|
||||
if count >= 3:
|
||||
server.close()
|
||||
#+END_SRC
|
||||
|
||||
|
||||
This will accept three connections, then close the server.
|
||||
|
||||
Try launching it and connecting. After the third connection, the server will simply exit. No need explicitly break from the loop.
|
||||
|
||||
|
||||
* Handling Clients
|
||||
|
||||
** One client
|
||||
|
||||
It's easy to extract framed bytes out of a StreamReader and turn them into messages.
|
||||
|
||||
Here is code that turns a StreamReader into an aiter of readline messages.
|
||||
|
||||
#+BEGIN_SRC python
|
||||
async def stream_reader_to_line_aiter(sr):
|
||||
while True:
|
||||
r = await sr.readline()
|
||||
if len(r) == 0:
|
||||
break
|
||||
yield r
|
||||
#+END_SRC
|
||||
|
||||
It's pretty easy to see how this could be adapted for more complex message formats, either text or binary.
|
||||
|
||||
Now try ~examples/2-line-server.py~. This accepts a single connection, then accepts messages terminated with a "\n"
|
||||
character, and echos them. Try connecting with ~nc~ and type a few lines. Then exit with control-C (harsh exit) or
|
||||
control-D (clean exit). You'll see that the server exits cleanly, indicating that the ~line_aiter~ completed. The
|
||||
error and the clean exit code paths are the same.
|
||||
|
||||
** Multiple Clients
|
||||
|
||||
server => aiter of (StreamReader, StreamWriter)
|
||||
|
||||
We have a function that makes the following transformation:
|
||||
|
||||
(StreamReader, StreamWriter) => aiter of (message, StreamWriter)
|
||||
|
||||
So we see how we can turn
|
||||
|
||||
aiter of (StreamReader, StreamWriter) => aiter of (aiter of (message, StreamWriter))
|
||||
|
||||
So we have an aiter of aiters. Whenever you see this construct, the thing you want is a ~join_aiter~.
|
||||
This turns an aiter of aiters into a single aiter that is a union of the objects coming out of each constituent
|
||||
aiter.
|
||||
|
||||
This gives us a transformation from
|
||||
|
||||
aiter of (StreamReader, StreamWriter) => aiter (message, StreamWriter)
|
||||
|
||||
Now we see we can write one method to handle streams from *all* clients at once.
|
||||
|
||||
See ~examples/3-multi-client-server.py~
|
||||
|
||||
* Finishing the Pipeline
|
||||
|
||||
Our main loop fetches events and processes them one at a time. This choice is fairly
|
||||
arbitrary; processing events could also be considered a transformation that
|
||||
accepts events and produces a result. This result could be something very simple, such
|
||||
as ~None~ or a summary of what happened to message on its way through the pipeline.
|
||||
|
||||
See ~examples/4-total-pipeline.py~ for an example.
|
||||
|
||||
Why bother doing this? This will become clear when we add scaling.
|
||||
|
||||
* Slow operations
|
||||
|
||||
Some events might launch a long or slow-running operation that takes a while
|
||||
to complete. If you look at the task model used in example 4, you'll see that
|
||||
only one event is handled at a time. Also note that the command "wait"
|
||||
is special, and takes five seconds. Try making two connections and you'll see
|
||||
that if you "wait" in one client, the other client becomes unresponsive.
|
||||
|
||||
This is clearly suboptimal. Luckily, there is an easy fix in ~map_aiter~:
|
||||
use multiple workers.
|
||||
|
||||
Using ~map_aiter~ has the side-effect that the order of items may change,
|
||||
since (obviously) fast events can be handled more quickly than slow events.
|
||||
|
||||
See ~examples/5-parallel-processing.py~ for an example.
|
13
lib/aiter/aiter/__init__.py
Normal file
13
lib/aiter/aiter/__init__.py
Normal file
|
@ -0,0 +1,13 @@
|
|||
"Useful patterns building upon asynchronous iterators"
|
||||
|
||||
__version__ = "0.1.2"
|
||||
|
||||
|
||||
__all__ = [
|
||||
"active_aiter", "aiter_forker", "aiter_to_iter", "azip", "flatten_aiter", "gated_aiter",
|
||||
"iter_to_aiter", "join_aiters", "map_aiter", "map_filter_aiter", "preload_aiter",
|
||||
"push_aiter", "sharable_aiter", "stoppable_aiter"
|
||||
]
|
||||
|
||||
for _ in __all__:
|
||||
exec("from .%s import %s" % (_, _))
|
35
lib/aiter/aiter/active_aiter.py
Normal file
35
lib/aiter/aiter/active_aiter.py
Normal file
|
@ -0,0 +1,35 @@
|
|||
import asyncio
|
||||
|
||||
from .push_aiter import push_aiter
|
||||
|
||||
|
||||
async def active_aiter(aiter):
|
||||
"""
|
||||
Wrap an aiter with a task that actively yanks out the items
|
||||
and puts them into a :class:`aiter.push_aiter <push_aiter>`.
|
||||
|
||||
This might be useful if you have an iterator that needs its elements
|
||||
pulled out as soon as they are created and cached in memory, even if
|
||||
the consumer is not yet ready. Be careful though, since getting too
|
||||
far behind can mean lots of memory is consumed, especially if each
|
||||
element uses a lot of memory, and can interfere with the flow control of
|
||||
TCP (for example) that depends on a data backlog.
|
||||
|
||||
:type aiter: aiter
|
||||
:param aiter: an async iterator
|
||||
|
||||
:return: a :class:`aiter.push_aiter <push_aiter>` yielding the same elements as aiter
|
||||
:rtype: :class:`aiter.push_aiter <push_aiter>`
|
||||
"""
|
||||
q = push_aiter()
|
||||
|
||||
async def _pull_task(aiter):
|
||||
async for _ in aiter:
|
||||
q.push(_)
|
||||
q.stop()
|
||||
|
||||
task = asyncio.ensure_future(_pull_task(aiter))
|
||||
|
||||
async for _ in q:
|
||||
yield _
|
||||
await task
|
53
lib/aiter/aiter/aiter_forker.py
Normal file
53
lib/aiter/aiter/aiter_forker.py
Normal file
|
@ -0,0 +1,53 @@
|
|||
from .deferred_coroutine import deferred_coroutine
|
||||
|
||||
|
||||
class _aiter_fork:
|
||||
"""
|
||||
Implementation of an aiter fork. Traces through a linked list of aiter elements, waiting
|
||||
when necessary.
|
||||
"""
|
||||
def __init__(self, next_awaitable, is_active=False):
|
||||
self._next_awaitable = next_awaitable
|
||||
self._is_active = is_active
|
||||
|
||||
def __aiter__(self):
|
||||
return self
|
||||
|
||||
async def __anext__(self):
|
||||
this_item, next_awaitable = await self._next_awaitable.wait(is_active=self._is_active)
|
||||
self._next_awaitable = next_awaitable
|
||||
return this_item
|
||||
|
||||
def fork(self, is_active=True):
|
||||
"""
|
||||
Create a new fork: either active, which uses the current task to await the next
|
||||
item; or passive, which waits until an active fork awaits it.
|
||||
"""
|
||||
return _aiter_fork(self._next_awaitable, is_active=is_active)
|
||||
|
||||
|
||||
def aiter_forker(aiter):
|
||||
"""
|
||||
If you have an aiter that you would like to fork (split into multiple
|
||||
iterators, each of which produces the same elements), wrap it with this
|
||||
function.
|
||||
|
||||
Returns a :class:`aiter._aiter_fork <_aiter_fork>` object that will yield
|
||||
the same objects in the same order. This object supports
|
||||
:py:func:`fork <aiter._aiter_fork.fork>`, which will let you create a
|
||||
duplicate stream.
|
||||
|
||||
:type aiter: aiter
|
||||
:param aiter: an async iterator
|
||||
|
||||
:return: a :class:`aiter._aiter_fork <_aiter_fork>`
|
||||
:rtype: :class:`aiter._aiter_fork <_aiter_fork>`
|
||||
"""
|
||||
_open_aiter = aiter.__aiter__()
|
||||
|
||||
async def get_next():
|
||||
next_item = await _open_aiter.__anext__()
|
||||
return next_item, deferred_coroutine(get_next)
|
||||
|
||||
next_awaitable = deferred_coroutine(get_next)
|
||||
return _aiter_fork(next_awaitable, is_active=True)
|
26
lib/aiter/aiter/aiter_to_iter.py
Normal file
26
lib/aiter/aiter/aiter_to_iter.py
Normal file
|
@ -0,0 +1,26 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
def aiter_to_iter(aiter, loop=None):
|
||||
"""
|
||||
Convert an async iterator to a regular iterator by invoking
|
||||
run_until_complete repeatedly.
|
||||
|
||||
:type aiter: aiter
|
||||
:param aiter: an async iterator
|
||||
|
||||
:type loop: asyncio event loop
|
||||
:param loop: the loop which will run *aiter*
|
||||
|
||||
:return: a *synchronous* iterator returning the same elements as aiter
|
||||
:rtype: a *synchronous* iterator
|
||||
"""
|
||||
if loop is None:
|
||||
loop = asyncio.get_event_loop()
|
||||
underlying_aiter = aiter.__aiter__()
|
||||
while True:
|
||||
try:
|
||||
_ = loop.run_until_complete(underlying_aiter.__anext__())
|
||||
yield _
|
||||
except StopAsyncIteration:
|
||||
break
|
25
lib/aiter/aiter/azip.py
Normal file
25
lib/aiter/aiter/azip.py
Normal file
|
@ -0,0 +1,25 @@
|
|||
async def azip(*aiters):
|
||||
"""
|
||||
async version of zip
|
||||
This function takes a list of async iterators and returns a single async iterator
|
||||
that yields tuples of elements.
|
||||
|
||||
This iterator advances as slow its slowest component (obviously).
|
||||
|
||||
example:
|
||||
async for a, b, c in azip(aiter1, aiter2, aiter3):
|
||||
print(a, b, c)
|
||||
|
||||
:type aiters: aiters
|
||||
:param aiters: one or more async iterators
|
||||
|
||||
:return: an aiter returning N-tuples similar to zip
|
||||
:rtype: an aiter
|
||||
"""
|
||||
anext_tuple = tuple([_.__aiter__() for _ in aiters])
|
||||
while True:
|
||||
try:
|
||||
next_tuple = tuple([await _.__anext__() for _ in anext_tuple])
|
||||
except StopAsyncIteration:
|
||||
break
|
||||
yield next_tuple
|
39
lib/aiter/aiter/deferred_coroutine.py
Normal file
39
lib/aiter/aiter/deferred_coroutine.py
Normal file
|
@ -0,0 +1,39 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
class deferred_coroutine:
|
||||
"""
|
||||
This class allows a co-routine to be invoked later by one of
|
||||
potentially many tasks, and only "borrow" the execution from
|
||||
the first task that wants the result.
|
||||
|
||||
Although lambda_coroutine can technically be any awaitable, the typical use
|
||||
case is a 0-argument function that returns a coroutine, since it's going to be
|
||||
await'ed.
|
||||
|
||||
:type lambda_coroutine: function
|
||||
:param lambda_coroutine: a 0-argument function returning an awaitable (usually a coroutine)
|
||||
"""
|
||||
def __init__(self, lambda_coroutine):
|
||||
self._next_future = asyncio.Future()
|
||||
self._active_invoked = False
|
||||
self._lambda_coroutine = lambda_coroutine
|
||||
|
||||
async def wait(self, is_active=True):
|
||||
"""
|
||||
The first time this is invoked with is_active True, the awaitable returned from
|
||||
lambda_coroutine is awaited. Then the awaited value is returned.
|
||||
|
||||
Subsequent calls return the awaited value, since the evaluating function is
|
||||
already in progress.
|
||||
"""
|
||||
|
||||
if is_active and not self._active_invoked:
|
||||
try:
|
||||
self._active_invoked = True
|
||||
_ = await self._lambda_coroutine()
|
||||
self._next_future.set_result(_)
|
||||
except Exception as ex:
|
||||
self._next_future.set_exception(ex)
|
||||
|
||||
return await self._next_future
|
35
lib/aiter/aiter/event_stream.py
Normal file
35
lib/aiter/aiter/event_stream.py
Normal file
|
@ -0,0 +1,35 @@
|
|||
from .map_aiter import map_aiter
|
||||
from .join_aiters import join_aiters
|
||||
|
||||
|
||||
def message_stream_to_event_stream(event_template, message_stream):
|
||||
"""
|
||||
This tweaks each message from message_stream by wrapping it with a dictionary
|
||||
populated with the given template, putting the message is at the top
|
||||
level under "message".
|
||||
"""
|
||||
|
||||
template = dict(event_template)
|
||||
|
||||
def adaptor(message):
|
||||
event = dict(template)
|
||||
event.update(message=message)
|
||||
return event
|
||||
|
||||
return map_aiter(adaptor, message_stream)
|
||||
|
||||
|
||||
def rws_to_event_aiter(rws_aiter, reader_to_message_stream):
|
||||
|
||||
def rws_to_reader_event_template_adaptor(rws):
|
||||
return rws, rws["reader"]
|
||||
|
||||
def reader_event_template_to_event_stream_adaptor(rws_reader):
|
||||
rws, reader = rws_reader
|
||||
return message_stream_to_event_stream(rws, reader_to_message_stream(reader))
|
||||
|
||||
def adaptor(rws):
|
||||
return reader_event_template_to_event_stream_adaptor(
|
||||
rws_to_reader_event_template_adaptor(rws))
|
||||
|
||||
return join_aiters(map_aiter(adaptor, rws_aiter))
|
17
lib/aiter/aiter/flatten_aiter.py
Normal file
17
lib/aiter/aiter/flatten_aiter.py
Normal file
|
@ -0,0 +1,17 @@
|
|||
async def flatten_aiter(aiter):
|
||||
"""
|
||||
Take an async iterator that returns lists and return the individual
|
||||
elements.
|
||||
|
||||
:type aiter: aiter
|
||||
:param aiter: an async iterator yielding lists
|
||||
|
||||
:return: an async iterator where the elements are the flattened inputs
|
||||
:rtype: an async iterator
|
||||
"""
|
||||
async for items in aiter:
|
||||
try:
|
||||
for _ in items:
|
||||
yield _
|
||||
except Exception:
|
||||
pass
|
48
lib/aiter/aiter/gated_aiter.py
Normal file
48
lib/aiter/aiter/gated_aiter.py
Normal file
|
@ -0,0 +1,48 @@
|
|||
import asyncio
|
||||
|
||||
from .azip import azip
|
||||
from .active_aiter import active_aiter
|
||||
from .map_filter_aiter import map_filter_aiter
|
||||
from .push_aiter import push_aiter
|
||||
|
||||
|
||||
class gated_aiter:
|
||||
"""
|
||||
Returns an aiter that you can "push" integer values into.
|
||||
When a number is pushed, that many items are allowed out through the gate.
|
||||
|
||||
This is kind of like a discrete version of an electronic transistor.
|
||||
|
||||
:type aiter: aiter
|
||||
:param aiter: an async iterator
|
||||
|
||||
:return: an async iterator yielding the same values as the original aiter
|
||||
:rtype: :class:`aiter.gated_aiter <gated_aiter>`
|
||||
"""
|
||||
def __init__(self, aiter):
|
||||
self._gate = push_aiter()
|
||||
self._open_aiter = active_aiter(azip(aiter, map_filter_aiter(range, self._gate))).__aiter__()
|
||||
self._semaphore = asyncio.Semaphore()
|
||||
|
||||
def __aiter__(self):
|
||||
return self
|
||||
|
||||
async def __anext__(self):
|
||||
async with self._semaphore:
|
||||
return (await self._open_aiter.__anext__())[0]
|
||||
|
||||
def push(self, count):
|
||||
"""
|
||||
Note that several additional items are allowed through the gated_aiter.
|
||||
|
||||
:type count: int
|
||||
:param count: the number of items that can be allowed out the aiter. These are cumulative.
|
||||
"""
|
||||
if not self._gate.is_stopped():
|
||||
self._gate.push(count)
|
||||
|
||||
def stop(self):
|
||||
"""
|
||||
After the previously authorized items (from `push`) are pulled out the aiter, the aiter will exit.
|
||||
"""
|
||||
self._gate.stop()
|
9
lib/aiter/aiter/iter_to_aiter.py
Normal file
9
lib/aiter/aiter/iter_to_aiter.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
async def iter_to_aiter(iter):
|
||||
"""
|
||||
:type iter: synchronous iterator
|
||||
:param iter: a synchronous iterator
|
||||
|
||||
This converts a regular iterator to an async iterator.
|
||||
"""
|
||||
for _ in iter:
|
||||
yield _
|
49
lib/aiter/aiter/join_aiters.py
Normal file
49
lib/aiter/aiter/join_aiters.py
Normal file
|
@ -0,0 +1,49 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
async def join_aiters(aiter_of_aiters):
|
||||
"""
|
||||
This wrapper takes an aiter of aiters and pipe the items coming out of all of them into a
|
||||
single aiter.
|
||||
|
||||
:type aiter_of_aiters: async iterator
|
||||
:param aiter_of_aiters: an aiter that yields aiters
|
||||
|
||||
:return: an aiter returning elements that come from any of the underlying aiters
|
||||
:rtype: async iterator
|
||||
"""
|
||||
|
||||
async def _aiter_to_next_job(aiter):
|
||||
"""
|
||||
Return two lists: a list of items to yield, and a list of jobs to add to queue.
|
||||
"""
|
||||
try:
|
||||
items = [await aiter.__anext__()]
|
||||
jobs = [asyncio.ensure_future(_aiter_to_next_job(aiter))]
|
||||
except StopAsyncIteration:
|
||||
items = jobs = []
|
||||
return items, jobs
|
||||
|
||||
async def _main_aiter_to_next_job(aiter_of_aiters):
|
||||
"""
|
||||
Return two lists: a list of items to yield, and a list of jobs to add to queue.
|
||||
"""
|
||||
try:
|
||||
items = []
|
||||
new_aiter = await aiter_of_aiters.__anext__()
|
||||
jobs = [
|
||||
asyncio.ensure_future(_aiter_to_next_job(new_aiter.__aiter__())),
|
||||
asyncio.ensure_future(_main_aiter_to_next_job(aiter_of_aiters))]
|
||||
except StopAsyncIteration:
|
||||
jobs = []
|
||||
return items, jobs
|
||||
|
||||
jobs = set([_main_aiter_to_next_job(aiter_of_aiters.__aiter__())])
|
||||
|
||||
while jobs:
|
||||
done, jobs = await asyncio.wait(jobs, return_when=asyncio.FIRST_COMPLETED)
|
||||
for _ in done:
|
||||
new_items, new_jobs = await _
|
||||
for _ in new_items:
|
||||
yield _
|
||||
jobs.update(_ for _ in new_jobs)
|
40
lib/aiter/aiter/map_aiter.py
Normal file
40
lib/aiter/aiter/map_aiter.py
Normal file
|
@ -0,0 +1,40 @@
|
|||
import inspect
|
||||
|
||||
from .parallel_map_aiter import parallel_map_aiter
|
||||
from .simple_map_aiter import simple_map_aiter
|
||||
|
||||
|
||||
def map_aiter(map_f, aiter, worker_count=1):
|
||||
"""
|
||||
Take an async iterator and a map function, and apply the function
|
||||
to everything coming out of the iterator before passing it on.
|
||||
In this case, the map_f must return a list, which will be flattened.
|
||||
Empty lists are okay, so you can filter items by excluding them from the list.
|
||||
|
||||
Note that since there are multiple workers, the order or processed elements
|
||||
might not match the input order.
|
||||
|
||||
:type aiter: async iterator
|
||||
:param aiter: an aiter
|
||||
|
||||
:type map_f: a function, regular or async, that accepts a single parameter and returns
|
||||
a list (or other iterable)
|
||||
:param map_f: the mapping function
|
||||
|
||||
:type worker_count: int
|
||||
:param worker_count: the number of worker tasks that pull items out of aiter
|
||||
|
||||
:return: an aiter returning transformed items that have been processed through map_f
|
||||
:rtype: an async iterator
|
||||
"""
|
||||
|
||||
if (worker_count > 1 and
|
||||
not inspect.iscoroutinefunction(map_f) and
|
||||
not inspect.isasyncgenfunction(map_f)):
|
||||
raise ValueError(
|
||||
"map_f is not a coroutine, which makes "
|
||||
"it pointless to use more than 1 worker")
|
||||
|
||||
if worker_count > 1:
|
||||
return parallel_map_aiter(map_f, aiter, worker_count)
|
||||
return simple_map_aiter(map_f, aiter)
|
34
lib/aiter/aiter/map_filter_aiter.py
Normal file
34
lib/aiter/aiter/map_filter_aiter.py
Normal file
|
@ -0,0 +1,34 @@
|
|||
import asyncio
|
||||
import logging
|
||||
|
||||
|
||||
async def map_filter_aiter(map_f, aiter):
|
||||
"""
|
||||
Take an async iterator and a map function, and apply the function
|
||||
to everything coming out of the iterator before passing it on.
|
||||
In this case, the map_f must return a list, which will be flattened.
|
||||
Empty lists are okay, so you can filter items by excluding them from the list.
|
||||
|
||||
:type aiter: async iterator
|
||||
:param aiter: an aiter
|
||||
|
||||
:type map_f: a function, regular or async, that accepts a single parameter and returns
|
||||
a list (or other iterable)
|
||||
:param map_f: the mapping function
|
||||
|
||||
:return: an aiter returning transformed items that have been processed through map_f
|
||||
:rtype: an async iterator
|
||||
"""
|
||||
if asyncio.iscoroutinefunction(map_f):
|
||||
_map_f = map_f
|
||||
else:
|
||||
async def _map_f(_):
|
||||
return map_f(_)
|
||||
|
||||
async for _ in aiter:
|
||||
try:
|
||||
items = await _map_f(_)
|
||||
for _ in items:
|
||||
yield _
|
||||
except Exception:
|
||||
logging.exception("unhandled mapping function %s worker exception on %s", map_f, _)
|
31
lib/aiter/aiter/parallel_map_aiter.py
Normal file
31
lib/aiter/aiter/parallel_map_aiter.py
Normal file
|
@ -0,0 +1,31 @@
|
|||
from .iter_to_aiter import iter_to_aiter
|
||||
from .join_aiters import join_aiters
|
||||
from .sharable_aiter import sharable_aiter
|
||||
from .simple_map_aiter import simple_map_aiter
|
||||
|
||||
|
||||
def parallel_map_aiter(map_f, aiter, worker_count):
|
||||
"""
|
||||
Take an async iterator and a map function, and apply the function
|
||||
to everything coming out of the iterator before passing it on.
|
||||
|
||||
Note that if there are multiple workers, the order or processed elements
|
||||
might not match the input order.
|
||||
|
||||
:type aiter: async iterator
|
||||
:param aiter: an aiter
|
||||
|
||||
:type map_f: a function, regular or async, that accepts a single parameter and returns
|
||||
a list (or other iterable)
|
||||
:param map_f: the mapping function
|
||||
|
||||
:type worker_count: int
|
||||
:param worker_count: the number of worker tasks that pull items out of aiter
|
||||
|
||||
:return: an aiter returning transformed items that have been processed through map_f
|
||||
:rtype: an async iterator
|
||||
"""
|
||||
shared_aiter = sharable_aiter(aiter)
|
||||
aiters = [simple_map_aiter(
|
||||
map_f, shared_aiter) for _ in range(worker_count)]
|
||||
return join_aiters(iter_to_aiter(aiters))
|
25
lib/aiter/aiter/preload_aiter.py
Normal file
25
lib/aiter/aiter/preload_aiter.py
Normal file
|
@ -0,0 +1,25 @@
|
|||
from .gated_aiter import gated_aiter
|
||||
|
||||
|
||||
async def preload_aiter(preload_size, aiter):
|
||||
"""
|
||||
This aiter wraps around another aiter, and forces a preloaded
|
||||
buffer of the given size. When an element is removed, the loader is
|
||||
given a kick to try to refill the preload buffer.
|
||||
|
||||
:type preload_size: int
|
||||
:param preload_size: the maximum number of items to attempt to preload
|
||||
|
||||
:type aiter: async iterator
|
||||
:param aiter: an aiter
|
||||
|
||||
:return: an async iterator yielding the same values as the original aiter
|
||||
:rtype: async iterator
|
||||
"""
|
||||
|
||||
gate = gated_aiter(aiter)
|
||||
gate.push(preload_size)
|
||||
async for _ in gate:
|
||||
yield _
|
||||
gate.push(1)
|
||||
gate.stop()
|
105
lib/aiter/aiter/push_aiter.py
Normal file
105
lib/aiter/aiter/push_aiter.py
Normal file
|
@ -0,0 +1,105 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
class push_aiter:
|
||||
"""
|
||||
An asynchronous iterator based on a linked-list.
|
||||
Data goes in the head via "push".
|
||||
Allows peeking to determine how many elements are ready.
|
||||
|
||||
This is functionally similar to an :class:`async.Queue <async.Queue>`
|
||||
object. It creates an aiter that you can `push` items into.
|
||||
Unlike a `Queue` object, you can also invoke :py:func:`stop <stop>`, which will
|
||||
raise a `StopAsyncIteration` on the listener's side, allowing for a
|
||||
clean exit.
|
||||
|
||||
You'd use this when you want to "turn around" execution, ie. have
|
||||
a task that is occasionally invoked (like a hardware interrupt)
|
||||
to produce a new event for an aiter.
|
||||
"""
|
||||
def __init__(self):
|
||||
self._head = self._tail = asyncio.Future()
|
||||
|
||||
def push(self, *items):
|
||||
"""
|
||||
Accept one or more item and push them to the end of the
|
||||
aiter's queue.
|
||||
"""
|
||||
if self._head.cancelled():
|
||||
raise ValueError("%s closed" % self)
|
||||
for item in items:
|
||||
f = asyncio.Future()
|
||||
self._head.set_result((item, f))
|
||||
self._head = f
|
||||
|
||||
def stop(self):
|
||||
"""
|
||||
Raise a `StopAsyncIteration` exception on the listener side
|
||||
once no more already-queued elements are pending.
|
||||
"""
|
||||
self._head.cancel()
|
||||
|
||||
async def __aiter__(self):
|
||||
try:
|
||||
while True:
|
||||
_, self._tail = await self._tail
|
||||
yield _
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
def available_iter(self):
|
||||
"""
|
||||
Return a *synchronous* iterator of elements that are immediately
|
||||
available to be consumed without waiting for a task switch.
|
||||
"""
|
||||
tail = self._tail
|
||||
try:
|
||||
while tail.done():
|
||||
_, tail = tail.result()
|
||||
yield _
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
def is_stopped(self):
|
||||
"""
|
||||
Return a boolean indicating whether or not :py:func:`stop <stop>`
|
||||
has been called. Additional elements may still be available.
|
||||
|
||||
:return: whether or not the aiter has been stopped
|
||||
:rtype: bool
|
||||
"""
|
||||
return self._tail.cancelled()
|
||||
|
||||
def is_item_available(self):
|
||||
"""
|
||||
Return a boolean indicating whether or not an element is available without
|
||||
blocking for a task switch.
|
||||
|
||||
:return: whether or not the aiter has been stopped
|
||||
:rtype: bool
|
||||
"""
|
||||
return self.is_len_at_least(1)
|
||||
|
||||
def is_len_at_least(self, n):
|
||||
"""
|
||||
Return a boolean indicating whether or not `n` elements are available without
|
||||
blocking for a task switch.
|
||||
|
||||
:type n: int
|
||||
:param n: count of items
|
||||
|
||||
:return: True iff n items are available
|
||||
:rtype: bool
|
||||
"""
|
||||
for _, item in enumerate(self.available_iter()):
|
||||
if _+1 >= n:
|
||||
return True
|
||||
return False
|
||||
|
||||
def __len__(self):
|
||||
"""
|
||||
:return: number of items immediately available withouth blocking
|
||||
:rtype: int
|
||||
"""
|
||||
|
||||
return sum(1 for _ in self.available_iter())
|
22
lib/aiter/aiter/server.py
Normal file
22
lib/aiter/aiter/server.py
Normal file
|
@ -0,0 +1,22 @@
|
|||
import asyncio
|
||||
|
||||
from .push_aiter import push_aiter
|
||||
|
||||
|
||||
async def aiter_server(start_f, *args, **kwargs):
|
||||
aiter = push_aiter()
|
||||
server = await start_f(
|
||||
client_connected_cb=lambda r, w: aiter.push((r, w)), *args, **kwargs)
|
||||
aiter.task = asyncio.ensure_future(
|
||||
server.wait_closed()).add_done_callback(lambda f: aiter.stop())
|
||||
return server, aiter
|
||||
|
||||
|
||||
async def start_server_aiter(port, *args, **kwargs):
|
||||
return await aiter_server(
|
||||
asyncio.start_server, port=port, *args, **kwargs)
|
||||
|
||||
|
||||
async def start_unix_server_aiter(path, *args, **kwargs):
|
||||
return await aiter_server(
|
||||
asyncio.start_unix_server, path=path, *args, **kwargs)
|
20
lib/aiter/aiter/sharable_aiter.py
Normal file
20
lib/aiter/aiter/sharable_aiter.py
Normal file
|
@ -0,0 +1,20 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
class sharable_aiter:
|
||||
"""
|
||||
Not all iterators can have multiple consumers. For example, asynchronous
|
||||
generators don't allow it. But if you wrap it with one of these,
|
||||
you'll be okay.
|
||||
"""
|
||||
|
||||
def __init__(self, aiter):
|
||||
self._opened_aiter = aiter.__aiter__()
|
||||
self._semaphore = asyncio.Semaphore()
|
||||
|
||||
def __aiter__(self):
|
||||
return self
|
||||
|
||||
async def __anext__(self):
|
||||
async with self._semaphore:
|
||||
return await self._opened_aiter.__anext__()
|
29
lib/aiter/aiter/simple_map_aiter.py
Normal file
29
lib/aiter/aiter/simple_map_aiter.py
Normal file
|
@ -0,0 +1,29 @@
|
|||
import asyncio
|
||||
import logging
|
||||
|
||||
|
||||
async def simple_map_aiter(map_f, aiter):
|
||||
"""
|
||||
Take an async iterator and a map function, and apply the function
|
||||
to everything coming out of the iterator before passing it on.
|
||||
|
||||
:type aiter: async iterator
|
||||
:param aiter: an aiter
|
||||
|
||||
:type map_f: a function, regular or async, that accepts a single parameter
|
||||
:param map_f: the mapping function
|
||||
|
||||
:return: an aiter returning transformed items that have been processed through map_f
|
||||
:rtype: async iterator
|
||||
"""
|
||||
if asyncio.iscoroutinefunction(map_f):
|
||||
_map_f = map_f
|
||||
else:
|
||||
async def _map_f(_):
|
||||
return map_f(_)
|
||||
|
||||
async for _ in aiter:
|
||||
try:
|
||||
yield await _map_f(_)
|
||||
except Exception:
|
||||
logging.exception("unhandled mapping function %s worker exception on %s", map_f, _)
|
24
lib/aiter/aiter/stoppable_aiter.py
Normal file
24
lib/aiter/aiter/stoppable_aiter.py
Normal file
|
@ -0,0 +1,24 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
class stoppable_aiter:
|
||||
"""
|
||||
A wrapper around an iterator that supports a manual shut-off.
|
||||
"""
|
||||
|
||||
def __init__(self, aiter):
|
||||
self._open_aiter = aiter.__aiter__()
|
||||
self._is_stopping = False
|
||||
self._semaphore = asyncio.Semaphore()
|
||||
|
||||
def __aiter__(self):
|
||||
return self
|
||||
|
||||
async def __anext__(self):
|
||||
if self._is_stopping:
|
||||
raise StopAsyncIteration
|
||||
async with self._semaphore:
|
||||
return await self._open_aiter.__anext__()
|
||||
|
||||
def stop(self):
|
||||
self._is_stopping = True
|
19
lib/aiter/docs/Makefile
Normal file
19
lib/aiter/docs/Makefile
Normal file
|
@ -0,0 +1,19 @@
|
|||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line.
|
||||
SPHINXOPTS =
|
||||
SPHINXBUILD = sphinx-build
|
||||
SOURCEDIR = source
|
||||
BUILDDIR = build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
180
lib/aiter/docs/source/conf.py
Normal file
180
lib/aiter/docs/source/conf.py
Normal file
|
@ -0,0 +1,180 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file does only contain a selection of the most common options. For a
|
||||
# full list see the documentation:
|
||||
# http://www.sphinx-doc.org/en/master/config
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
# import os
|
||||
# import sys
|
||||
# sys.path.insert(0, os.path.abspath('.'))
|
||||
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'aiter'
|
||||
copyright = '2019, Richard Kiss'
|
||||
author = 'Richard Kiss'
|
||||
|
||||
# The short X.Y version
|
||||
version = ''
|
||||
# The full version, including alpha/beta/rc tags
|
||||
release = ''
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#
|
||||
# needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'sphinx.ext.autodoc',
|
||||
'sphinx.ext.viewcode',
|
||||
]
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
#
|
||||
# source_suffix = ['.rst', '.md']
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = []
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = None
|
||||
|
||||
autoclass_content = "both"
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = 'alabaster'
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#
|
||||
# html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
|
||||
# Custom sidebar templates, must be a dictionary that maps document names
|
||||
# to template names.
|
||||
#
|
||||
# The default sidebars (for documents that don't match any pattern) are
|
||||
# defined by theme itself. Builtin themes are using these templates by
|
||||
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
|
||||
# 'searchbox.html']``.
|
||||
#
|
||||
# html_sidebars = {}
|
||||
|
||||
|
||||
# -- Options for HTMLHelp output ---------------------------------------------
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'aiterdoc'
|
||||
|
||||
|
||||
# -- Options for LaTeX output ------------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#
|
||||
# 'papersize': 'letterpaper',
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#
|
||||
# 'pointsize': '10pt',
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#
|
||||
# 'preamble': '',
|
||||
|
||||
# Latex figure (float) alignment
|
||||
#
|
||||
# 'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, 'aiter.tex', 'aiter Documentation',
|
||||
'Richard Kiss', 'manual'),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for manual page output ------------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(master_doc, 'aiter', 'aiter Documentation',
|
||||
[author], 1)
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Texinfo output ----------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(master_doc, 'aiter', 'aiter Documentation',
|
||||
author, 'aiter', 'One line description of project.',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
|
||||
# -- Options for Epub output -------------------------------------------------
|
||||
|
||||
# Bibliographic Dublin Core info.
|
||||
epub_title = project
|
||||
|
||||
# The unique identifier of the text. This can be a ISBN number
|
||||
# or the project homepage.
|
||||
#
|
||||
# epub_identifier = ''
|
||||
|
||||
# A unique identification for the text.
|
||||
#
|
||||
# epub_uid = ''
|
||||
|
||||
# A list of files that should not be packed into the epub file.
|
||||
epub_exclude_files = ['search.html']
|
||||
|
||||
|
||||
# -- Extension configuration -------------------------------------------------
|
40
lib/aiter/docs/source/index.rst
Normal file
40
lib/aiter/docs/source/index.rst
Normal file
|
@ -0,0 +1,40 @@
|
|||
.. aiter documentation master file, created by
|
||||
sphinx-quickstart on Sun Feb 17 15:23:07 2019.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to aiter's documentation!
|
||||
=================================
|
||||
|
||||
.. image:: https://codecov.io/github/richardkiss/aiter/coverage.svg?branch=master
|
||||
:target: https://codecov.io/github/richardkiss/aiter
|
||||
|
||||
.. image:: https://img.shields.io/pypi/l/aiter.svg
|
||||
:target: https://pypi.python.org/pypi/aiter
|
||||
|
||||
.. image:: https://img.shields.io/pypi/pyversions/aiter.svg
|
||||
:target: https://pypi.python.org/pypi/aiter
|
||||
|
||||
.. image:: https://travis-ci.org/richardkiss/aiter.svg?branch=master
|
||||
:target: https://travis-ci.org/richardkiss/aiter
|
||||
|
||||
This documentation is a work-in-progress, and your contributions are welcome
|
||||
at <https://github.com/richardkiss/aiter>.
|
||||
|
||||
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents:
|
||||
|
||||
.. automodule:: aiter
|
||||
:members:
|
||||
|
||||
|
||||
Indices and tables
|
||||
==================
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
13
lib/aiter/examples/1-simple-server.py
Normal file
13
lib/aiter/examples/1-simple-server.py
Normal file
|
@ -0,0 +1,13 @@
|
|||
import asyncio
|
||||
|
||||
from aiter.server import start_server_aiter
|
||||
|
||||
|
||||
async def main():
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
async for _ in aiter:
|
||||
print(_)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.get_event_loop().run_until_complete(main())
|
27
lib/aiter/examples/2-line-server.py
Normal file
27
lib/aiter/examples/2-line-server.py
Normal file
|
@ -0,0 +1,27 @@
|
|||
import asyncio
|
||||
|
||||
from aiter.server import start_server_aiter
|
||||
|
||||
|
||||
async def stream_reader_to_line_aiter(sr):
|
||||
while True:
|
||||
r = await sr.readline()
|
||||
if len(r) == 0:
|
||||
break
|
||||
yield r
|
||||
|
||||
|
||||
async def main():
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
async for sr, sw in aiter:
|
||||
print(sr)
|
||||
line_aiter = stream_reader_to_line_aiter(sr)
|
||||
# this hack means we only accept one connection
|
||||
break
|
||||
|
||||
async for line in line_aiter:
|
||||
print(line)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.get_event_loop().run_until_complete(main())
|
34
lib/aiter/examples/3-multi-client-server.py
Normal file
34
lib/aiter/examples/3-multi-client-server.py
Normal file
|
@ -0,0 +1,34 @@
|
|||
import asyncio
|
||||
|
||||
from aiter import join_aiters, map_aiter
|
||||
from aiter.server import start_server_aiter
|
||||
|
||||
|
||||
async def stream_reader_writer_to_line_writer_aiter(pair):
|
||||
sr, sw = pair
|
||||
while True:
|
||||
r = await sr.readline()
|
||||
if len(r) == 0:
|
||||
break
|
||||
yield r, sw
|
||||
|
||||
|
||||
async def main():
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
line_writer_aiter_aiter = map_aiter(
|
||||
stream_reader_writer_to_line_writer_aiter,
|
||||
aiter)
|
||||
line_writer_aiter = join_aiters(line_writer_aiter_aiter)
|
||||
|
||||
async for line, sw in line_writer_aiter:
|
||||
print(line)
|
||||
await sw.drain()
|
||||
if line == b"\n":
|
||||
sw.close()
|
||||
sw.write(line)
|
||||
if line == b"quit\n":
|
||||
server.close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.get_event_loop().run_until_complete(main())
|
48
lib/aiter/examples/4-total-pipeline.py
Normal file
48
lib/aiter/examples/4-total-pipeline.py
Normal file
|
@ -0,0 +1,48 @@
|
|||
import asyncio
|
||||
import functools
|
||||
|
||||
from aiter import join_aiters, map_aiter
|
||||
from aiter.server import start_server_aiter
|
||||
|
||||
|
||||
async def handle_event(server_line_sw_tuple):
|
||||
server, line, sw = server_line_sw_tuple
|
||||
await sw.drain()
|
||||
if line == b"\n":
|
||||
sw.close()
|
||||
sw.write(line)
|
||||
if line == b"quit\n":
|
||||
server.close()
|
||||
if line == b"wait\n":
|
||||
await asyncio.sleep(5)
|
||||
return line
|
||||
|
||||
|
||||
async def stream_reader_writer_to_line_writer_aiter(server, pair):
|
||||
sr, sw = pair
|
||||
while True:
|
||||
line = await sr.readline()
|
||||
if len(line) == 0:
|
||||
break
|
||||
yield server, line, sw
|
||||
|
||||
|
||||
async def main():
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
|
||||
line_writer_aiter_aiter = map_aiter(
|
||||
functools.partial(
|
||||
stream_reader_writer_to_line_writer_aiter,
|
||||
server),
|
||||
aiter)
|
||||
line_writer_aiter = join_aiters(line_writer_aiter_aiter)
|
||||
completed_event_aiter = map_aiter(
|
||||
handle_event,
|
||||
line_writer_aiter)
|
||||
|
||||
async for line in completed_event_aiter:
|
||||
print(line)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.get_event_loop().run_until_complete(main())
|
51
lib/aiter/examples/5-parallel-processing.py
Normal file
51
lib/aiter/examples/5-parallel-processing.py
Normal file
|
@ -0,0 +1,51 @@
|
|||
import asyncio
|
||||
import functools
|
||||
|
||||
from aiter import (
|
||||
join_aiters, map_aiter
|
||||
)
|
||||
from aiter.server import start_server_aiter
|
||||
|
||||
|
||||
async def handle_event(server_line_sw_tuple):
|
||||
server, line, sw = server_line_sw_tuple
|
||||
await sw.drain()
|
||||
if line == b"\n":
|
||||
sw.close()
|
||||
sw.write(line)
|
||||
if line == b"quit\n":
|
||||
server.close()
|
||||
if line == b"wait\n":
|
||||
await asyncio.sleep(5)
|
||||
return line
|
||||
|
||||
|
||||
async def stream_reader_writer_to_line_writer_aiter(server, pair):
|
||||
sr, sw = pair
|
||||
while True:
|
||||
line = await sr.readline()
|
||||
if len(line) == 0:
|
||||
break
|
||||
yield server, line, sw
|
||||
|
||||
|
||||
async def main():
|
||||
server, aiter = await start_server_aiter(7777)
|
||||
|
||||
line_writer_aiter_aiter = map_aiter(
|
||||
functools.partial(
|
||||
stream_reader_writer_to_line_writer_aiter,
|
||||
server),
|
||||
aiter)
|
||||
line_writer_aiter = join_aiters(line_writer_aiter_aiter)
|
||||
completed_event_aiter = map_aiter(
|
||||
handle_event,
|
||||
line_writer_aiter,
|
||||
worker_count=5)
|
||||
|
||||
async for line in completed_event_aiter:
|
||||
print(line)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.get_event_loop().run_until_complete(main())
|
41
lib/aiter/setup.py
Executable file
41
lib/aiter/setup.py
Executable file
|
@ -0,0 +1,41 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
import re
|
||||
|
||||
import setuptools
|
||||
|
||||
NAME = "aiter"
|
||||
|
||||
packages = setuptools.find_packages(exclude=["tests"])
|
||||
test_requirements = "pytest>=2.8.0"
|
||||
|
||||
with open('%s/__init__.py' % NAME, 'r') as fd:
|
||||
version = re.search(r'^__version__\s*=\s*[\'"]([^\'"]*)[\'"]',
|
||||
fd.read(), re.MULTILINE).group(1)
|
||||
|
||||
with open('README.md') as f:
|
||||
readme = f.read()
|
||||
|
||||
with open('HISTORY.md') as f:
|
||||
history = f.read()
|
||||
|
||||
|
||||
setuptools.setup(
|
||||
name=NAME,
|
||||
description="Useful patterns building upon asynchronous iterators.",
|
||||
long_description=readme + '\n\n' + history,
|
||||
long_description_content_type="text/markdown",
|
||||
author="Richard Kiss",
|
||||
author_email="him@richardkiss.com",
|
||||
version=version,
|
||||
packages=packages,
|
||||
package_data={'': ['LICENSE', 'NOTICE'], 'requests': ['*.pem']},
|
||||
url="https://github.com/richardkiss/%s" % NAME,
|
||||
license="http://opensource.org/licenses/MIT",
|
||||
classifiers=[
|
||||
'Development Status :: 3 - Alpha',
|
||||
'Intended Audience :: Developers',
|
||||
'Programming Language :: Python :: 3.6',
|
||||
'Programming Language :: Python :: 3.7',
|
||||
'License :: OSI Approved :: MIT License',
|
||||
],)
|
0
lib/aiter/tests/__init__.py
Normal file
0
lib/aiter/tests/__init__.py
Normal file
19
lib/aiter/tests/helpers.py
Normal file
19
lib/aiter/tests/helpers.py
Normal file
|
@ -0,0 +1,19 @@
|
|||
import asyncio
|
||||
|
||||
|
||||
def run(f):
|
||||
return asyncio.get_event_loop().run_until_complete(f)
|
||||
|
||||
|
||||
async def get_n(aiter, n=0):
|
||||
"""
|
||||
Get n items.
|
||||
"""
|
||||
r = []
|
||||
count = 0
|
||||
async for _ in aiter:
|
||||
r.append(_)
|
||||
count += 1
|
||||
if count >= n and n != 0:
|
||||
break
|
||||
return r
|
49
lib/aiter/tests/test_aiter_forker.py
Normal file
49
lib/aiter/tests/test_aiter_forker.py
Normal file
|
@ -0,0 +1,49 @@
|
|||
import asyncio
|
||||
import unittest
|
||||
|
||||
from aiter import aiter_forker, push_aiter
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aiter_forker(unittest.TestCase):
|
||||
|
||||
def test_aiter_forker(self):
|
||||
|
||||
q = push_aiter()
|
||||
forker = aiter_forker(q)
|
||||
q.push(1, 2, 3, 4, 5)
|
||||
r0 = run(get_n(forker, 3))
|
||||
f2 = forker.fork()
|
||||
q.push(*range(7, 14))
|
||||
q.stop()
|
||||
r1 = run(get_n(forker))
|
||||
r2 = run(get_n(f2))
|
||||
|
||||
self.assertEqual(r0, [1, 2, 3])
|
||||
self.assertEqual(r1, [4, 5, 7, 8, 9, 10, 11, 12, 13])
|
||||
self.assertEqual(r2, [4, 5, 7, 8, 9, 10, 11, 12, 13])
|
||||
|
||||
def test_aiter_forker_multiple_active(self):
|
||||
"""
|
||||
Multiple forks of an aiter_forker both asking for empty q information
|
||||
at the same time. Make sure the second one doesn't block.
|
||||
"""
|
||||
|
||||
q = push_aiter()
|
||||
forker = aiter_forker(q)
|
||||
fork_1 = forker.fork(is_active=True)
|
||||
fork_2 = forker.fork(is_active=True)
|
||||
f1 = asyncio.ensure_future(get_n(fork_1, 1))
|
||||
f2 = asyncio.ensure_future(get_n(fork_2, 1))
|
||||
run(asyncio.wait([f1, f2], timeout=0.1))
|
||||
self.assertFalse(f1.done())
|
||||
self.assertFalse(f2.done())
|
||||
q.push(1)
|
||||
run(asyncio.wait([f1, f2], timeout=0.1))
|
||||
self.assertTrue(f1.done())
|
||||
self.assertTrue(f2.done())
|
||||
r1 = run(f1)
|
||||
r2 = run(f2)
|
||||
self.assertEqual(r1, [1])
|
||||
self.assertEqual(r2, [1])
|
21
lib/aiter/tests/test_azip.py
Normal file
21
lib/aiter/tests/test_azip.py
Normal file
|
@ -0,0 +1,21 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import azip, iter_to_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_azip(self):
|
||||
i1 = ("abcdefgh")
|
||||
i2 = list(range(20))
|
||||
i3 = list(str(_) for _ in range(20))
|
||||
ai1 = iter_to_aiter(i1)
|
||||
ai2 = iter_to_aiter(i2)
|
||||
ai3 = iter_to_aiter(i3)
|
||||
ai = azip(ai1, ai2, ai3)
|
||||
r = run(get_n(ai))
|
||||
self.assertEqual(r, list(zip(i1, i2, i3)))
|
65
lib/aiter/tests/test_flatten_aiter.py
Normal file
65
lib/aiter/tests/test_flatten_aiter.py
Normal file
|
@ -0,0 +1,65 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import flatten_aiter, map_aiter, push_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_flatten_aiter(self):
|
||||
q = push_aiter()
|
||||
fi = flatten_aiter(q)
|
||||
r = []
|
||||
q.push([0, 1, 2, 3])
|
||||
r.extend(run(get_n(fi, 3)))
|
||||
q.push([4, 5, 6, 7])
|
||||
r.extend(run(get_n(fi, 5)))
|
||||
q.stop()
|
||||
r.extend(run(get_n(fi)))
|
||||
self.assertEqual(r, list(range(8)))
|
||||
|
||||
def test_make_simple_pipeline(self):
|
||||
q = push_aiter()
|
||||
aiter = flatten_aiter(flatten_aiter(q))
|
||||
q.push([
|
||||
(0, 0, 1, 0),
|
||||
(1, 1, 1, 1),
|
||||
(2, 0, 0, 1),
|
||||
(3, 1, 2, 0),
|
||||
(0, 0, 0, 7),
|
||||
])
|
||||
r = run(get_n(aiter, 11))
|
||||
self.assertEqual(r, [0, 0, 1, 0, 1, 1, 1, 1, 2, 0, 0])
|
||||
r.extend(run(get_n(aiter, 8)))
|
||||
q.stop()
|
||||
r.extend(run(get_n(aiter)))
|
||||
self.assertEqual(r, [0, 0, 1, 0, 1, 1, 1, 1, 2, 0, 0, 1, 3, 1, 2, 0, 0, 0, 0, 7])
|
||||
|
||||
def test_filter_pipeline(self):
|
||||
async def filter(item_list_of_lists):
|
||||
r = []
|
||||
for l1 in item_list_of_lists:
|
||||
for item in l1:
|
||||
if item != 0:
|
||||
r.append(item)
|
||||
return r
|
||||
|
||||
TEST_CASE = [
|
||||
(0, 0, 0, 7),
|
||||
(5, 0, 0, 0),
|
||||
(0, 0, 1, 0),
|
||||
(1, 1, 1, 1),
|
||||
(2, 0, 0, 1),
|
||||
(3, 1, 2, 0),
|
||||
]
|
||||
|
||||
q = push_aiter()
|
||||
aiter = flatten_aiter(map_aiter(filter, q))
|
||||
q.push(TEST_CASE)
|
||||
q.stop()
|
||||
r = run(get_n(aiter, 12))
|
||||
r1 = [7, 5, 1, 1, 1, 1, 1, 2, 1, 3, 1, 2]
|
||||
self.assertEqual(r, r1)
|
20
lib/aiter/tests/test_gated_aiter.py
Normal file
20
lib/aiter/tests/test_gated_aiter.py
Normal file
|
@ -0,0 +1,20 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import iter_to_aiter, gated_aiter
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_gated_aiter(self):
|
||||
ai = iter_to_aiter(range(3000000000))
|
||||
aiter = gated_aiter(ai)
|
||||
aiter.push(9)
|
||||
r = run(get_n(aiter, 3))
|
||||
r.extend(run(get_n(aiter, 4)))
|
||||
aiter.push(11)
|
||||
aiter.stop()
|
||||
r.extend(run(get_n(aiter)))
|
||||
self.assertEqual(r, list(range(20)))
|
5
lib/aiter/tests/test_iter_to_aiter.py
Normal file
5
lib/aiter/tests/test_iter_to_aiter.py
Normal file
|
@ -0,0 +1,5 @@
|
|||
import unittest
|
||||
|
||||
|
||||
class test_iter_to_aiter(unittest.TestCase):
|
||||
pass
|
68
lib/aiter/tests/test_join_aiters.py
Normal file
68
lib/aiter/tests/test_join_aiters.py
Normal file
|
@ -0,0 +1,68 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import iter_to_aiter, join_aiters, push_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_join_aiters(self):
|
||||
int_vals = [1, 2, 3, 4]
|
||||
str_vals = "abcdefg"
|
||||
|
||||
list_of_lists = [int_vals, str_vals]
|
||||
iter_of_aiters = [iter_to_aiter(_) for _ in list_of_lists]
|
||||
aiter_of_aiters = iter_to_aiter(iter_of_aiters)
|
||||
r = run(get_n(join_aiters(aiter_of_aiters)))
|
||||
|
||||
r1 = [_ for _ in r if isinstance(_, int)]
|
||||
r2 = [_ for _ in r if isinstance(_, str)]
|
||||
self.assertEqual(r1, int_vals)
|
||||
self.assertEqual(r2, list(str_vals))
|
||||
|
||||
def test_join_aiters_1(self):
|
||||
# make sure nothing's dropped
|
||||
# even if lots of events come in at once
|
||||
main_aiter = push_aiter()
|
||||
child_aiters = []
|
||||
aiter = join_aiters(main_aiter)
|
||||
|
||||
child_aiters.append(push_aiter())
|
||||
child_aiters[0].push(100)
|
||||
main_aiter.push(child_aiters[0])
|
||||
|
||||
t = run(get_n(aiter, 1))
|
||||
self.assertEqual(t, [100])
|
||||
|
||||
child_aiters.append(push_aiter())
|
||||
child_aiters[0].push(101)
|
||||
child_aiters[1].push(200)
|
||||
child_aiters[1].push(201)
|
||||
main_aiter.push(child_aiters[1])
|
||||
|
||||
t = run(get_n(aiter, 3))
|
||||
self.assertEqual(set(t), set([101, 200, 201]))
|
||||
|
||||
for _ in range(3):
|
||||
child_aiters.append(push_aiter())
|
||||
main_aiter.push(child_aiters[-1])
|
||||
for _, ca in enumerate(child_aiters):
|
||||
ca.push((_+1) * 100)
|
||||
ca.push((_+1) * 100 + 1)
|
||||
|
||||
t = run(get_n(aiter, len(child_aiters) * 2))
|
||||
self.assertEqual(set(t), set([100, 101, 200, 201, 300, 301, 400, 401, 500, 501]))
|
||||
|
||||
child_aiters[-1].push(5000)
|
||||
main_aiter.stop()
|
||||
t = run(get_n(aiter, 1))
|
||||
self.assertEqual(t, [5000])
|
||||
|
||||
for ca in child_aiters:
|
||||
ca.push(99)
|
||||
ca.stop()
|
||||
t = run(get_n(aiter))
|
||||
self.assertEqual(t, [99] * len(child_aiters))
|
66
lib/aiter/tests/test_map_aiter.py
Normal file
66
lib/aiter/tests/test_map_aiter.py
Normal file
|
@ -0,0 +1,66 @@
|
|||
import asyncio
|
||||
import unittest
|
||||
|
||||
|
||||
from aiter import map_aiter, push_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_asyncmap(self):
|
||||
|
||||
def make_async_transformation_f(results):
|
||||
async def async_transformation_f(item):
|
||||
results.append(item)
|
||||
return item + 1
|
||||
return async_transformation_f
|
||||
|
||||
results = []
|
||||
q = push_aiter()
|
||||
q.push(5, 4, 3)
|
||||
q.stop()
|
||||
r = list(q.available_iter())
|
||||
self.assertEqual(r, [5, 4, 3])
|
||||
aiter = map_aiter(make_async_transformation_f(results), q)
|
||||
r = run(get_n(aiter))
|
||||
self.assertEqual(r, [6, 5, 4])
|
||||
self.assertEqual(results, [5, 4, 3])
|
||||
|
||||
def test_syncmap(self):
|
||||
|
||||
def make_sync_transformation_f(results):
|
||||
def sync_transformation_f(item):
|
||||
results.append(item)
|
||||
return item + 1
|
||||
return sync_transformation_f
|
||||
|
||||
results = []
|
||||
q = push_aiter()
|
||||
q.push(5, 4, 3)
|
||||
q.stop()
|
||||
r = list(q.available_iter())
|
||||
self.assertEqual(r, [5, 4, 3])
|
||||
aiter = map_aiter(make_sync_transformation_f(results), q)
|
||||
r = run(get_n(aiter))
|
||||
self.assertEqual(r, [6, 5, 4])
|
||||
self.assertEqual(results, [5, 4, 3])
|
||||
|
||||
def test_make_pipe(self):
|
||||
async def map_f(x):
|
||||
await asyncio.sleep(x / 100.0)
|
||||
return x * x
|
||||
|
||||
q = push_aiter()
|
||||
aiter = map_aiter(map_f, q)
|
||||
for _ in range(4):
|
||||
q.push(_)
|
||||
for _ in range(3, 9):
|
||||
q.push(_)
|
||||
r = run(get_n(aiter, 10))
|
||||
q.stop()
|
||||
r.extend(run(get_n(aiter)))
|
||||
r1 = sorted([_*_ for _ in range(4)] + [_ * _ for _ in range(3, 9)])
|
||||
self.assertEqual(r, r1)
|
5
lib/aiter/tests/test_map_filter_aiter.py
Normal file
5
lib/aiter/tests/test_map_filter_aiter.py
Normal file
|
@ -0,0 +1,5 @@
|
|||
import unittest
|
||||
|
||||
|
||||
class test_map_filter_aiter(unittest.TestCase):
|
||||
pass
|
41
lib/aiter/tests/test_parallel_map_aiter.py
Normal file
41
lib/aiter/tests/test_parallel_map_aiter.py
Normal file
|
@ -0,0 +1,41 @@
|
|||
import asyncio
|
||||
import unittest
|
||||
|
||||
|
||||
from aiter import map_aiter, push_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_make_delayed_pipeline(self):
|
||||
def make_wait_index(idx):
|
||||
|
||||
async def wait(item):
|
||||
await asyncio.sleep(item[idx] / 10.)
|
||||
return item
|
||||
|
||||
return wait
|
||||
|
||||
TEST_CASE = [
|
||||
(0, 0, 0, 7),
|
||||
(5, 0, 0, 0),
|
||||
(0, 0, 1, 0),
|
||||
(1, 1, 1, 1),
|
||||
(2, 0, 0, 1),
|
||||
(3, 1, 2, 0),
|
||||
]
|
||||
|
||||
q = push_aiter()
|
||||
aiter = map_aiter(
|
||||
make_wait_index(0), map_aiter(
|
||||
make_wait_index(1), map_aiter(
|
||||
make_wait_index(2), map_aiter(
|
||||
make_wait_index(3), q, 10), 10), 10), 10)
|
||||
q.push(*TEST_CASE)
|
||||
q.stop()
|
||||
r = run(get_n(aiter))
|
||||
r1 = sorted(r, key=lambda x: sum(x))
|
||||
self.assertEqual(r, r1)
|
31
lib/aiter/tests/test_preload_aiter.py
Normal file
31
lib/aiter/tests/test_preload_aiter.py
Normal file
|
@ -0,0 +1,31 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import preload_aiter, push_aiter
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_preload_aiter(self):
|
||||
q = push_aiter()
|
||||
q.push(*list(range(1000)))
|
||||
q.stop()
|
||||
|
||||
self.assertEqual(len(q), 1000)
|
||||
aiter = preload_aiter(50, q)
|
||||
|
||||
self.assertEqual(len(q), 1000)
|
||||
|
||||
r = run(get_n(aiter, 1))
|
||||
self.assertEqual(len(q), 949)
|
||||
self.assertEqual(r, [0])
|
||||
|
||||
r = run(get_n(aiter, 10))
|
||||
self.assertEqual(r, list(range(1, 11)))
|
||||
self.assertEqual(len(q), 939)
|
||||
|
||||
r = run(get_n(aiter))
|
||||
self.assertEqual(r, list(range(11, 1000)))
|
||||
self.assertEqual(len(q), 0)
|
24
lib/aiter/tests/test_push_aiter.py
Normal file
24
lib/aiter/tests/test_push_aiter.py
Normal file
|
@ -0,0 +1,24 @@
|
|||
import unittest
|
||||
|
||||
|
||||
from aiter import push_aiter
|
||||
|
||||
|
||||
from .helpers import run, get_n
|
||||
|
||||
|
||||
class test_aitertools(unittest.TestCase):
|
||||
|
||||
def test_push_aiter(self):
|
||||
q = push_aiter()
|
||||
self.assertEqual(len(q), 0)
|
||||
q.push(5, 4)
|
||||
self.assertEqual(len(q), 2)
|
||||
q.push(3)
|
||||
self.assertEqual(len(q), 3)
|
||||
q.stop()
|
||||
self.assertRaises(ValueError, lambda: q.push(2))
|
||||
results = list(q.available_iter())
|
||||
self.assertEqual(results, [5, 4, 3])
|
||||
results = run(get_n(q))
|
||||
self.assertEqual(results, [5, 4, 3])
|
|
@ -1 +0,0 @@
|
|||
Subproject commit 34c2281e315c51f5270321101dc733c1cf26214f
|
70
lib/bip158/lib/pybind11/.appveyor.yml
Normal file
70
lib/bip158/lib/pybind11/.appveyor.yml
Normal file
|
@ -0,0 +1,70 @@
|
|||
version: 1.0.{build}
|
||||
image:
|
||||
- Visual Studio 2017
|
||||
- Visual Studio 2015
|
||||
test: off
|
||||
skip_branch_with_pr: true
|
||||
build:
|
||||
parallel: true
|
||||
platform:
|
||||
- x64
|
||||
- x86
|
||||
environment:
|
||||
matrix:
|
||||
- PYTHON: 36
|
||||
CPP: 14
|
||||
CONFIG: Debug
|
||||
- PYTHON: 27
|
||||
CPP: 14
|
||||
CONFIG: Debug
|
||||
- CONDA: 36
|
||||
CPP: latest
|
||||
CONFIG: Release
|
||||
matrix:
|
||||
exclude:
|
||||
- image: Visual Studio 2015
|
||||
platform: x86
|
||||
- image: Visual Studio 2015
|
||||
CPP: latest
|
||||
- image: Visual Studio 2017
|
||||
CPP: latest
|
||||
platform: x86
|
||||
install:
|
||||
- ps: |
|
||||
if ($env:PLATFORM -eq "x64") { $env:CMAKE_ARCH = "x64" }
|
||||
if ($env:APPVEYOR_JOB_NAME -like "*Visual Studio 2017*") {
|
||||
$env:CMAKE_GENERATOR = "Visual Studio 15 2017"
|
||||
$env:CMAKE_INCLUDE_PATH = "C:\Libraries\boost_1_64_0"
|
||||
$env:CXXFLAGS = "-permissive-"
|
||||
} else {
|
||||
$env:CMAKE_GENERATOR = "Visual Studio 14 2015"
|
||||
}
|
||||
if ($env:PYTHON) {
|
||||
if ($env:PLATFORM -eq "x64") { $env:PYTHON = "$env:PYTHON-x64" }
|
||||
$env:PATH = "C:\Python$env:PYTHON\;C:\Python$env:PYTHON\Scripts\;$env:PATH"
|
||||
python -W ignore -m pip install --upgrade pip wheel
|
||||
python -W ignore -m pip install pytest numpy --no-warn-script-location
|
||||
} elseif ($env:CONDA) {
|
||||
if ($env:CONDA -eq "27") { $env:CONDA = "" }
|
||||
if ($env:PLATFORM -eq "x64") { $env:CONDA = "$env:CONDA-x64" }
|
||||
$env:PATH = "C:\Miniconda$env:CONDA\;C:\Miniconda$env:CONDA\Scripts\;$env:PATH"
|
||||
$env:PYTHONHOME = "C:\Miniconda$env:CONDA"
|
||||
conda --version
|
||||
conda install -y -q pytest numpy scipy
|
||||
}
|
||||
- ps: |
|
||||
Start-FileDownload 'http://bitbucket.org/eigen/eigen/get/3.3.3.zip'
|
||||
7z x 3.3.3.zip -y > $null
|
||||
$env:CMAKE_INCLUDE_PATH = "eigen-eigen-67e894c6cd8f;$env:CMAKE_INCLUDE_PATH"
|
||||
build_script:
|
||||
- cmake -G "%CMAKE_GENERATOR%" -A "%CMAKE_ARCH%"
|
||||
-DPYBIND11_CPP_STANDARD=/std:c++%CPP%
|
||||
-DPYBIND11_WERROR=ON
|
||||
-DDOWNLOAD_CATCH=ON
|
||||
-DCMAKE_SUPPRESS_REGENERATION=1
|
||||
.
|
||||
- set MSBuildLogger="C:\Program Files\AppVeyor\BuildAgent\Appveyor.MSBuildLogger.dll"
|
||||
- cmake --build . --config %CONFIG% --target pytest -- /m /v:m /logger:%MSBuildLogger%
|
||||
- cmake --build . --config %CONFIG% --target cpptest -- /m /v:m /logger:%MSBuildLogger%
|
||||
- if "%CPP%"=="latest" (cmake --build . --config %CONFIG% --target test_cmake_build -- /m /v:m /logger:%MSBuildLogger%)
|
||||
on_failure: if exist "tests\test_cmake_build" type tests\test_cmake_build\*.log*
|
38
lib/bip158/lib/pybind11/.gitignore
vendored
Normal file
38
lib/bip158/lib/pybind11/.gitignore
vendored
Normal file
|
@ -0,0 +1,38 @@
|
|||
CMakeCache.txt
|
||||
CMakeFiles
|
||||
Makefile
|
||||
cmake_install.cmake
|
||||
.DS_Store
|
||||
*.so
|
||||
*.pyd
|
||||
*.dll
|
||||
*.sln
|
||||
*.sdf
|
||||
*.opensdf
|
||||
*.vcxproj
|
||||
*.filters
|
||||
example.dir
|
||||
Win32
|
||||
x64
|
||||
Release
|
||||
Debug
|
||||
.vs
|
||||
CTestTestfile.cmake
|
||||
Testing
|
||||
autogen
|
||||
MANIFEST
|
||||
/.ninja_*
|
||||
/*.ninja
|
||||
/docs/.build
|
||||
*.py[co]
|
||||
*.egg-info
|
||||
*~
|
||||
.*.swp
|
||||
.DS_Store
|
||||
/dist
|
||||
/build
|
||||
/cmake/
|
||||
.cache/
|
||||
sosize-*.txt
|
||||
pybind11Config*.cmake
|
||||
pybind11Targets.cmake
|
3
lib/bip158/lib/pybind11/.gitmodules
vendored
Normal file
3
lib/bip158/lib/pybind11/.gitmodules
vendored
Normal file
|
@ -0,0 +1,3 @@
|
|||
[submodule "tools/clang"]
|
||||
path = tools/clang
|
||||
url = ../../wjakob/clang-cindex-python3
|
3
lib/bip158/lib/pybind11/.readthedocs.yml
Normal file
3
lib/bip158/lib/pybind11/.readthedocs.yml
Normal file
|
@ -0,0 +1,3 @@
|
|||
python:
|
||||
version: 3
|
||||
requirements_file: docs/requirements.txt
|
305
lib/bip158/lib/pybind11/.travis.yml
Normal file
305
lib/bip158/lib/pybind11/.travis.yml
Normal file
|
@ -0,0 +1,305 @@
|
|||
language: cpp
|
||||
matrix:
|
||||
include:
|
||||
# This config does a few things:
|
||||
# - Checks C++ and Python code styles (check-style.sh and flake8).
|
||||
# - Makes sure sphinx can build the docs without any errors or warnings.
|
||||
# - Tests setup.py sdist and install (all header files should be present).
|
||||
# - Makes sure that everything still works without optional deps (numpy/scipy/eigen) and
|
||||
# also tests the automatic discovery functions in CMake (Python version, C++ standard).
|
||||
- os: linux
|
||||
dist: xenial # Necessary to run doxygen 1.8.15
|
||||
name: Style, docs, and pip
|
||||
cache: false
|
||||
before_install:
|
||||
- pyenv global $(pyenv whence 2to3) # activate all python versions
|
||||
- PY_CMD=python3
|
||||
- $PY_CMD -m pip install --user --upgrade pip wheel setuptools
|
||||
install:
|
||||
- $PY_CMD -m pip install --user --upgrade sphinx sphinx_rtd_theme breathe flake8 pep8-naming pytest
|
||||
- curl -fsSL https://sourceforge.net/projects/doxygen/files/rel-1.8.15/doxygen-1.8.15.linux.bin.tar.gz/download | tar xz
|
||||
- export PATH="$PWD/doxygen-1.8.15/bin:$PATH"
|
||||
script:
|
||||
- tools/check-style.sh
|
||||
- flake8
|
||||
- $PY_CMD -m sphinx -W -b html docs docs/.build
|
||||
- |
|
||||
# Make sure setup.py distributes and installs all the headers
|
||||
$PY_CMD setup.py sdist
|
||||
$PY_CMD -m pip install --user -U ./dist/*
|
||||
installed=$($PY_CMD -c "import pybind11; print(pybind11.get_include(True) + '/pybind11')")
|
||||
diff -rq $installed ./include/pybind11
|
||||
- |
|
||||
# Barebones build
|
||||
cmake -DCMAKE_BUILD_TYPE=Debug -DPYBIND11_WERROR=ON -DDOWNLOAD_CATCH=ON -DPYTHON_EXECUTABLE=$(which $PY_CMD) .
|
||||
make pytest -j 2 && make cpptest -j 2
|
||||
# The following are regular test configurations, including optional dependencies.
|
||||
# With regard to each other they differ in Python version, C++ standard and compiler.
|
||||
- os: linux
|
||||
dist: trusty
|
||||
name: Python 2.7, c++11, gcc 4.8
|
||||
env: PYTHON=2.7 CPP=11 GCC=4.8
|
||||
addons:
|
||||
apt:
|
||||
packages:
|
||||
- cmake=2.\*
|
||||
- cmake-data=2.\*
|
||||
- os: linux
|
||||
dist: trusty
|
||||
name: Python 3.6, c++11, gcc 4.8
|
||||
env: PYTHON=3.6 CPP=11 GCC=4.8
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- deadsnakes
|
||||
packages:
|
||||
- python3.6-dev
|
||||
- python3.6-venv
|
||||
- cmake=2.\*
|
||||
- cmake-data=2.\*
|
||||
- os: linux
|
||||
dist: trusty
|
||||
env: PYTHON=2.7 CPP=14 GCC=6 CMAKE=1
|
||||
name: Python 2.7, c++14, gcc 6, CMake test
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- ubuntu-toolchain-r-test
|
||||
packages:
|
||||
- g++-6
|
||||
- os: linux
|
||||
dist: trusty
|
||||
name: Python 3.5, c++14, gcc 6, Debug build
|
||||
# N.B. `ensurepip` could be installed transitively by `python3.5-venv`, but
|
||||
# seems to have apt conflicts (at least for Trusty). Use Docker instead.
|
||||
services: docker
|
||||
env: DOCKER=debian:stretch PYTHON=3.5 CPP=14 GCC=6 DEBUG=1
|
||||
- os: linux
|
||||
dist: xenial
|
||||
env: PYTHON=3.6 CPP=17 GCC=7
|
||||
name: Python 3.6, c++17, gcc 7
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- deadsnakes
|
||||
- ubuntu-toolchain-r-test
|
||||
packages:
|
||||
- g++-7
|
||||
- python3.6-dev
|
||||
- python3.6-venv
|
||||
- os: linux
|
||||
dist: xenial
|
||||
env: PYTHON=3.6 CPP=17 CLANG=7
|
||||
name: Python 3.6, c++17, Clang 7
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- deadsnakes
|
||||
- llvm-toolchain-xenial-7
|
||||
packages:
|
||||
- python3.6-dev
|
||||
- python3.6-venv
|
||||
- clang-7
|
||||
- libclang-7-dev
|
||||
- llvm-7-dev
|
||||
- lld-7
|
||||
- libc++-7-dev
|
||||
- libc++abi-7-dev # Why is this necessary???
|
||||
- os: linux
|
||||
dist: xenial
|
||||
env: PYTHON=3.8 CPP=17 GCC=7
|
||||
name: Python 3.8, c++17, gcc 7 (w/o numpy/scipy) # TODO: update build name when the numpy/scipy wheels become available
|
||||
addons:
|
||||
apt:
|
||||
sources:
|
||||
- deadsnakes
|
||||
- ubuntu-toolchain-r-test
|
||||
packages:
|
||||
- g++-7
|
||||
- python3.8-dev
|
||||
- python3.8-venv
|
||||
# Currently there is no numpy/scipy wheels available for python3.8
|
||||
# TODO: remove next before_install, install and script clause when the wheels become available
|
||||
before_install:
|
||||
- pyenv global $(pyenv whence 2to3) # activate all python versions
|
||||
- PY_CMD=python3
|
||||
- $PY_CMD -m pip install --user --upgrade pip wheel setuptools
|
||||
install:
|
||||
- $PY_CMD -m pip install --user --upgrade pytest
|
||||
script:
|
||||
- |
|
||||
# Barebones build
|
||||
cmake -DCMAKE_BUILD_TYPE=Debug -DPYBIND11_WERROR=ON -DDOWNLOAD_CATCH=ON -DPYTHON_EXECUTABLE=$(which $PY_CMD) .
|
||||
make pytest -j 2 && make cpptest -j 2
|
||||
- os: osx
|
||||
name: Python 2.7, c++14, AppleClang 7.3, CMake test
|
||||
osx_image: xcode7.3
|
||||
env: PYTHON=2.7 CPP=14 CLANG CMAKE=1
|
||||
- os: osx
|
||||
name: Python 3.7, c++14, AppleClang 9, Debug build
|
||||
osx_image: xcode9
|
||||
env: PYTHON=3.7 CPP=14 CLANG DEBUG=1
|
||||
# Test a PyPy 2.7 build
|
||||
- os: linux
|
||||
dist: trusty
|
||||
env: PYPY=5.8 PYTHON=2.7 CPP=11 GCC=4.8
|
||||
name: PyPy 5.8, Python 2.7, c++11, gcc 4.8
|
||||
addons:
|
||||
apt:
|
||||
packages:
|
||||
- libblas-dev
|
||||
- liblapack-dev
|
||||
- gfortran
|
||||
# Build in 32-bit mode and tests against the CMake-installed version
|
||||
- os: linux
|
||||
dist: trusty
|
||||
services: docker
|
||||
env: DOCKER=i386/debian:stretch PYTHON=3.5 CPP=14 GCC=6 INSTALL=1
|
||||
name: Python 3.5, c++14, gcc 6, 32-bit
|
||||
script:
|
||||
- |
|
||||
# Consolidated 32-bit Docker Build + Install
|
||||
set -ex
|
||||
$SCRIPT_RUN_PREFIX sh -c "
|
||||
set -ex
|
||||
cmake ${CMAKE_EXTRA_ARGS} -DPYBIND11_INSTALL=1 -DPYBIND11_TEST=0 .
|
||||
make install
|
||||
cp -a tests /pybind11-tests
|
||||
mkdir /build-tests && cd /build-tests
|
||||
cmake ../pybind11-tests ${CMAKE_EXTRA_ARGS} -DPYBIND11_WERROR=ON
|
||||
make pytest -j 2"
|
||||
set +ex
|
||||
cache:
|
||||
directories:
|
||||
- $HOME/.local/bin
|
||||
- $HOME/.local/lib
|
||||
- $HOME/.local/include
|
||||
- $HOME/Library/Python
|
||||
before_install:
|
||||
- |
|
||||
# Configure build variables
|
||||
set -ex
|
||||
if [ "$TRAVIS_OS_NAME" = "linux" ]; then
|
||||
if [ -n "$CLANG" ]; then
|
||||
export CXX=clang++-$CLANG CC=clang-$CLANG
|
||||
EXTRA_PACKAGES+=" clang-$CLANG llvm-$CLANG-dev"
|
||||
else
|
||||
if [ -z "$GCC" ]; then GCC=4.8
|
||||
else EXTRA_PACKAGES+=" g++-$GCC"
|
||||
fi
|
||||
export CXX=g++-$GCC CC=gcc-$GCC
|
||||
fi
|
||||
elif [ "$TRAVIS_OS_NAME" = "osx" ]; then
|
||||
export CXX=clang++ CC=clang;
|
||||
fi
|
||||
if [ -n "$CPP" ]; then CPP=-std=c++$CPP; fi
|
||||
if [ "${PYTHON:0:1}" = "3" ]; then PY=3; fi
|
||||
if [ -n "$DEBUG" ]; then CMAKE_EXTRA_ARGS+=" -DCMAKE_BUILD_TYPE=Debug"; fi
|
||||
set +ex
|
||||
- |
|
||||
# Initialize environment
|
||||
set -ex
|
||||
if [ -n "$DOCKER" ]; then
|
||||
docker pull $DOCKER
|
||||
|
||||
containerid=$(docker run --detach --tty \
|
||||
--volume="$PWD":/pybind11 --workdir=/pybind11 \
|
||||
--env="CC=$CC" --env="CXX=$CXX" --env="DEBIAN_FRONTEND=$DEBIAN_FRONTEND" \
|
||||
--env=GCC_COLORS=\ \
|
||||
$DOCKER)
|
||||
SCRIPT_RUN_PREFIX="docker exec --tty $containerid"
|
||||
$SCRIPT_RUN_PREFIX sh -c 'for s in 0 15; do sleep $s; apt-get update && apt-get -qy dist-upgrade && break; done'
|
||||
else
|
||||
if [ "$PYPY" = "5.8" ]; then
|
||||
curl -fSL https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.8.0-linux64.tar.bz2 | tar xj
|
||||
PY_CMD=$(echo `pwd`/pypy2-v5.8.0-linux64/bin/pypy)
|
||||
CMAKE_EXTRA_ARGS+=" -DPYTHON_EXECUTABLE:FILEPATH=$PY_CMD"
|
||||
else
|
||||
PY_CMD=python$PYTHON
|
||||
if [ "$TRAVIS_OS_NAME" = "osx" ]; then
|
||||
if [ "$PY" = "3" ]; then
|
||||
brew update && brew upgrade python
|
||||
else
|
||||
curl -fsSL https://bootstrap.pypa.io/get-pip.py | $PY_CMD - --user
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
if [ "$PY" = 3 ] || [ -n "$PYPY" ]; then
|
||||
$PY_CMD -m ensurepip --user
|
||||
fi
|
||||
$PY_CMD --version
|
||||
$PY_CMD -m pip install --user --upgrade pip wheel
|
||||
fi
|
||||
set +ex
|
||||
install:
|
||||
- |
|
||||
# Install dependencies
|
||||
set -ex
|
||||
cmake --version
|
||||
if [ -n "$DOCKER" ]; then
|
||||
if [ -n "$DEBUG" ]; then
|
||||
PY_DEBUG="python$PYTHON-dbg python$PY-scipy-dbg"
|
||||
CMAKE_EXTRA_ARGS+=" -DPYTHON_EXECUTABLE=/usr/bin/python${PYTHON}dm"
|
||||
fi
|
||||
$SCRIPT_RUN_PREFIX sh -c "for s in 0 15; do sleep \$s; \
|
||||
apt-get -qy --no-install-recommends install \
|
||||
$PY_DEBUG python$PYTHON-dev python$PY-pytest python$PY-scipy \
|
||||
libeigen3-dev libboost-dev cmake make ${EXTRA_PACKAGES} && break; done"
|
||||
else
|
||||
|
||||
if [ "$CLANG" = "7" ]; then
|
||||
export CXXFLAGS="-stdlib=libc++"
|
||||
fi
|
||||
|
||||
export NPY_NUM_BUILD_JOBS=2
|
||||
echo "Installing pytest, numpy, scipy..."
|
||||
local PIP_CMD=""
|
||||
if [ -n $PYPY ]; then
|
||||
# For expediency, install only versions that are available on the extra index.
|
||||
travis_wait 30 \
|
||||
$PY_CMD -m pip install --user --upgrade --extra-index-url https://imaginary.ca/trusty-pypi \
|
||||
pytest numpy==1.15.4 scipy==1.2.0
|
||||
else
|
||||
$PY_CMD -m pip install --user --upgrade pytest numpy scipy
|
||||
fi
|
||||
echo "done."
|
||||
|
||||
mkdir eigen
|
||||
curl -fsSL https://bitbucket.org/eigen/eigen/get/3.3.4.tar.bz2 | \
|
||||
tar --extract -j --directory=eigen --strip-components=1
|
||||
export CMAKE_INCLUDE_PATH="${CMAKE_INCLUDE_PATH:+$CMAKE_INCLUDE_PATH:}$PWD/eigen"
|
||||
fi
|
||||
set +ex
|
||||
script:
|
||||
- |
|
||||
# CMake Configuration
|
||||
set -ex
|
||||
$SCRIPT_RUN_PREFIX cmake ${CMAKE_EXTRA_ARGS} \
|
||||
-DPYBIND11_PYTHON_VERSION=$PYTHON \
|
||||
-DPYBIND11_CPP_STANDARD=$CPP \
|
||||
-DPYBIND11_WERROR=${WERROR:-ON} \
|
||||
-DDOWNLOAD_CATCH=${DOWNLOAD_CATCH:-ON} \
|
||||
.
|
||||
set +ex
|
||||
- |
|
||||
# pytest
|
||||
set -ex
|
||||
$SCRIPT_RUN_PREFIX make pytest -j 2 VERBOSE=1
|
||||
set +ex
|
||||
- |
|
||||
# cpptest
|
||||
set -ex
|
||||
$SCRIPT_RUN_PREFIX make cpptest -j 2
|
||||
set +ex
|
||||
- |
|
||||
# CMake Build Interface
|
||||
set -ex
|
||||
if [ -n "$CMAKE" ]; then $SCRIPT_RUN_PREFIX make test_cmake_build; fi
|
||||
set +ex
|
||||
after_failure: cat tests/test_cmake_build/*.log*
|
||||
after_script:
|
||||
- |
|
||||
# Cleanup (Docker)
|
||||
set -ex
|
||||
if [ -n "$DOCKER" ]; then docker stop "$containerid"; docker rm "$containerid"; fi
|
||||
set +ex
|
157
lib/bip158/lib/pybind11/CMakeLists.txt
Normal file
157
lib/bip158/lib/pybind11/CMakeLists.txt
Normal file
|
@ -0,0 +1,157 @@
|
|||
# CMakeLists.txt -- Build system for the pybind11 modules
|
||||
#
|
||||
# Copyright (c) 2015 Wenzel Jakob <wenzel@inf.ethz.ch>
|
||||
#
|
||||
# All rights reserved. Use of this source code is governed by a
|
||||
# BSD-style license that can be found in the LICENSE file.
|
||||
|
||||
cmake_minimum_required(VERSION 2.8.12)
|
||||
|
||||
if (POLICY CMP0048)
|
||||
# cmake warns if loaded from a min-3.0-required parent dir, so silence the warning:
|
||||
cmake_policy(SET CMP0048 NEW)
|
||||
endif()
|
||||
|
||||
# CMake versions < 3.4.0 do not support try_compile/pthread checks without C as active language.
|
||||
if(CMAKE_VERSION VERSION_LESS 3.4.0)
|
||||
project(pybind11)
|
||||
else()
|
||||
project(pybind11 CXX)
|
||||
endif()
|
||||
|
||||
# Check if pybind11 is being used directly or via add_subdirectory
|
||||
set(PYBIND11_MASTER_PROJECT OFF)
|
||||
if (CMAKE_CURRENT_SOURCE_DIR STREQUAL CMAKE_SOURCE_DIR)
|
||||
set(PYBIND11_MASTER_PROJECT ON)
|
||||
endif()
|
||||
|
||||
option(PYBIND11_INSTALL "Install pybind11 header files?" ${PYBIND11_MASTER_PROJECT})
|
||||
option(PYBIND11_TEST "Build pybind11 test suite?" ${PYBIND11_MASTER_PROJECT})
|
||||
|
||||
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/tools")
|
||||
|
||||
include(pybind11Tools)
|
||||
|
||||
# Cache variables so pybind11_add_module can be used in parent projects
|
||||
set(PYBIND11_INCLUDE_DIR "${CMAKE_CURRENT_LIST_DIR}/include" CACHE INTERNAL "")
|
||||
set(PYTHON_INCLUDE_DIRS ${PYTHON_INCLUDE_DIRS} CACHE INTERNAL "")
|
||||
set(PYTHON_LIBRARIES ${PYTHON_LIBRARIES} CACHE INTERNAL "")
|
||||
set(PYTHON_MODULE_PREFIX ${PYTHON_MODULE_PREFIX} CACHE INTERNAL "")
|
||||
set(PYTHON_MODULE_EXTENSION ${PYTHON_MODULE_EXTENSION} CACHE INTERNAL "")
|
||||
set(PYTHON_VERSION_MAJOR ${PYTHON_VERSION_MAJOR} CACHE INTERNAL "")
|
||||
set(PYTHON_VERSION_MINOR ${PYTHON_VERSION_MINOR} CACHE INTERNAL "")
|
||||
|
||||
# NB: when adding a header don't forget to also add it to setup.py
|
||||
set(PYBIND11_HEADERS
|
||||
include/pybind11/detail/class.h
|
||||
include/pybind11/detail/common.h
|
||||
include/pybind11/detail/descr.h
|
||||
include/pybind11/detail/init.h
|
||||
include/pybind11/detail/internals.h
|
||||
include/pybind11/detail/typeid.h
|
||||
include/pybind11/attr.h
|
||||
include/pybind11/buffer_info.h
|
||||
include/pybind11/cast.h
|
||||
include/pybind11/chrono.h
|
||||
include/pybind11/common.h
|
||||
include/pybind11/complex.h
|
||||
include/pybind11/options.h
|
||||
include/pybind11/eigen.h
|
||||
include/pybind11/embed.h
|
||||
include/pybind11/eval.h
|
||||
include/pybind11/functional.h
|
||||
include/pybind11/numpy.h
|
||||
include/pybind11/operators.h
|
||||
include/pybind11/pybind11.h
|
||||
include/pybind11/pytypes.h
|
||||
include/pybind11/stl.h
|
||||
include/pybind11/stl_bind.h
|
||||
)
|
||||
string(REPLACE "include/" "${CMAKE_CURRENT_SOURCE_DIR}/include/"
|
||||
PYBIND11_HEADERS "${PYBIND11_HEADERS}")
|
||||
|
||||
if (PYBIND11_TEST)
|
||||
add_subdirectory(tests)
|
||||
endif()
|
||||
|
||||
include(GNUInstallDirs)
|
||||
include(CMakePackageConfigHelpers)
|
||||
|
||||
# extract project version from source
|
||||
file(STRINGS "${PYBIND11_INCLUDE_DIR}/pybind11/detail/common.h" pybind11_version_defines
|
||||
REGEX "#define PYBIND11_VERSION_(MAJOR|MINOR|PATCH) ")
|
||||
foreach(ver ${pybind11_version_defines})
|
||||
if (ver MATCHES "#define PYBIND11_VERSION_(MAJOR|MINOR|PATCH) +([^ ]+)$")
|
||||
set(PYBIND11_VERSION_${CMAKE_MATCH_1} "${CMAKE_MATCH_2}" CACHE INTERNAL "")
|
||||
endif()
|
||||
endforeach()
|
||||
set(${PROJECT_NAME}_VERSION ${PYBIND11_VERSION_MAJOR}.${PYBIND11_VERSION_MINOR}.${PYBIND11_VERSION_PATCH})
|
||||
message(STATUS "pybind11 v${${PROJECT_NAME}_VERSION}")
|
||||
|
||||
option (USE_PYTHON_INCLUDE_DIR "Install pybind11 headers in Python include directory instead of default installation prefix" OFF)
|
||||
if (USE_PYTHON_INCLUDE_DIR)
|
||||
file(RELATIVE_PATH CMAKE_INSTALL_INCLUDEDIR ${CMAKE_INSTALL_PREFIX} ${PYTHON_INCLUDE_DIRS})
|
||||
endif()
|
||||
|
||||
if(NOT (CMAKE_VERSION VERSION_LESS 3.0)) # CMake >= 3.0
|
||||
# Build an interface library target:
|
||||
add_library(pybind11 INTERFACE)
|
||||
add_library(pybind11::pybind11 ALIAS pybind11) # to match exported target
|
||||
target_include_directories(pybind11 INTERFACE $<BUILD_INTERFACE:${PYBIND11_INCLUDE_DIR}>
|
||||
$<BUILD_INTERFACE:${PYTHON_INCLUDE_DIRS}>
|
||||
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
|
||||
target_compile_options(pybind11 INTERFACE $<BUILD_INTERFACE:${PYBIND11_CPP_STANDARD}>)
|
||||
|
||||
add_library(module INTERFACE)
|
||||
add_library(pybind11::module ALIAS module)
|
||||
if(NOT MSVC)
|
||||
target_compile_options(module INTERFACE -fvisibility=hidden)
|
||||
endif()
|
||||
target_link_libraries(module INTERFACE pybind11::pybind11)
|
||||
if(WIN32 OR CYGWIN)
|
||||
target_link_libraries(module INTERFACE $<BUILD_INTERFACE:${PYTHON_LIBRARIES}>)
|
||||
elseif(APPLE)
|
||||
target_link_libraries(module INTERFACE "-undefined dynamic_lookup")
|
||||
endif()
|
||||
|
||||
add_library(embed INTERFACE)
|
||||
add_library(pybind11::embed ALIAS embed)
|
||||
target_link_libraries(embed INTERFACE pybind11::pybind11 $<BUILD_INTERFACE:${PYTHON_LIBRARIES}>)
|
||||
endif()
|
||||
|
||||
if (PYBIND11_INSTALL)
|
||||
install(DIRECTORY ${PYBIND11_INCLUDE_DIR}/pybind11 DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
|
||||
# GNUInstallDirs "DATADIR" wrong here; CMake search path wants "share".
|
||||
set(PYBIND11_CMAKECONFIG_INSTALL_DIR "share/cmake/${PROJECT_NAME}" CACHE STRING "install path for pybind11Config.cmake")
|
||||
|
||||
configure_package_config_file(tools/${PROJECT_NAME}Config.cmake.in
|
||||
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
|
||||
INSTALL_DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})
|
||||
# Remove CMAKE_SIZEOF_VOID_P from ConfigVersion.cmake since the library does
|
||||
# not depend on architecture specific settings or libraries.
|
||||
set(_PYBIND11_CMAKE_SIZEOF_VOID_P ${CMAKE_SIZEOF_VOID_P})
|
||||
unset(CMAKE_SIZEOF_VOID_P)
|
||||
write_basic_package_version_file(${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake
|
||||
VERSION ${${PROJECT_NAME}_VERSION}
|
||||
COMPATIBILITY AnyNewerVersion)
|
||||
set(CMAKE_SIZEOF_VOID_P ${_PYBIND11_CMAKE_SIZEOF_VOID_P})
|
||||
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake
|
||||
${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake
|
||||
tools/FindPythonLibsNew.cmake
|
||||
tools/pybind11Tools.cmake
|
||||
DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})
|
||||
|
||||
if(NOT (CMAKE_VERSION VERSION_LESS 3.0))
|
||||
if(NOT PYBIND11_EXPORT_NAME)
|
||||
set(PYBIND11_EXPORT_NAME "${PROJECT_NAME}Targets")
|
||||
endif()
|
||||
|
||||
install(TARGETS pybind11 module embed
|
||||
EXPORT "${PYBIND11_EXPORT_NAME}")
|
||||
if(PYBIND11_MASTER_PROJECT)
|
||||
install(EXPORT "${PYBIND11_EXPORT_NAME}"
|
||||
NAMESPACE "${PROJECT_NAME}::"
|
||||
DESTINATION ${PYBIND11_CMAKECONFIG_INSTALL_DIR})
|
||||
endif()
|
||||
endif()
|
||||
endif()
|
49
lib/bip158/lib/pybind11/CONTRIBUTING.md
Normal file
49
lib/bip158/lib/pybind11/CONTRIBUTING.md
Normal file
|
@ -0,0 +1,49 @@
|
|||
Thank you for your interest in this project! Please refer to the following
|
||||
sections on how to contribute code and bug reports.
|
||||
|
||||
### Reporting bugs
|
||||
|
||||
At the moment, this project is run in the spare time of a single person
|
||||
([Wenzel Jakob](http://rgl.epfl.ch/people/wjakob)) with very limited resources
|
||||
for issue tracker tickets. Thus, before submitting a question or bug report,
|
||||
please take a moment of your time and ensure that your issue isn't already
|
||||
discussed in the project documentation provided at
|
||||
[http://pybind11.readthedocs.org/en/latest](http://pybind11.readthedocs.org/en/latest).
|
||||
|
||||
Assuming that you have identified a previously unknown problem or an important
|
||||
question, it's essential that you submit a self-contained and minimal piece of
|
||||
code that reproduces the problem. In other words: no external dependencies,
|
||||
isolate the function(s) that cause breakage, submit matched and complete C++
|
||||
and Python snippets that can be easily compiled and run on my end.
|
||||
|
||||
## Pull requests
|
||||
Contributions are submitted, reviewed, and accepted using Github pull requests.
|
||||
Please refer to [this
|
||||
article](https://help.github.com/articles/using-pull-requests) for details and
|
||||
adhere to the following rules to make the process as smooth as possible:
|
||||
|
||||
* Make a new branch for every feature you're working on.
|
||||
* Make small and clean pull requests that are easy to review but make sure they
|
||||
do add value by themselves.
|
||||
* Add tests for any new functionality and run the test suite (``make pytest``)
|
||||
to ensure that no existing features break.
|
||||
* Please run ``flake8`` and ``tools/check-style.sh`` to check your code matches
|
||||
the project style. (Note that ``check-style.sh`` requires ``gawk``.)
|
||||
* This project has a strong focus on providing general solutions using a
|
||||
minimal amount of code, thus small pull requests are greatly preferred.
|
||||
|
||||
### Licensing of contributions
|
||||
|
||||
pybind11 is provided under a BSD-style license that can be found in the
|
||||
``LICENSE`` file. By using, distributing, or contributing to this project, you
|
||||
agree to the terms and conditions of this license.
|
||||
|
||||
You are under no obligation whatsoever to provide any bug fixes, patches, or
|
||||
upgrades to the features, functionality or performance of the source code
|
||||
("Enhancements") to anyone; however, if you choose to make your Enhancements
|
||||
available either publicly, or directly to the author of this software, without
|
||||
imposing a separate written license agreement for such Enhancements, then you
|
||||
hereby grant the following license: a non-exclusive, royalty-free perpetual
|
||||
license to install, use, modify, prepare derivative works, incorporate into
|
||||
other computer software, distribute, and sublicense such enhancements or
|
||||
derivative works thereof, in binary and source code form.
|
17
lib/bip158/lib/pybind11/ISSUE_TEMPLATE.md
Normal file
17
lib/bip158/lib/pybind11/ISSUE_TEMPLATE.md
Normal file
|
@ -0,0 +1,17 @@
|
|||
Make sure you've completed the following steps before submitting your issue -- thank you!
|
||||
|
||||
1. Check if your question has already been answered in the [FAQ](http://pybind11.readthedocs.io/en/latest/faq.html) section.
|
||||
2. Make sure you've read the [documentation](http://pybind11.readthedocs.io/en/latest/). Your issue may be addressed there.
|
||||
3. If those resources didn't help and you only have a short question (not a bug report), consider asking in the [Gitter chat room](https://gitter.im/pybind/Lobby).
|
||||
4. If you have a genuine bug report or a more complex question which is not answered in the previous items (or not suitable for chat), please fill in the details below.
|
||||
5. Include a self-contained and minimal piece of code that reproduces the problem. If that's not possible, try to make the description as clear as possible.
|
||||
|
||||
*After reading, remove this checklist and the template text in parentheses below.*
|
||||
|
||||
## Issue description
|
||||
|
||||
(Provide a short description, state the expected behavior and what actually happens.)
|
||||
|
||||
## Reproducible example code
|
||||
|
||||
(The code should be minimal, have no external dependencies, isolate the function(s) that cause breakage. Submit matched and complete C++ and Python snippets that can be easily compiled and run to diagnose the issue.)
|
29
lib/bip158/lib/pybind11/LICENSE
Normal file
29
lib/bip158/lib/pybind11/LICENSE
Normal file
|
@ -0,0 +1,29 @@
|
|||
Copyright (c) 2016 Wenzel Jakob <wenzel.jakob@epfl.ch>, All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
3. Neither the name of the copyright holder nor the names of its contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
||||
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
||||
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
||||
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
||||
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
Please also refer to the file CONTRIBUTING.md, which clarifies licensing of
|
||||
external contributions to this project including patches, pull requests, etc.
|
2
lib/bip158/lib/pybind11/MANIFEST.in
Normal file
2
lib/bip158/lib/pybind11/MANIFEST.in
Normal file
|
@ -0,0 +1,2 @@
|
|||
recursive-include include/pybind11 *.h
|
||||
include LICENSE README.md CONTRIBUTING.md
|
129
lib/bip158/lib/pybind11/README.md
Normal file
129
lib/bip158/lib/pybind11/README.md
Normal file
|
@ -0,0 +1,129 @@
|
|||
![pybind11 logo](https://github.com/pybind/pybind11/raw/master/docs/pybind11-logo.png)
|
||||
|
||||
# pybind11 — Seamless operability between C++11 and Python
|
||||
|
||||
[![Documentation Status](https://readthedocs.org/projects/pybind11/badge/?version=master)](http://pybind11.readthedocs.org/en/master/?badge=master)
|
||||
[![Documentation Status](https://readthedocs.org/projects/pybind11/badge/?version=stable)](http://pybind11.readthedocs.org/en/stable/?badge=stable)
|
||||
[![Gitter chat](https://img.shields.io/gitter/room/gitterHQ/gitter.svg)](https://gitter.im/pybind/Lobby)
|
||||
[![Build Status](https://travis-ci.org/pybind/pybind11.svg?branch=master)](https://travis-ci.org/pybind/pybind11)
|
||||
[![Build status](https://ci.appveyor.com/api/projects/status/riaj54pn4h08xy40?svg=true)](https://ci.appveyor.com/project/wjakob/pybind11)
|
||||
|
||||
**pybind11** is a lightweight header-only library that exposes C++ types in Python
|
||||
and vice versa, mainly to create Python bindings of existing C++ code. Its
|
||||
goals and syntax are similar to the excellent
|
||||
[Boost.Python](http://www.boost.org/doc/libs/1_58_0/libs/python/doc/) library
|
||||
by David Abrahams: to minimize boilerplate code in traditional extension
|
||||
modules by inferring type information using compile-time introspection.
|
||||
|
||||
The main issue with Boost.Python—and the reason for creating such a similar
|
||||
project—is Boost. Boost is an enormously large and complex suite of utility
|
||||
libraries that works with almost every C++ compiler in existence. This
|
||||
compatibility has its cost: arcane template tricks and workarounds are
|
||||
necessary to support the oldest and buggiest of compiler specimens. Now that
|
||||
C++11-compatible compilers are widely available, this heavy machinery has
|
||||
become an excessively large and unnecessary dependency.
|
||||
|
||||
Think of this library as a tiny self-contained version of Boost.Python with
|
||||
everything stripped away that isn't relevant for binding generation. Without
|
||||
comments, the core header files only require ~4K lines of code and depend on
|
||||
Python (2.7 or 3.x, or PyPy2.7 >= 5.7) and the C++ standard library. This
|
||||
compact implementation was possible thanks to some of the new C++11 language
|
||||
features (specifically: tuples, lambda functions and variadic templates). Since
|
||||
its creation, this library has grown beyond Boost.Python in many ways, leading
|
||||
to dramatically simpler binding code in many common situations.
|
||||
|
||||
Tutorial and reference documentation is provided at
|
||||
[http://pybind11.readthedocs.org/en/master](http://pybind11.readthedocs.org/en/master).
|
||||
A PDF version of the manual is available
|
||||
[here](https://media.readthedocs.org/pdf/pybind11/master/pybind11.pdf).
|
||||
|
||||
## Core features
|
||||
pybind11 can map the following core C++ features to Python
|
||||
|
||||
- Functions accepting and returning custom data structures per value, reference, or pointer
|
||||
- Instance methods and static methods
|
||||
- Overloaded functions
|
||||
- Instance attributes and static attributes
|
||||
- Arbitrary exception types
|
||||
- Enumerations
|
||||
- Callbacks
|
||||
- Iterators and ranges
|
||||
- Custom operators
|
||||
- Single and multiple inheritance
|
||||
- STL data structures
|
||||
- Smart pointers with reference counting like ``std::shared_ptr``
|
||||
- Internal references with correct reference counting
|
||||
- C++ classes with virtual (and pure virtual) methods can be extended in Python
|
||||
|
||||
## Goodies
|
||||
In addition to the core functionality, pybind11 provides some extra goodies:
|
||||
|
||||
- Python 2.7, 3.x, and PyPy (PyPy2.7 >= 5.7) are supported with an
|
||||
implementation-agnostic interface.
|
||||
|
||||
- It is possible to bind C++11 lambda functions with captured variables. The
|
||||
lambda capture data is stored inside the resulting Python function object.
|
||||
|
||||
- pybind11 uses C++11 move constructors and move assignment operators whenever
|
||||
possible to efficiently transfer custom data types.
|
||||
|
||||
- It's easy to expose the internal storage of custom data types through
|
||||
Pythons' buffer protocols. This is handy e.g. for fast conversion between
|
||||
C++ matrix classes like Eigen and NumPy without expensive copy operations.
|
||||
|
||||
- pybind11 can automatically vectorize functions so that they are transparently
|
||||
applied to all entries of one or more NumPy array arguments.
|
||||
|
||||
- Python's slice-based access and assignment operations can be supported with
|
||||
just a few lines of code.
|
||||
|
||||
- Everything is contained in just a few header files; there is no need to link
|
||||
against any additional libraries.
|
||||
|
||||
- Binaries are generally smaller by a factor of at least 2 compared to
|
||||
equivalent bindings generated by Boost.Python. A recent pybind11 conversion
|
||||
of PyRosetta, an enormous Boost.Python binding project,
|
||||
[reported](http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf) a binary
|
||||
size reduction of **5.4x** and compile time reduction by **5.8x**.
|
||||
|
||||
- Function signatures are precomputed at compile time (using ``constexpr``),
|
||||
leading to smaller binaries.
|
||||
|
||||
- With little extra effort, C++ types can be pickled and unpickled similar to
|
||||
regular Python objects.
|
||||
|
||||
## Supported compilers
|
||||
|
||||
1. Clang/LLVM 3.3 or newer (for Apple Xcode's clang, this is 5.0.0 or newer)
|
||||
2. GCC 4.8 or newer
|
||||
3. Microsoft Visual Studio 2015 Update 3 or newer
|
||||
4. Intel C++ compiler 17 or newer (16 with pybind11 v2.0 and 15 with pybind11 v2.0 and a [workaround](https://github.com/pybind/pybind11/issues/276))
|
||||
5. Cygwin/GCC (tested on 2.5.1)
|
||||
|
||||
## About
|
||||
|
||||
This project was created by [Wenzel Jakob](http://rgl.epfl.ch/people/wjakob).
|
||||
Significant features and/or improvements to the code were contributed by
|
||||
Jonas Adler,
|
||||
Lori A. Burns,
|
||||
Sylvain Corlay,
|
||||
Trent Houliston,
|
||||
Axel Huebl,
|
||||
@hulucc,
|
||||
Sergey Lyskov
|
||||
Johan Mabille,
|
||||
Tomasz Miąsko,
|
||||
Dean Moldovan,
|
||||
Ben Pritchard,
|
||||
Jason Rhinelander,
|
||||
Boris Schäling,
|
||||
Pim Schellart,
|
||||
Henry Schreiner,
|
||||
Ivan Smirnov, and
|
||||
Patrick Stewart.
|
||||
|
||||
### License
|
||||
|
||||
pybind11 is provided under a BSD-style license that can be found in the
|
||||
``LICENSE`` file. By using, distributing, or contributing to this project,
|
||||
you agree to the terms and conditions of this license.
|
20
lib/bip158/lib/pybind11/docs/Doxyfile
Normal file
20
lib/bip158/lib/pybind11/docs/Doxyfile
Normal file
|
@ -0,0 +1,20 @@
|
|||
PROJECT_NAME = pybind11
|
||||
INPUT = ../include/pybind11/
|
||||
RECURSIVE = YES
|
||||
|
||||
GENERATE_HTML = NO
|
||||
GENERATE_LATEX = NO
|
||||
GENERATE_XML = YES
|
||||
XML_OUTPUT = .build/doxygenxml
|
||||
XML_PROGRAMLISTING = YES
|
||||
|
||||
MACRO_EXPANSION = YES
|
||||
EXPAND_ONLY_PREDEF = YES
|
||||
EXPAND_AS_DEFINED = PYBIND11_RUNTIME_EXCEPTION
|
||||
|
||||
ALIASES = "rst=\verbatim embed:rst"
|
||||
ALIASES += "endrst=\endverbatim"
|
||||
|
||||
QUIET = YES
|
||||
WARNINGS = YES
|
||||
WARN_IF_UNDOCUMENTED = NO
|
11
lib/bip158/lib/pybind11/docs/_static/theme_overrides.css
vendored
Normal file
11
lib/bip158/lib/pybind11/docs/_static/theme_overrides.css
vendored
Normal file
|
@ -0,0 +1,11 @@
|
|||
.wy-table-responsive table td,
|
||||
.wy-table-responsive table th {
|
||||
white-space: initial !important;
|
||||
}
|
||||
.rst-content table.docutils td {
|
||||
vertical-align: top !important;
|
||||
}
|
||||
div[class^='highlight'] pre {
|
||||
white-space: pre;
|
||||
white-space: pre-wrap;
|
||||
}
|
81
lib/bip158/lib/pybind11/docs/advanced/cast/chrono.rst
Normal file
81
lib/bip158/lib/pybind11/docs/advanced/cast/chrono.rst
Normal file
|
@ -0,0 +1,81 @@
|
|||
Chrono
|
||||
======
|
||||
|
||||
When including the additional header file :file:`pybind11/chrono.h` conversions
|
||||
from C++11 chrono datatypes to python datetime objects are automatically enabled.
|
||||
This header also enables conversions of python floats (often from sources such
|
||||
as ``time.monotonic()``, ``time.perf_counter()`` and ``time.process_time()``)
|
||||
into durations.
|
||||
|
||||
An overview of clocks in C++11
|
||||
------------------------------
|
||||
|
||||
A point of confusion when using these conversions is the differences between
|
||||
clocks provided in C++11. There are three clock types defined by the C++11
|
||||
standard and users can define their own if needed. Each of these clocks have
|
||||
different properties and when converting to and from python will give different
|
||||
results.
|
||||
|
||||
The first clock defined by the standard is ``std::chrono::system_clock``. This
|
||||
clock measures the current date and time. However, this clock changes with to
|
||||
updates to the operating system time. For example, if your time is synchronised
|
||||
with a time server this clock will change. This makes this clock a poor choice
|
||||
for timing purposes but good for measuring the wall time.
|
||||
|
||||
The second clock defined in the standard is ``std::chrono::steady_clock``.
|
||||
This clock ticks at a steady rate and is never adjusted. This makes it excellent
|
||||
for timing purposes, however the value in this clock does not correspond to the
|
||||
current date and time. Often this clock will be the amount of time your system
|
||||
has been on, although it does not have to be. This clock will never be the same
|
||||
clock as the system clock as the system clock can change but steady clocks
|
||||
cannot.
|
||||
|
||||
The third clock defined in the standard is ``std::chrono::high_resolution_clock``.
|
||||
This clock is the clock that has the highest resolution out of the clocks in the
|
||||
system. It is normally a typedef to either the system clock or the steady clock
|
||||
but can be its own independent clock. This is important as when using these
|
||||
conversions as the types you get in python for this clock might be different
|
||||
depending on the system.
|
||||
If it is a typedef of the system clock, python will get datetime objects, but if
|
||||
it is a different clock they will be timedelta objects.
|
||||
|
||||
Provided conversions
|
||||
--------------------
|
||||
|
||||
.. rubric:: C++ to Python
|
||||
|
||||
- ``std::chrono::system_clock::time_point`` → ``datetime.datetime``
|
||||
System clock times are converted to python datetime instances. They are
|
||||
in the local timezone, but do not have any timezone information attached
|
||||
to them (they are naive datetime objects).
|
||||
|
||||
- ``std::chrono::duration`` → ``datetime.timedelta``
|
||||
Durations are converted to timedeltas, any precision in the duration
|
||||
greater than microseconds is lost by rounding towards zero.
|
||||
|
||||
- ``std::chrono::[other_clocks]::time_point`` → ``datetime.timedelta``
|
||||
Any clock time that is not the system clock is converted to a time delta.
|
||||
This timedelta measures the time from the clocks epoch to now.
|
||||
|
||||
.. rubric:: Python to C++
|
||||
|
||||
- ``datetime.datetime`` or ``datetime.date`` or ``datetime.time`` → ``std::chrono::system_clock::time_point``
|
||||
Date/time objects are converted into system clock timepoints. Any
|
||||
timezone information is ignored and the type is treated as a naive
|
||||
object.
|
||||
|
||||
- ``datetime.timedelta`` → ``std::chrono::duration``
|
||||
Time delta are converted into durations with microsecond precision.
|
||||
|
||||
- ``datetime.timedelta`` → ``std::chrono::[other_clocks]::time_point``
|
||||
Time deltas that are converted into clock timepoints are treated as
|
||||
the amount of time from the start of the clocks epoch.
|
||||
|
||||
- ``float`` → ``std::chrono::duration``
|
||||
Floats that are passed to C++ as durations be interpreted as a number of
|
||||
seconds. These will be converted to the duration using ``duration_cast``
|
||||
from the float.
|
||||
|
||||
- ``float`` → ``std::chrono::[other_clocks]::time_point``
|
||||
Floats that are passed to C++ as time points will be interpreted as the
|
||||
number of seconds from the start of the clocks epoch.
|
91
lib/bip158/lib/pybind11/docs/advanced/cast/custom.rst
Normal file
91
lib/bip158/lib/pybind11/docs/advanced/cast/custom.rst
Normal file
|
@ -0,0 +1,91 @@
|
|||
Custom type casters
|
||||
===================
|
||||
|
||||
In very rare cases, applications may require custom type casters that cannot be
|
||||
expressed using the abstractions provided by pybind11, thus requiring raw
|
||||
Python C API calls. This is fairly advanced usage and should only be pursued by
|
||||
experts who are familiar with the intricacies of Python reference counting.
|
||||
|
||||
The following snippets demonstrate how this works for a very simple ``inty``
|
||||
type that that should be convertible from Python types that provide a
|
||||
``__int__(self)`` method.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct inty { long long_value; };
|
||||
|
||||
void print(inty s) {
|
||||
std::cout << s.long_value << std::endl;
|
||||
}
|
||||
|
||||
The following Python snippet demonstrates the intended usage from the Python side:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class A:
|
||||
def __int__(self):
|
||||
return 123
|
||||
|
||||
from example import print
|
||||
print(A())
|
||||
|
||||
To register the necessary conversion routines, it is necessary to add
|
||||
a partial overload to the ``pybind11::detail::type_caster<T>`` template.
|
||||
Although this is an implementation detail, adding partial overloads to this
|
||||
type is explicitly allowed.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
namespace pybind11 { namespace detail {
|
||||
template <> struct type_caster<inty> {
|
||||
public:
|
||||
/**
|
||||
* This macro establishes the name 'inty' in
|
||||
* function signatures and declares a local variable
|
||||
* 'value' of type inty
|
||||
*/
|
||||
PYBIND11_TYPE_CASTER(inty, _("inty"));
|
||||
|
||||
/**
|
||||
* Conversion part 1 (Python->C++): convert a PyObject into a inty
|
||||
* instance or return false upon failure. The second argument
|
||||
* indicates whether implicit conversions should be applied.
|
||||
*/
|
||||
bool load(handle src, bool) {
|
||||
/* Extract PyObject from handle */
|
||||
PyObject *source = src.ptr();
|
||||
/* Try converting into a Python integer value */
|
||||
PyObject *tmp = PyNumber_Long(source);
|
||||
if (!tmp)
|
||||
return false;
|
||||
/* Now try to convert into a C++ int */
|
||||
value.long_value = PyLong_AsLong(tmp);
|
||||
Py_DECREF(tmp);
|
||||
/* Ensure return code was OK (to avoid out-of-range errors etc) */
|
||||
return !(value.long_value == -1 && !PyErr_Occurred());
|
||||
}
|
||||
|
||||
/**
|
||||
* Conversion part 2 (C++ -> Python): convert an inty instance into
|
||||
* a Python object. The second and third arguments are used to
|
||||
* indicate the return value policy and parent object (for
|
||||
* ``return_value_policy::reference_internal``) and are generally
|
||||
* ignored by implicit casters.
|
||||
*/
|
||||
static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
|
||||
return PyLong_FromLong(src.long_value);
|
||||
}
|
||||
};
|
||||
}} // namespace pybind11::detail
|
||||
|
||||
.. note::
|
||||
|
||||
A ``type_caster<T>`` defined with ``PYBIND11_TYPE_CASTER(T, ...)`` requires
|
||||
that ``T`` is default-constructible (``value`` is first default constructed
|
||||
and then ``load()`` assigns to it).
|
||||
|
||||
.. warning::
|
||||
|
||||
When using custom type casters, it's important to declare them consistently
|
||||
in every compilation unit of the Python extension module. Otherwise,
|
||||
undefined behavior can ensue.
|
310
lib/bip158/lib/pybind11/docs/advanced/cast/eigen.rst
Normal file
310
lib/bip158/lib/pybind11/docs/advanced/cast/eigen.rst
Normal file
|
@ -0,0 +1,310 @@
|
|||
Eigen
|
||||
#####
|
||||
|
||||
`Eigen <http://eigen.tuxfamily.org>`_ is C++ header-based library for dense and
|
||||
sparse linear algebra. Due to its popularity and widespread adoption, pybind11
|
||||
provides transparent conversion and limited mapping support between Eigen and
|
||||
Scientific Python linear algebra data types.
|
||||
|
||||
To enable the built-in Eigen support you must include the optional header file
|
||||
:file:`pybind11/eigen.h`.
|
||||
|
||||
Pass-by-value
|
||||
=============
|
||||
|
||||
When binding a function with ordinary Eigen dense object arguments (for
|
||||
example, ``Eigen::MatrixXd``), pybind11 will accept any input value that is
|
||||
already (or convertible to) a ``numpy.ndarray`` with dimensions compatible with
|
||||
the Eigen type, copy its values into a temporary Eigen variable of the
|
||||
appropriate type, then call the function with this temporary variable.
|
||||
|
||||
Sparse matrices are similarly copied to or from
|
||||
``scipy.sparse.csr_matrix``/``scipy.sparse.csc_matrix`` objects.
|
||||
|
||||
Pass-by-reference
|
||||
=================
|
||||
|
||||
One major limitation of the above is that every data conversion implicitly
|
||||
involves a copy, which can be both expensive (for large matrices) and disallows
|
||||
binding functions that change their (Matrix) arguments. Pybind11 allows you to
|
||||
work around this by using Eigen's ``Eigen::Ref<MatrixType>`` class much as you
|
||||
would when writing a function taking a generic type in Eigen itself (subject to
|
||||
some limitations discussed below).
|
||||
|
||||
When calling a bound function accepting a ``Eigen::Ref<const MatrixType>``
|
||||
type, pybind11 will attempt to avoid copying by using an ``Eigen::Map`` object
|
||||
that maps into the source ``numpy.ndarray`` data: this requires both that the
|
||||
data types are the same (e.g. ``dtype='float64'`` and ``MatrixType::Scalar`` is
|
||||
``double``); and that the storage is layout compatible. The latter limitation
|
||||
is discussed in detail in the section below, and requires careful
|
||||
consideration: by default, numpy matrices and Eigen matrices are *not* storage
|
||||
compatible.
|
||||
|
||||
If the numpy matrix cannot be used as is (either because its types differ, e.g.
|
||||
passing an array of integers to an Eigen parameter requiring doubles, or
|
||||
because the storage is incompatible), pybind11 makes a temporary copy and
|
||||
passes the copy instead.
|
||||
|
||||
When a bound function parameter is instead ``Eigen::Ref<MatrixType>`` (note the
|
||||
lack of ``const``), pybind11 will only allow the function to be called if it
|
||||
can be mapped *and* if the numpy array is writeable (that is
|
||||
``a.flags.writeable`` is true). Any access (including modification) made to
|
||||
the passed variable will be transparently carried out directly on the
|
||||
``numpy.ndarray``.
|
||||
|
||||
This means you can can write code such as the following and have it work as
|
||||
expected:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void scale_by_2(Eigen::Ref<Eigen::VectorXd> v) {
|
||||
v *= 2;
|
||||
}
|
||||
|
||||
Note, however, that you will likely run into limitations due to numpy and
|
||||
Eigen's difference default storage order for data; see the below section on
|
||||
:ref:`storage_orders` for details on how to bind code that won't run into such
|
||||
limitations.
|
||||
|
||||
.. note::
|
||||
|
||||
Passing by reference is not supported for sparse types.
|
||||
|
||||
Returning values to Python
|
||||
==========================
|
||||
|
||||
When returning an ordinary dense Eigen matrix type to numpy (e.g.
|
||||
``Eigen::MatrixXd`` or ``Eigen::RowVectorXf``) pybind11 keeps the matrix and
|
||||
returns a numpy array that directly references the Eigen matrix: no copy of the
|
||||
data is performed. The numpy array will have ``array.flags.owndata`` set to
|
||||
``False`` to indicate that it does not own the data, and the lifetime of the
|
||||
stored Eigen matrix will be tied to the returned ``array``.
|
||||
|
||||
If you bind a function with a non-reference, ``const`` return type (e.g.
|
||||
``const Eigen::MatrixXd``), the same thing happens except that pybind11 also
|
||||
sets the numpy array's ``writeable`` flag to false.
|
||||
|
||||
If you return an lvalue reference or pointer, the usual pybind11 rules apply,
|
||||
as dictated by the binding function's return value policy (see the
|
||||
documentation on :ref:`return_value_policies` for full details). That means,
|
||||
without an explicit return value policy, lvalue references will be copied and
|
||||
pointers will be managed by pybind11. In order to avoid copying, you should
|
||||
explicitly specify an appropriate return value policy, as in the following
|
||||
example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class MyClass {
|
||||
Eigen::MatrixXd big_mat = Eigen::MatrixXd::Zero(10000, 10000);
|
||||
public:
|
||||
Eigen::MatrixXd &getMatrix() { return big_mat; }
|
||||
const Eigen::MatrixXd &viewMatrix() { return big_mat; }
|
||||
};
|
||||
|
||||
// Later, in binding code:
|
||||
py::class_<MyClass>(m, "MyClass")
|
||||
.def(py::init<>())
|
||||
.def("copy_matrix", &MyClass::getMatrix) // Makes a copy!
|
||||
.def("get_matrix", &MyClass::getMatrix, py::return_value_policy::reference_internal)
|
||||
.def("view_matrix", &MyClass::viewMatrix, py::return_value_policy::reference_internal)
|
||||
;
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
a = MyClass()
|
||||
m = a.get_matrix() # flags.writeable = True, flags.owndata = False
|
||||
v = a.view_matrix() # flags.writeable = False, flags.owndata = False
|
||||
c = a.copy_matrix() # flags.writeable = True, flags.owndata = True
|
||||
# m[5,6] and v[5,6] refer to the same element, c[5,6] does not.
|
||||
|
||||
Note in this example that ``py::return_value_policy::reference_internal`` is
|
||||
used to tie the life of the MyClass object to the life of the returned arrays.
|
||||
|
||||
You may also return an ``Eigen::Ref``, ``Eigen::Map`` or other map-like Eigen
|
||||
object (for example, the return value of ``matrix.block()`` and related
|
||||
methods) that map into a dense Eigen type. When doing so, the default
|
||||
behaviour of pybind11 is to simply reference the returned data: you must take
|
||||
care to ensure that this data remains valid! You may ask pybind11 to
|
||||
explicitly *copy* such a return value by using the
|
||||
``py::return_value_policy::copy`` policy when binding the function. You may
|
||||
also use ``py::return_value_policy::reference_internal`` or a
|
||||
``py::keep_alive`` to ensure the data stays valid as long as the returned numpy
|
||||
array does.
|
||||
|
||||
When returning such a reference of map, pybind11 additionally respects the
|
||||
readonly-status of the returned value, marking the numpy array as non-writeable
|
||||
if the reference or map was itself read-only.
|
||||
|
||||
.. note::
|
||||
|
||||
Sparse types are always copied when returned.
|
||||
|
||||
.. _storage_orders:
|
||||
|
||||
Storage orders
|
||||
==============
|
||||
|
||||
Passing arguments via ``Eigen::Ref`` has some limitations that you must be
|
||||
aware of in order to effectively pass matrices by reference. First and
|
||||
foremost is that the default ``Eigen::Ref<MatrixType>`` class requires
|
||||
contiguous storage along columns (for column-major types, the default in Eigen)
|
||||
or rows if ``MatrixType`` is specifically an ``Eigen::RowMajor`` storage type.
|
||||
The former, Eigen's default, is incompatible with ``numpy``'s default row-major
|
||||
storage, and so you will not be able to pass numpy arrays to Eigen by reference
|
||||
without making one of two changes.
|
||||
|
||||
(Note that this does not apply to vectors (or column or row matrices): for such
|
||||
types the "row-major" and "column-major" distinction is meaningless).
|
||||
|
||||
The first approach is to change the use of ``Eigen::Ref<MatrixType>`` to the
|
||||
more general ``Eigen::Ref<MatrixType, 0, Eigen::Stride<Eigen::Dynamic,
|
||||
Eigen::Dynamic>>`` (or similar type with a fully dynamic stride type in the
|
||||
third template argument). Since this is a rather cumbersome type, pybind11
|
||||
provides a ``py::EigenDRef<MatrixType>`` type alias for your convenience (along
|
||||
with EigenDMap for the equivalent Map, and EigenDStride for just the stride
|
||||
type).
|
||||
|
||||
This type allows Eigen to map into any arbitrary storage order. This is not
|
||||
the default in Eigen for performance reasons: contiguous storage allows
|
||||
vectorization that cannot be done when storage is not known to be contiguous at
|
||||
compile time. The default ``Eigen::Ref`` stride type allows non-contiguous
|
||||
storage along the outer dimension (that is, the rows of a column-major matrix
|
||||
or columns of a row-major matrix), but not along the inner dimension.
|
||||
|
||||
This type, however, has the added benefit of also being able to map numpy array
|
||||
slices. For example, the following (contrived) example uses Eigen with a numpy
|
||||
slice to multiply by 2 all coefficients that are both on even rows (0, 2, 4,
|
||||
...) and in columns 2, 5, or 8:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("scale", [](py::EigenDRef<Eigen::MatrixXd> m, double c) { m *= c; });
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# a = np.array(...)
|
||||
scale_by_2(myarray[0::2, 2:9:3])
|
||||
|
||||
The second approach to avoid copying is more intrusive: rearranging the
|
||||
underlying data types to not run into the non-contiguous storage problem in the
|
||||
first place. In particular, that means using matrices with ``Eigen::RowMajor``
|
||||
storage, where appropriate, such as:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
using RowMatrixXd = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
|
||||
// Use RowMatrixXd instead of MatrixXd
|
||||
|
||||
Now bound functions accepting ``Eigen::Ref<RowMatrixXd>`` arguments will be
|
||||
callable with numpy's (default) arrays without involving a copying.
|
||||
|
||||
You can, alternatively, change the storage order that numpy arrays use by
|
||||
adding the ``order='F'`` option when creating an array:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
myarray = np.array(source, order='F')
|
||||
|
||||
Such an object will be passable to a bound function accepting an
|
||||
``Eigen::Ref<MatrixXd>`` (or similar column-major Eigen type).
|
||||
|
||||
One major caveat with this approach, however, is that it is not entirely as
|
||||
easy as simply flipping all Eigen or numpy usage from one to the other: some
|
||||
operations may alter the storage order of a numpy array. For example, ``a2 =
|
||||
array.transpose()`` results in ``a2`` being a view of ``array`` that references
|
||||
the same data, but in the opposite storage order!
|
||||
|
||||
While this approach allows fully optimized vectorized calculations in Eigen, it
|
||||
cannot be used with array slices, unlike the first approach.
|
||||
|
||||
When *returning* a matrix to Python (either a regular matrix, a reference via
|
||||
``Eigen::Ref<>``, or a map/block into a matrix), no special storage
|
||||
consideration is required: the created numpy array will have the required
|
||||
stride that allows numpy to properly interpret the array, whatever its storage
|
||||
order.
|
||||
|
||||
Failing rather than copying
|
||||
===========================
|
||||
|
||||
The default behaviour when binding ``Eigen::Ref<const MatrixType>`` Eigen
|
||||
references is to copy matrix values when passed a numpy array that does not
|
||||
conform to the element type of ``MatrixType`` or does not have a compatible
|
||||
stride layout. If you want to explicitly avoid copying in such a case, you
|
||||
should bind arguments using the ``py::arg().noconvert()`` annotation (as
|
||||
described in the :ref:`nonconverting_arguments` documentation).
|
||||
|
||||
The following example shows an example of arguments that don't allow data
|
||||
copying to take place:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// The method and function to be bound:
|
||||
class MyClass {
|
||||
// ...
|
||||
double some_method(const Eigen::Ref<const MatrixXd> &matrix) { /* ... */ }
|
||||
};
|
||||
float some_function(const Eigen::Ref<const MatrixXf> &big,
|
||||
const Eigen::Ref<const MatrixXf> &small) {
|
||||
// ...
|
||||
}
|
||||
|
||||
// The associated binding code:
|
||||
using namespace pybind11::literals; // for "arg"_a
|
||||
py::class_<MyClass>(m, "MyClass")
|
||||
// ... other class definitions
|
||||
.def("some_method", &MyClass::some_method, py::arg().noconvert());
|
||||
|
||||
m.def("some_function", &some_function,
|
||||
"big"_a.noconvert(), // <- Don't allow copying for this arg
|
||||
"small"_a // <- This one can be copied if needed
|
||||
);
|
||||
|
||||
With the above binding code, attempting to call the the ``some_method(m)``
|
||||
method on a ``MyClass`` object, or attempting to call ``some_function(m, m2)``
|
||||
will raise a ``RuntimeError`` rather than making a temporary copy of the array.
|
||||
It will, however, allow the ``m2`` argument to be copied into a temporary if
|
||||
necessary.
|
||||
|
||||
Note that explicitly specifying ``.noconvert()`` is not required for *mutable*
|
||||
Eigen references (e.g. ``Eigen::Ref<MatrixXd>`` without ``const`` on the
|
||||
``MatrixXd``): mutable references will never be called with a temporary copy.
|
||||
|
||||
Vectors versus column/row matrices
|
||||
==================================
|
||||
|
||||
Eigen and numpy have fundamentally different notions of a vector. In Eigen, a
|
||||
vector is simply a matrix with the number of columns or rows set to 1 at
|
||||
compile time (for a column vector or row vector, respectively). Numpy, in
|
||||
contrast, has comparable 2-dimensional 1xN and Nx1 arrays, but *also* has
|
||||
1-dimensional arrays of size N.
|
||||
|
||||
When passing a 2-dimensional 1xN or Nx1 array to Eigen, the Eigen type must
|
||||
have matching dimensions: That is, you cannot pass a 2-dimensional Nx1 numpy
|
||||
array to an Eigen value expecting a row vector, or a 1xN numpy array as a
|
||||
column vector argument.
|
||||
|
||||
On the other hand, pybind11 allows you to pass 1-dimensional arrays of length N
|
||||
as Eigen parameters. If the Eigen type can hold a column vector of length N it
|
||||
will be passed as such a column vector. If not, but the Eigen type constraints
|
||||
will accept a row vector, it will be passed as a row vector. (The column
|
||||
vector takes precedence when both are supported, for example, when passing a
|
||||
1D numpy array to a MatrixXd argument). Note that the type need not be
|
||||
explicitly a vector: it is permitted to pass a 1D numpy array of size 5 to an
|
||||
Eigen ``Matrix<double, Dynamic, 5>``: you would end up with a 1x5 Eigen matrix.
|
||||
Passing the same to an ``Eigen::MatrixXd`` would result in a 5x1 Eigen matrix.
|
||||
|
||||
When returning an Eigen vector to numpy, the conversion is ambiguous: a row
|
||||
vector of length 4 could be returned as either a 1D array of length 4, or as a
|
||||
2D array of size 1x4. When encountering such a situation, pybind11 compromises
|
||||
by considering the returned Eigen type: if it is a compile-time vector--that
|
||||
is, the type has either the number of rows or columns set to 1 at compile
|
||||
time--pybind11 converts to a 1D numpy array when returning the value. For
|
||||
instances that are a vector only at run-time (e.g. ``MatrixXd``,
|
||||
``Matrix<float, Dynamic, 4>``), pybind11 returns the vector as a 2D array to
|
||||
numpy. If this isn't want you want, you can use ``array.reshape(...)`` to get
|
||||
a view of the same data in the desired dimensions.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_eigen.cpp` contains a complete example that
|
||||
shows how to pass Eigen sparse and dense data types in more detail.
|
109
lib/bip158/lib/pybind11/docs/advanced/cast/functional.rst
Normal file
109
lib/bip158/lib/pybind11/docs/advanced/cast/functional.rst
Normal file
|
@ -0,0 +1,109 @@
|
|||
Functional
|
||||
##########
|
||||
|
||||
The following features must be enabled by including :file:`pybind11/functional.h`.
|
||||
|
||||
|
||||
Callbacks and passing anonymous functions
|
||||
=========================================
|
||||
|
||||
The C++11 standard brought lambda functions and the generic polymorphic
|
||||
function wrapper ``std::function<>`` to the C++ programming language, which
|
||||
enable powerful new ways of working with functions. Lambda functions come in
|
||||
two flavors: stateless lambda function resemble classic function pointers that
|
||||
link to an anonymous piece of code, while stateful lambda functions
|
||||
additionally depend on captured variables that are stored in an anonymous
|
||||
*lambda closure object*.
|
||||
|
||||
Here is a simple example of a C++ function that takes an arbitrary function
|
||||
(stateful or stateless) with signature ``int -> int`` as an argument and runs
|
||||
it with the value 10.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
int func_arg(const std::function<int(int)> &f) {
|
||||
return f(10);
|
||||
}
|
||||
|
||||
The example below is more involved: it takes a function of signature ``int -> int``
|
||||
and returns another function of the same kind. The return value is a stateful
|
||||
lambda function, which stores the value ``f`` in the capture object and adds 1 to
|
||||
its return value upon execution.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
std::function<int(int)> func_ret(const std::function<int(int)> &f) {
|
||||
return [f](int i) {
|
||||
return f(i) + 1;
|
||||
};
|
||||
}
|
||||
|
||||
This example demonstrates using python named parameters in C++ callbacks which
|
||||
requires using ``py::cpp_function`` as a wrapper. Usage is similar to defining
|
||||
methods of classes:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::cpp_function func_cpp() {
|
||||
return py::cpp_function([](int i) { return i+1; },
|
||||
py::arg("number"));
|
||||
}
|
||||
|
||||
After including the extra header file :file:`pybind11/functional.h`, it is almost
|
||||
trivial to generate binding code for all of these functions.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/functional.h>
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
m.def("func_arg", &func_arg);
|
||||
m.def("func_ret", &func_ret);
|
||||
m.def("func_cpp", &func_cpp);
|
||||
}
|
||||
|
||||
The following interactive session shows how to call them from Python.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
$ python
|
||||
>>> import example
|
||||
>>> def square(i):
|
||||
... return i * i
|
||||
...
|
||||
>>> example.func_arg(square)
|
||||
100L
|
||||
>>> square_plus_1 = example.func_ret(square)
|
||||
>>> square_plus_1(4)
|
||||
17L
|
||||
>>> plus_1 = func_cpp()
|
||||
>>> plus_1(number=43)
|
||||
44L
|
||||
|
||||
.. warning::
|
||||
|
||||
Keep in mind that passing a function from C++ to Python (or vice versa)
|
||||
will instantiate a piece of wrapper code that translates function
|
||||
invocations between the two languages. Naturally, this translation
|
||||
increases the computational cost of each function call somewhat. A
|
||||
problematic situation can arise when a function is copied back and forth
|
||||
between Python and C++ many times in a row, in which case the underlying
|
||||
wrappers will accumulate correspondingly. The resulting long sequence of
|
||||
C++ -> Python -> C++ -> ... roundtrips can significantly decrease
|
||||
performance.
|
||||
|
||||
There is one exception: pybind11 detects case where a stateless function
|
||||
(i.e. a function pointer or a lambda function without captured variables)
|
||||
is passed as an argument to another C++ function exposed in Python. In this
|
||||
case, there is no overhead. Pybind11 will extract the underlying C++
|
||||
function pointer from the wrapped function to sidestep a potential C++ ->
|
||||
Python -> C++ roundtrip. This is demonstrated in :file:`tests/test_callbacks.cpp`.
|
||||
|
||||
.. note::
|
||||
|
||||
This functionality is very useful when generating bindings for callbacks in
|
||||
C++ libraries (e.g. GUI libraries, asynchronous networking libraries, etc.).
|
||||
|
||||
The file :file:`tests/test_callbacks.cpp` contains a complete example
|
||||
that demonstrates how to work with callbacks and anonymous functions in
|
||||
more detail.
|
42
lib/bip158/lib/pybind11/docs/advanced/cast/index.rst
Normal file
42
lib/bip158/lib/pybind11/docs/advanced/cast/index.rst
Normal file
|
@ -0,0 +1,42 @@
|
|||
Type conversions
|
||||
################
|
||||
|
||||
Apart from enabling cross-language function calls, a fundamental problem
|
||||
that a binding tool like pybind11 must address is to provide access to
|
||||
native Python types in C++ and vice versa. There are three fundamentally
|
||||
different ways to do this—which approach is preferable for a particular type
|
||||
depends on the situation at hand.
|
||||
|
||||
1. Use a native C++ type everywhere. In this case, the type must be wrapped
|
||||
using pybind11-generated bindings so that Python can interact with it.
|
||||
|
||||
2. Use a native Python type everywhere. It will need to be wrapped so that
|
||||
C++ functions can interact with it.
|
||||
|
||||
3. Use a native C++ type on the C++ side and a native Python type on the
|
||||
Python side. pybind11 refers to this as a *type conversion*.
|
||||
|
||||
Type conversions are the most "natural" option in the sense that native
|
||||
(non-wrapped) types are used everywhere. The main downside is that a copy
|
||||
of the data must be made on every Python ↔ C++ transition: this is
|
||||
needed since the C++ and Python versions of the same type generally won't
|
||||
have the same memory layout.
|
||||
|
||||
pybind11 can perform many kinds of conversions automatically. An overview
|
||||
is provided in the table ":ref:`conversion_table`".
|
||||
|
||||
The following subsections discuss the differences between these options in more
|
||||
detail. The main focus in this section is on type conversions, which represent
|
||||
the last case of the above list.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
overview
|
||||
strings
|
||||
stl
|
||||
functional
|
||||
chrono
|
||||
eigen
|
||||
custom
|
||||
|
165
lib/bip158/lib/pybind11/docs/advanced/cast/overview.rst
Normal file
165
lib/bip158/lib/pybind11/docs/advanced/cast/overview.rst
Normal file
|
@ -0,0 +1,165 @@
|
|||
Overview
|
||||
########
|
||||
|
||||
.. rubric:: 1. Native type in C++, wrapper in Python
|
||||
|
||||
Exposing a custom C++ type using :class:`py::class_` was covered in detail
|
||||
in the :doc:`/classes` section. There, the underlying data structure is
|
||||
always the original C++ class while the :class:`py::class_` wrapper provides
|
||||
a Python interface. Internally, when an object like this is sent from C++ to
|
||||
Python, pybind11 will just add the outer wrapper layer over the native C++
|
||||
object. Getting it back from Python is just a matter of peeling off the
|
||||
wrapper.
|
||||
|
||||
.. rubric:: 2. Wrapper in C++, native type in Python
|
||||
|
||||
This is the exact opposite situation. Now, we have a type which is native to
|
||||
Python, like a ``tuple`` or a ``list``. One way to get this data into C++ is
|
||||
with the :class:`py::object` family of wrappers. These are explained in more
|
||||
detail in the :doc:`/advanced/pycpp/object` section. We'll just give a quick
|
||||
example here:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void print_list(py::list my_list) {
|
||||
for (auto item : my_list)
|
||||
std::cout << item << " ";
|
||||
}
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print_list([1, 2, 3])
|
||||
1 2 3
|
||||
|
||||
The Python ``list`` is not converted in any way -- it's just wrapped in a C++
|
||||
:class:`py::list` class. At its core it's still a Python object. Copying a
|
||||
:class:`py::list` will do the usual reference-counting like in Python.
|
||||
Returning the object to Python will just remove the thin wrapper.
|
||||
|
||||
.. rubric:: 3. Converting between native C++ and Python types
|
||||
|
||||
In the previous two cases we had a native type in one language and a wrapper in
|
||||
the other. Now, we have native types on both sides and we convert between them.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void print_vector(const std::vector<int> &v) {
|
||||
for (auto item : v)
|
||||
std::cout << item << "\n";
|
||||
}
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print_vector([1, 2, 3])
|
||||
1 2 3
|
||||
|
||||
In this case, pybind11 will construct a new ``std::vector<int>`` and copy each
|
||||
element from the Python ``list``. The newly constructed object will be passed
|
||||
to ``print_vector``. The same thing happens in the other direction: a new
|
||||
``list`` is made to match the value returned from C++.
|
||||
|
||||
Lots of these conversions are supported out of the box, as shown in the table
|
||||
below. They are very convenient, but keep in mind that these conversions are
|
||||
fundamentally based on copying data. This is perfectly fine for small immutable
|
||||
types but it may become quite expensive for large data structures. This can be
|
||||
avoided by overriding the automatic conversion with a custom wrapper (i.e. the
|
||||
above-mentioned approach 1). This requires some manual effort and more details
|
||||
are available in the :ref:`opaque` section.
|
||||
|
||||
.. _conversion_table:
|
||||
|
||||
List of all builtin conversions
|
||||
-------------------------------
|
||||
|
||||
The following basic data types are supported out of the box (some may require
|
||||
an additional extension header to be included). To pass other data structures
|
||||
as arguments and return values, refer to the section on binding :ref:`classes`.
|
||||
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| Data type | Description | Header file |
|
||||
+====================================+===========================+===============================+
|
||||
| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``char`` | Character literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``char16_t`` | UTF-16 character literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``char32_t`` | UTF-32 character literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``const char16_t *`` | UTF-16 string literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``const char32_t *`` | UTF-32 string literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::u16string`` | STL dynamic UTF-16 string | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::u32string`` | STL dynamic UTF-32 string | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::string_view``, | STL C++17 string views | :file:`pybind11/pybind11.h` |
|
||||
| ``std::u16string_view``, etc. | | |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::deque<T>`` | STL double-ended queue | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::valarray<T>`` | STL value array | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::optional<T>`` | STL optional type (C++17) | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::experimental::optional<T>`` | STL optional type (exp.) | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``std::chrono::time_point<...>`` | STL date/time | :file:`pybind11/chrono.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
||||
| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
|
||||
+------------------------------------+---------------------------+-------------------------------+
|
240
lib/bip158/lib/pybind11/docs/advanced/cast/stl.rst
Normal file
240
lib/bip158/lib/pybind11/docs/advanced/cast/stl.rst
Normal file
|
@ -0,0 +1,240 @@
|
|||
STL containers
|
||||
##############
|
||||
|
||||
Automatic conversion
|
||||
====================
|
||||
|
||||
When including the additional header file :file:`pybind11/stl.h`, conversions
|
||||
between ``std::vector<>``/``std::deque<>``/``std::list<>``/``std::array<>``,
|
||||
``std::set<>``/``std::unordered_set<>``, and
|
||||
``std::map<>``/``std::unordered_map<>`` and the Python ``list``, ``set`` and
|
||||
``dict`` data structures are automatically enabled. The types ``std::pair<>``
|
||||
and ``std::tuple<>`` are already supported out of the box with just the core
|
||||
:file:`pybind11/pybind11.h` header.
|
||||
|
||||
The major downside of these implicit conversions is that containers must be
|
||||
converted (i.e. copied) on every Python->C++ and C++->Python transition, which
|
||||
can have implications on the program semantics and performance. Please read the
|
||||
next sections for more details and alternative approaches that avoid this.
|
||||
|
||||
.. note::
|
||||
|
||||
Arbitrary nesting of any of these types is possible.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_stl.cpp` contains a complete
|
||||
example that demonstrates how to pass STL data types in more detail.
|
||||
|
||||
.. _cpp17_container_casters:
|
||||
|
||||
C++17 library containers
|
||||
========================
|
||||
|
||||
The :file:`pybind11/stl.h` header also includes support for ``std::optional<>``
|
||||
and ``std::variant<>``. These require a C++17 compiler and standard library.
|
||||
In C++14 mode, ``std::experimental::optional<>`` is supported if available.
|
||||
|
||||
Various versions of these containers also exist for C++11 (e.g. in Boost).
|
||||
pybind11 provides an easy way to specialize the ``type_caster`` for such
|
||||
types:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// `boost::optional` as an example -- can be any `std::optional`-like container
|
||||
namespace pybind11 { namespace detail {
|
||||
template <typename T>
|
||||
struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {};
|
||||
}}
|
||||
|
||||
The above should be placed in a header file and included in all translation units
|
||||
where automatic conversion is needed. Similarly, a specialization can be provided
|
||||
for custom variant types:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// `boost::variant` as an example -- can be any `std::variant`-like container
|
||||
namespace pybind11 { namespace detail {
|
||||
template <typename... Ts>
|
||||
struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {};
|
||||
|
||||
// Specifies the function used to visit the variant -- `apply_visitor` instead of `visit`
|
||||
template <>
|
||||
struct visit_helper<boost::variant> {
|
||||
template <typename... Args>
|
||||
static auto call(Args &&...args) -> decltype(boost::apply_visitor(args...)) {
|
||||
return boost::apply_visitor(args...);
|
||||
}
|
||||
};
|
||||
}} // namespace pybind11::detail
|
||||
|
||||
The ``visit_helper`` specialization is not required if your ``name::variant`` provides
|
||||
a ``name::visit()`` function. For any other function name, the specialization must be
|
||||
included to tell pybind11 how to visit the variant.
|
||||
|
||||
.. note::
|
||||
|
||||
pybind11 only supports the modern implementation of ``boost::variant``
|
||||
which makes use of variadic templates. This requires Boost 1.56 or newer.
|
||||
Additionally, on Windows, MSVC 2017 is required because ``boost::variant``
|
||||
falls back to the old non-variadic implementation on MSVC 2015.
|
||||
|
||||
.. _opaque:
|
||||
|
||||
Making opaque types
|
||||
===================
|
||||
|
||||
pybind11 heavily relies on a template matching mechanism to convert parameters
|
||||
and return values that are constructed from STL data types such as vectors,
|
||||
linked lists, hash tables, etc. This even works in a recursive manner, for
|
||||
instance to deal with lists of hash maps of pairs of elementary and custom
|
||||
types, etc.
|
||||
|
||||
However, a fundamental limitation of this approach is that internal conversions
|
||||
between Python and C++ types involve a copy operation that prevents
|
||||
pass-by-reference semantics. What does this mean?
|
||||
|
||||
Suppose we bind the following function
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void append_1(std::vector<int> &v) {
|
||||
v.push_back(1);
|
||||
}
|
||||
|
||||
and call it from Python, the following happens:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> v = [5, 6]
|
||||
>>> append_1(v)
|
||||
>>> print(v)
|
||||
[5, 6]
|
||||
|
||||
As you can see, when passing STL data structures by reference, modifications
|
||||
are not propagated back the Python side. A similar situation arises when
|
||||
exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
|
||||
functions:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
/* ... definition ... */
|
||||
|
||||
class MyClass {
|
||||
std::vector<int> contents;
|
||||
};
|
||||
|
||||
/* ... binding code ... */
|
||||
|
||||
py::class_<MyClass>(m, "MyClass")
|
||||
.def(py::init<>())
|
||||
.def_readwrite("contents", &MyClass::contents);
|
||||
|
||||
In this case, properties can be read and written in their entirety. However, an
|
||||
``append`` operation involving such a list type has no effect:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> m = MyClass()
|
||||
>>> m.contents = [5, 6]
|
||||
>>> print(m.contents)
|
||||
[5, 6]
|
||||
>>> m.contents.append(7)
|
||||
>>> print(m.contents)
|
||||
[5, 6]
|
||||
|
||||
Finally, the involved copy operations can be costly when dealing with very
|
||||
large lists. To deal with all of the above situations, pybind11 provides a
|
||||
macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
|
||||
conversion machinery of types, thus rendering them *opaque*. The contents of
|
||||
opaque objects are never inspected or extracted, hence they *can* be passed by
|
||||
reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
|
||||
the declaration
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_MAKE_OPAQUE(std::vector<int>);
|
||||
|
||||
before any binding code (e.g. invocations to ``class_::def()``, etc.). This
|
||||
macro must be specified at the top level (and outside of any namespaces), since
|
||||
it instantiates a partial template overload. If your binding code consists of
|
||||
multiple compilation units, it must be present in every file (typically via a
|
||||
common header) preceding any usage of ``std::vector<int>``. Opaque types must
|
||||
also have a corresponding ``class_`` declaration to associate them with a name
|
||||
in Python, and to define a set of available operations, e.g.:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<std::vector<int>>(m, "IntVector")
|
||||
.def(py::init<>())
|
||||
.def("clear", &std::vector<int>::clear)
|
||||
.def("pop_back", &std::vector<int>::pop_back)
|
||||
.def("__len__", [](const std::vector<int> &v) { return v.size(); })
|
||||
.def("__iter__", [](std::vector<int> &v) {
|
||||
return py::make_iterator(v.begin(), v.end());
|
||||
}, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
|
||||
// ....
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_opaque_types.cpp` contains a complete
|
||||
example that demonstrates how to create and expose opaque types using
|
||||
pybind11 in more detail.
|
||||
|
||||
.. _stl_bind:
|
||||
|
||||
Binding STL containers
|
||||
======================
|
||||
|
||||
The ability to expose STL containers as native Python objects is a fairly
|
||||
common request, hence pybind11 also provides an optional header file named
|
||||
:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
|
||||
to match the behavior of their native Python counterparts as much as possible.
|
||||
|
||||
The following example showcases usage of :file:`pybind11/stl_bind.h`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Don't forget this
|
||||
#include <pybind11/stl_bind.h>
|
||||
|
||||
PYBIND11_MAKE_OPAQUE(std::vector<int>);
|
||||
PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
|
||||
|
||||
// ...
|
||||
|
||||
// later in binding code:
|
||||
py::bind_vector<std::vector<int>>(m, "VectorInt");
|
||||
py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
|
||||
|
||||
When binding STL containers pybind11 considers the types of the container's
|
||||
elements to decide whether the container should be confined to the local module
|
||||
(via the :ref:`module_local` feature). If the container element types are
|
||||
anything other than already-bound custom types bound without
|
||||
``py::module_local()`` the container binding will have ``py::module_local()``
|
||||
applied. This includes converting types such as numeric types, strings, Eigen
|
||||
types; and types that have not yet been bound at the time of the stl container
|
||||
binding. This module-local binding is designed to avoid potential conflicts
|
||||
between module bindings (for example, from two separate modules each attempting
|
||||
to bind ``std::vector<int>`` as a python type).
|
||||
|
||||
It is possible to override this behavior to force a definition to be either
|
||||
module-local or global. To do so, you can pass the attributes
|
||||
``py::module_local()`` (to make the binding module-local) or
|
||||
``py::module_local(false)`` (to make the binding global) into the
|
||||
``py::bind_vector`` or ``py::bind_map`` arguments:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
|
||||
|
||||
Note, however, that such a global binding would make it impossible to load this
|
||||
module at the same time as any other pybind module that also attempts to bind
|
||||
the same container type (``std::vector<int>`` in the above example).
|
||||
|
||||
See :ref:`module_local` for more details on module-local bindings.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_stl_binders.cpp` shows how to use the
|
||||
convenience STL container wrappers.
|
305
lib/bip158/lib/pybind11/docs/advanced/cast/strings.rst
Normal file
305
lib/bip158/lib/pybind11/docs/advanced/cast/strings.rst
Normal file
|
@ -0,0 +1,305 @@
|
|||
Strings, bytes and Unicode conversions
|
||||
######################################
|
||||
|
||||
.. note::
|
||||
|
||||
This section discusses string handling in terms of Python 3 strings. For
|
||||
Python 2.7, replace all occurrences of ``str`` with ``unicode`` and
|
||||
``bytes`` with ``str``. Python 2.7 users may find it best to use ``from
|
||||
__future__ import unicode_literals`` to avoid unintentionally using ``str``
|
||||
instead of ``unicode``.
|
||||
|
||||
Passing Python strings to C++
|
||||
=============================
|
||||
|
||||
When a Python ``str`` is passed from Python to a C++ function that accepts
|
||||
``std::string`` or ``char *`` as arguments, pybind11 will encode the Python
|
||||
string to UTF-8. All Python ``str`` can be encoded in UTF-8, so this operation
|
||||
does not fail.
|
||||
|
||||
The C++ language is encoding agnostic. It is the responsibility of the
|
||||
programmer to track encodings. It's often easiest to simply `use UTF-8
|
||||
everywhere <http://utf8everywhere.org/>`_.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
m.def("utf8_test",
|
||||
[](const std::string &s) {
|
||||
cout << "utf-8 is icing on the cake.\n";
|
||||
cout << s;
|
||||
}
|
||||
);
|
||||
m.def("utf8_charptr",
|
||||
[](const char *s) {
|
||||
cout << "My favorite food is\n";
|
||||
cout << s;
|
||||
}
|
||||
);
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> utf8_test('🎂')
|
||||
utf-8 is icing on the cake.
|
||||
🎂
|
||||
|
||||
>>> utf8_charptr('🍕')
|
||||
My favorite food is
|
||||
🍕
|
||||
|
||||
.. note::
|
||||
|
||||
Some terminal emulators do not support UTF-8 or emoji fonts and may not
|
||||
display the example above correctly.
|
||||
|
||||
The results are the same whether the C++ function accepts arguments by value or
|
||||
reference, and whether or not ``const`` is used.
|
||||
|
||||
Passing bytes to C++
|
||||
--------------------
|
||||
|
||||
A Python ``bytes`` object will be passed to C++ functions that accept
|
||||
``std::string`` or ``char*`` *without* conversion. On Python 3, in order to
|
||||
make a function *only* accept ``bytes`` (and not ``str``), declare it as taking
|
||||
a ``py::bytes`` argument.
|
||||
|
||||
|
||||
Returning C++ strings to Python
|
||||
===============================
|
||||
|
||||
When a C++ function returns a ``std::string`` or ``char*`` to a Python caller,
|
||||
**pybind11 will assume that the string is valid UTF-8** and will decode it to a
|
||||
native Python ``str``, using the same API as Python uses to perform
|
||||
``bytes.decode('utf-8')``. If this implicit conversion fails, pybind11 will
|
||||
raise a ``UnicodeDecodeError``.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
m.def("std_string_return",
|
||||
[]() {
|
||||
return std::string("This string needs to be UTF-8 encoded");
|
||||
}
|
||||
);
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> isinstance(example.std_string_return(), str)
|
||||
True
|
||||
|
||||
|
||||
Because UTF-8 is inclusive of pure ASCII, there is never any issue with
|
||||
returning a pure ASCII string to Python. If there is any possibility that the
|
||||
string is not pure ASCII, it is necessary to ensure the encoding is valid
|
||||
UTF-8.
|
||||
|
||||
.. warning::
|
||||
|
||||
Implicit conversion assumes that a returned ``char *`` is null-terminated.
|
||||
If there is no null terminator a buffer overrun will occur.
|
||||
|
||||
Explicit conversions
|
||||
--------------------
|
||||
|
||||
If some C++ code constructs a ``std::string`` that is not a UTF-8 string, one
|
||||
can perform a explicit conversion and return a ``py::str`` object. Explicit
|
||||
conversion has the same overhead as implicit conversion.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
// This uses the Python C API to convert Latin-1 to Unicode
|
||||
m.def("str_output",
|
||||
[]() {
|
||||
std::string s = "Send your r\xe9sum\xe9 to Alice in HR"; // Latin-1
|
||||
py::str py_s = PyUnicode_DecodeLatin1(s.data(), s.length());
|
||||
return py_s;
|
||||
}
|
||||
);
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> str_output()
|
||||
'Send your résumé to Alice in HR'
|
||||
|
||||
The `Python C API
|
||||
<https://docs.python.org/3/c-api/unicode.html#built-in-codecs>`_ provides
|
||||
several built-in codecs.
|
||||
|
||||
|
||||
One could also use a third party encoding library such as libiconv to transcode
|
||||
to UTF-8.
|
||||
|
||||
Return C++ strings without conversion
|
||||
-------------------------------------
|
||||
|
||||
If the data in a C++ ``std::string`` does not represent text and should be
|
||||
returned to Python as ``bytes``, then one can return the data as a
|
||||
``py::bytes`` object.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
m.def("return_bytes",
|
||||
[]() {
|
||||
std::string s("\xba\xd0\xba\xd0"); // Not valid UTF-8
|
||||
return py::bytes(s); // Return the data without transcoding
|
||||
}
|
||||
);
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> example.return_bytes()
|
||||
b'\xba\xd0\xba\xd0'
|
||||
|
||||
|
||||
Note the asymmetry: pybind11 will convert ``bytes`` to ``std::string`` without
|
||||
encoding, but cannot convert ``std::string`` back to ``bytes`` implicitly.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
m.def("asymmetry",
|
||||
[](std::string s) { // Accepts str or bytes from Python
|
||||
return s; // Looks harmless, but implicitly converts to str
|
||||
}
|
||||
);
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> isinstance(example.asymmetry(b"have some bytes"), str)
|
||||
True
|
||||
|
||||
>>> example.asymmetry(b"\xba\xd0\xba\xd0") # invalid utf-8 as bytes
|
||||
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xba in position 0: invalid start byte
|
||||
|
||||
|
||||
Wide character strings
|
||||
======================
|
||||
|
||||
When a Python ``str`` is passed to a C++ function expecting ``std::wstring``,
|
||||
``wchar_t*``, ``std::u16string`` or ``std::u32string``, the ``str`` will be
|
||||
encoded to UTF-16 or UTF-32 depending on how the C++ compiler implements each
|
||||
type, in the platform's native endianness. When strings of these types are
|
||||
returned, they are assumed to contain valid UTF-16 or UTF-32, and will be
|
||||
decoded to Python ``str``.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
#define UNICODE
|
||||
#include <windows.h>
|
||||
|
||||
m.def("set_window_text",
|
||||
[](HWND hwnd, std::wstring s) {
|
||||
// Call SetWindowText with null-terminated UTF-16 string
|
||||
::SetWindowText(hwnd, s.c_str());
|
||||
}
|
||||
);
|
||||
m.def("get_window_text",
|
||||
[](HWND hwnd) {
|
||||
const int buffer_size = ::GetWindowTextLength(hwnd) + 1;
|
||||
auto buffer = std::make_unique< wchar_t[] >(buffer_size);
|
||||
|
||||
::GetWindowText(hwnd, buffer.data(), buffer_size);
|
||||
|
||||
std::wstring text(buffer.get());
|
||||
|
||||
// wstring will be converted to Python str
|
||||
return text;
|
||||
}
|
||||
);
|
||||
|
||||
.. warning::
|
||||
|
||||
Wide character strings may not work as described on Python 2.7 or Python
|
||||
3.3 compiled with ``--enable-unicode=ucs2``.
|
||||
|
||||
Strings in multibyte encodings such as Shift-JIS must transcoded to a
|
||||
UTF-8/16/32 before being returned to Python.
|
||||
|
||||
|
||||
Character literals
|
||||
==================
|
||||
|
||||
C++ functions that accept character literals as input will receive the first
|
||||
character of a Python ``str`` as their input. If the string is longer than one
|
||||
Unicode character, trailing characters will be ignored.
|
||||
|
||||
When a character literal is returned from C++ (such as a ``char`` or a
|
||||
``wchar_t``), it will be converted to a ``str`` that represents the single
|
||||
character.
|
||||
|
||||
.. code-block:: c++
|
||||
|
||||
m.def("pass_char", [](char c) { return c; });
|
||||
m.def("pass_wchar", [](wchar_t w) { return w; });
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> example.pass_char('A')
|
||||
'A'
|
||||
|
||||
While C++ will cast integers to character types (``char c = 0x65;``), pybind11
|
||||
does not convert Python integers to characters implicitly. The Python function
|
||||
``chr()`` can be used to convert integers to characters.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> example.pass_char(0x65)
|
||||
TypeError
|
||||
|
||||
>>> example.pass_char(chr(0x65))
|
||||
'A'
|
||||
|
||||
If the desire is to work with an 8-bit integer, use ``int8_t`` or ``uint8_t``
|
||||
as the argument type.
|
||||
|
||||
Grapheme clusters
|
||||
-----------------
|
||||
|
||||
A single grapheme may be represented by two or more Unicode characters. For
|
||||
example 'é' is usually represented as U+00E9 but can also be expressed as the
|
||||
combining character sequence U+0065 U+0301 (that is, the letter 'e' followed by
|
||||
a combining acute accent). The combining character will be lost if the
|
||||
two-character sequence is passed as an argument, even though it renders as a
|
||||
single grapheme.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> example.pass_wchar('é')
|
||||
'é'
|
||||
|
||||
>>> combining_e_acute = 'e' + '\u0301'
|
||||
|
||||
>>> combining_e_acute
|
||||
'é'
|
||||
|
||||
>>> combining_e_acute == 'é'
|
||||
False
|
||||
|
||||
>>> example.pass_wchar(combining_e_acute)
|
||||
'e'
|
||||
|
||||
Normalizing combining characters before passing the character literal to C++
|
||||
may resolve *some* of these issues:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
>>> example.pass_wchar(unicodedata.normalize('NFC', combining_e_acute))
|
||||
'é'
|
||||
|
||||
In some languages (Thai for example), there are `graphemes that cannot be
|
||||
expressed as a single Unicode code point
|
||||
<http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries>`_, so there is
|
||||
no way to capture them in a C++ character type.
|
||||
|
||||
|
||||
C++17 string views
|
||||
==================
|
||||
|
||||
C++17 string views are automatically supported when compiling in C++17 mode.
|
||||
They follow the same rules for encoding and decoding as the corresponding STL
|
||||
string type (for example, a ``std::u16string_view`` argument will be passed
|
||||
UTF-16-encoded data, and a returned ``std::string_view`` will be decoded as
|
||||
UTF-8).
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
* `The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) <https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/>`_
|
||||
* `C++ - Using STL Strings at Win32 API Boundaries <https://msdn.microsoft.com/en-ca/magazine/mt238407.aspx>`_
|
1126
lib/bip158/lib/pybind11/docs/advanced/classes.rst
Normal file
1126
lib/bip158/lib/pybind11/docs/advanced/classes.rst
Normal file
File diff suppressed because it is too large
Load diff
261
lib/bip158/lib/pybind11/docs/advanced/embedding.rst
Normal file
261
lib/bip158/lib/pybind11/docs/advanced/embedding.rst
Normal file
|
@ -0,0 +1,261 @@
|
|||
.. _embedding:
|
||||
|
||||
Embedding the interpreter
|
||||
#########################
|
||||
|
||||
While pybind11 is mainly focused on extending Python using C++, it's also
|
||||
possible to do the reverse: embed the Python interpreter into a C++ program.
|
||||
All of the other documentation pages still apply here, so refer to them for
|
||||
general pybind11 usage. This section will cover a few extra things required
|
||||
for embedding.
|
||||
|
||||
Getting started
|
||||
===============
|
||||
|
||||
A basic executable with an embedded interpreter can be created with just a few
|
||||
lines of CMake and the ``pybind11::embed`` target, as shown below. For more
|
||||
information, see :doc:`/compiling`.
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
cmake_minimum_required(VERSION 3.0)
|
||||
project(example)
|
||||
|
||||
find_package(pybind11 REQUIRED) # or `add_subdirectory(pybind11)`
|
||||
|
||||
add_executable(example main.cpp)
|
||||
target_link_libraries(example PRIVATE pybind11::embed)
|
||||
|
||||
The essential structure of the ``main.cpp`` file looks like this:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h> // everything needed for embedding
|
||||
namespace py = pybind11;
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{}; // start the interpreter and keep it alive
|
||||
|
||||
py::print("Hello, World!"); // use the Python API
|
||||
}
|
||||
|
||||
The interpreter must be initialized before using any Python API, which includes
|
||||
all the functions and classes in pybind11. The RAII guard class `scoped_interpreter`
|
||||
takes care of the interpreter lifetime. After the guard is destroyed, the interpreter
|
||||
shuts down and clears its memory. No Python functions can be called after this.
|
||||
|
||||
Executing Python code
|
||||
=====================
|
||||
|
||||
There are a few different ways to run Python code. One option is to use `eval`,
|
||||
`exec` or `eval_file`, as explained in :ref:`eval`. Here is a quick example in
|
||||
the context of an executable with an embedded interpreter:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h>
|
||||
namespace py = pybind11;
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{};
|
||||
|
||||
py::exec(R"(
|
||||
kwargs = dict(name="World", number=42)
|
||||
message = "Hello, {name}! The answer is {number}".format(**kwargs)
|
||||
print(message)
|
||||
)");
|
||||
}
|
||||
|
||||
Alternatively, similar results can be achieved using pybind11's API (see
|
||||
:doc:`/advanced/pycpp/index` for more details).
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h>
|
||||
namespace py = pybind11;
|
||||
using namespace py::literals;
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{};
|
||||
|
||||
auto kwargs = py::dict("name"_a="World", "number"_a=42);
|
||||
auto message = "Hello, {name}! The answer is {number}"_s.format(**kwargs);
|
||||
py::print(message);
|
||||
}
|
||||
|
||||
The two approaches can also be combined:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h>
|
||||
#include <iostream>
|
||||
|
||||
namespace py = pybind11;
|
||||
using namespace py::literals;
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{};
|
||||
|
||||
auto locals = py::dict("name"_a="World", "number"_a=42);
|
||||
py::exec(R"(
|
||||
message = "Hello, {name}! The answer is {number}".format(**locals())
|
||||
)", py::globals(), locals);
|
||||
|
||||
auto message = locals["message"].cast<std::string>();
|
||||
std::cout << message;
|
||||
}
|
||||
|
||||
Importing modules
|
||||
=================
|
||||
|
||||
Python modules can be imported using `module::import()`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::module sys = py::module::import("sys");
|
||||
py::print(sys.attr("path"));
|
||||
|
||||
For convenience, the current working directory is included in ``sys.path`` when
|
||||
embedding the interpreter. This makes it easy to import local Python files:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
"""calc.py located in the working directory"""
|
||||
|
||||
def add(i, j):
|
||||
return i + j
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::module calc = py::module::import("calc");
|
||||
py::object result = calc.attr("add")(1, 2);
|
||||
int n = result.cast<int>();
|
||||
assert(n == 3);
|
||||
|
||||
Modules can be reloaded using `module::reload()` if the source is modified e.g.
|
||||
by an external process. This can be useful in scenarios where the application
|
||||
imports a user defined data processing script which needs to be updated after
|
||||
changes by the user. Note that this function does not reload modules recursively.
|
||||
|
||||
.. _embedding_modules:
|
||||
|
||||
Adding embedded modules
|
||||
=======================
|
||||
|
||||
Embedded binary modules can be added using the `PYBIND11_EMBEDDED_MODULE` macro.
|
||||
Note that the definition must be placed at global scope. They can be imported
|
||||
like any other module.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h>
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_EMBEDDED_MODULE(fast_calc, m) {
|
||||
// `m` is a `py::module` which is used to bind functions and classes
|
||||
m.def("add", [](int i, int j) {
|
||||
return i + j;
|
||||
});
|
||||
}
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{};
|
||||
|
||||
auto fast_calc = py::module::import("fast_calc");
|
||||
auto result = fast_calc.attr("add")(1, 2).cast<int>();
|
||||
assert(result == 3);
|
||||
}
|
||||
|
||||
Unlike extension modules where only a single binary module can be created, on
|
||||
the embedded side an unlimited number of modules can be added using multiple
|
||||
`PYBIND11_EMBEDDED_MODULE` definitions (as long as they have unique names).
|
||||
|
||||
These modules are added to Python's list of builtins, so they can also be
|
||||
imported in pure Python files loaded by the interpreter. Everything interacts
|
||||
naturally:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
"""py_module.py located in the working directory"""
|
||||
import cpp_module
|
||||
|
||||
a = cpp_module.a
|
||||
b = a + 1
|
||||
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/embed.h>
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_EMBEDDED_MODULE(cpp_module, m) {
|
||||
m.attr("a") = 1;
|
||||
}
|
||||
|
||||
int main() {
|
||||
py::scoped_interpreter guard{};
|
||||
|
||||
auto py_module = py::module::import("py_module");
|
||||
|
||||
auto locals = py::dict("fmt"_a="{} + {} = {}", **py_module.attr("__dict__"));
|
||||
assert(locals["a"].cast<int>() == 1);
|
||||
assert(locals["b"].cast<int>() == 2);
|
||||
|
||||
py::exec(R"(
|
||||
c = a + b
|
||||
message = fmt.format(a, b, c)
|
||||
)", py::globals(), locals);
|
||||
|
||||
assert(locals["c"].cast<int>() == 3);
|
||||
assert(locals["message"].cast<std::string>() == "1 + 2 = 3");
|
||||
}
|
||||
|
||||
|
||||
Interpreter lifetime
|
||||
====================
|
||||
|
||||
The Python interpreter shuts down when `scoped_interpreter` is destroyed. After
|
||||
this, creating a new instance will restart the interpreter. Alternatively, the
|
||||
`initialize_interpreter` / `finalize_interpreter` pair of functions can be used
|
||||
to directly set the state at any time.
|
||||
|
||||
Modules created with pybind11 can be safely re-initialized after the interpreter
|
||||
has been restarted. However, this may not apply to third-party extension modules.
|
||||
The issue is that Python itself cannot completely unload extension modules and
|
||||
there are several caveats with regard to interpreter restarting. In short, not
|
||||
all memory may be freed, either due to Python reference cycles or user-created
|
||||
global data. All the details can be found in the CPython documentation.
|
||||
|
||||
.. warning::
|
||||
|
||||
Creating two concurrent `scoped_interpreter` guards is a fatal error. So is
|
||||
calling `initialize_interpreter` for a second time after the interpreter
|
||||
has already been initialized.
|
||||
|
||||
Do not use the raw CPython API functions ``Py_Initialize`` and
|
||||
``Py_Finalize`` as these do not properly handle the lifetime of
|
||||
pybind11's internal data.
|
||||
|
||||
|
||||
Sub-interpreter support
|
||||
=======================
|
||||
|
||||
Creating multiple copies of `scoped_interpreter` is not possible because it
|
||||
represents the main Python interpreter. Sub-interpreters are something different
|
||||
and they do permit the existence of multiple interpreters. This is an advanced
|
||||
feature of the CPython API and should be handled with care. pybind11 does not
|
||||
currently offer a C++ interface for sub-interpreters, so refer to the CPython
|
||||
documentation for all the details regarding this feature.
|
||||
|
||||
We'll just mention a couple of caveats the sub-interpreters support in pybind11:
|
||||
|
||||
1. Sub-interpreters will not receive independent copies of embedded modules.
|
||||
Instead, these are shared and modifications in one interpreter may be
|
||||
reflected in another.
|
||||
|
||||
2. Managing multiple threads, multiple interpreters and the GIL can be
|
||||
challenging and there are several caveats here, even within the pure
|
||||
CPython API (please refer to the Python docs for details). As for
|
||||
pybind11, keep in mind that `gil_scoped_release` and `gil_scoped_acquire`
|
||||
do not take sub-interpreters into account.
|
144
lib/bip158/lib/pybind11/docs/advanced/exceptions.rst
Normal file
144
lib/bip158/lib/pybind11/docs/advanced/exceptions.rst
Normal file
|
@ -0,0 +1,144 @@
|
|||
Exceptions
|
||||
##########
|
||||
|
||||
Built-in exception translation
|
||||
==============================
|
||||
|
||||
When C++ code invoked from Python throws an ``std::exception``, it is
|
||||
automatically converted into a Python ``Exception``. pybind11 defines multiple
|
||||
special exception classes that will map to different types of Python
|
||||
exceptions:
|
||||
|
||||
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
|
||||
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| C++ exception type | Python exception type |
|
||||
+======================================+======================================+
|
||||
| :class:`std::exception` | ``RuntimeError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::bad_alloc` | ``MemoryError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::domain_error` | ``ValueError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::invalid_argument` | ``ValueError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::length_error` | ``ValueError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::out_of_range` | ``IndexError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::range_error` | ``ValueError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`std::overflow_error` | ``OverflowError`` |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`pybind11::stop_iteration` | ``StopIteration`` (used to implement |
|
||||
| | custom iterators) |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`pybind11::index_error` | ``IndexError`` (used to indicate out |
|
||||
| | of bounds access in ``__getitem__``, |
|
||||
| | ``__setitem__``, etc.) |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
|
||||
| | wrong value passed in |
|
||||
| | ``container.remove(...)``) |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`pybind11::key_error` | ``KeyError`` (used to indicate out |
|
||||
| | of bounds access in ``__getitem__``, |
|
||||
| | ``__setitem__`` in dict-like |
|
||||
| | objects, etc.) |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
| :class:`pybind11::error_already_set` | Indicates that the Python exception |
|
||||
| | flag has already been set via Python |
|
||||
| | API calls from C++ code; this C++ |
|
||||
| | exception is used to propagate such |
|
||||
| | a Python exception back to Python. |
|
||||
+--------------------------------------+--------------------------------------+
|
||||
|
||||
When a Python function invoked from C++ throws an exception, it is converted
|
||||
into a C++ exception of type :class:`error_already_set` whose string payload
|
||||
contains a textual summary.
|
||||
|
||||
There is also a special exception :class:`cast_error` that is thrown by
|
||||
:func:`handle::call` when the input arguments cannot be converted to Python
|
||||
objects.
|
||||
|
||||
Registering custom translators
|
||||
==============================
|
||||
|
||||
If the default exception conversion policy described above is insufficient,
|
||||
pybind11 also provides support for registering custom exception translators.
|
||||
To register a simple exception conversion that translates a C++ exception into
|
||||
a new Python exception using the C++ exception's ``what()`` method, a helper
|
||||
function is available:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::register_exception<CppExp>(module, "PyExp");
|
||||
|
||||
This call creates a Python exception class with the name ``PyExp`` in the given
|
||||
module and automatically converts any encountered exceptions of type ``CppExp``
|
||||
into Python exceptions of type ``PyExp``.
|
||||
|
||||
When more advanced exception translation is needed, the function
|
||||
``py::register_exception_translator(translator)`` can be used to register
|
||||
functions that can translate arbitrary exception types (and which may include
|
||||
additional logic to do so). The function takes a stateless callable (e.g. a
|
||||
function pointer or a lambda function without captured variables) with the call
|
||||
signature ``void(std::exception_ptr)``.
|
||||
|
||||
When a C++ exception is thrown, the registered exception translators are tried
|
||||
in reverse order of registration (i.e. the last registered translator gets the
|
||||
first shot at handling the exception).
|
||||
|
||||
Inside the translator, ``std::rethrow_exception`` should be used within
|
||||
a try block to re-throw the exception. One or more catch clauses to catch
|
||||
the appropriate exceptions should then be used with each clause using
|
||||
``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set
|
||||
the python exception to a custom exception type (see below).
|
||||
|
||||
To declare a custom Python exception type, declare a ``py::exception`` variable
|
||||
and use this in the associated exception translator (note: it is often useful
|
||||
to make this a static declaration when using it inside a lambda expression
|
||||
without requiring capturing).
|
||||
|
||||
|
||||
The following example demonstrates this for a hypothetical exception classes
|
||||
``MyCustomException`` and ``OtherException``: the first is translated to a
|
||||
custom python exception ``MyCustomError``, while the second is translated to a
|
||||
standard python RuntimeError:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
static py::exception<MyCustomException> exc(m, "MyCustomError");
|
||||
py::register_exception_translator([](std::exception_ptr p) {
|
||||
try {
|
||||
if (p) std::rethrow_exception(p);
|
||||
} catch (const MyCustomException &e) {
|
||||
exc(e.what());
|
||||
} catch (const OtherException &e) {
|
||||
PyErr_SetString(PyExc_RuntimeError, e.what());
|
||||
}
|
||||
});
|
||||
|
||||
Multiple exceptions can be handled by a single translator, as shown in the
|
||||
example above. If the exception is not caught by the current translator, the
|
||||
previously registered one gets a chance.
|
||||
|
||||
If none of the registered exception translators is able to handle the
|
||||
exception, it is handled by the default converter as described in the previous
|
||||
section.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_exceptions.cpp` contains examples
|
||||
of various custom exception translators and custom exception types.
|
||||
|
||||
.. note::
|
||||
|
||||
You must call either ``PyErr_SetString`` or a custom exception's call
|
||||
operator (``exc(string)``) for every exception caught in a custom exception
|
||||
translator. Failure to do so will cause Python to crash with ``SystemError:
|
||||
error return without exception set``.
|
||||
|
||||
Exceptions that you do not plan to handle should simply not be caught, or
|
||||
may be explicitly (re-)thrown to delegate it to the other,
|
||||
previously-declared existing exception translators.
|
507
lib/bip158/lib/pybind11/docs/advanced/functions.rst
Normal file
507
lib/bip158/lib/pybind11/docs/advanced/functions.rst
Normal file
|
@ -0,0 +1,507 @@
|
|||
Functions
|
||||
#########
|
||||
|
||||
Before proceeding with this section, make sure that you are already familiar
|
||||
with the basics of binding functions and classes, as explained in :doc:`/basics`
|
||||
and :doc:`/classes`. The following guide is applicable to both free and member
|
||||
functions, i.e. *methods* in Python.
|
||||
|
||||
.. _return_value_policies:
|
||||
|
||||
Return value policies
|
||||
=====================
|
||||
|
||||
Python and C++ use fundamentally different ways of managing the memory and
|
||||
lifetime of objects managed by them. This can lead to issues when creating
|
||||
bindings for functions that return a non-trivial type. Just by looking at the
|
||||
type information, it is not clear whether Python should take charge of the
|
||||
returned value and eventually free its resources, or if this is handled on the
|
||||
C++ side. For this reason, pybind11 provides a several *return value policy*
|
||||
annotations that can be passed to the :func:`module::def` and
|
||||
:func:`class_::def` functions. The default policy is
|
||||
:enum:`return_value_policy::automatic`.
|
||||
|
||||
Return value policies are tricky, and it's very important to get them right.
|
||||
Just to illustrate what can go wrong, consider the following simple example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
/* Function declaration */
|
||||
Data *get_data() { return _data; /* (pointer to a static data structure) */ }
|
||||
...
|
||||
|
||||
/* Binding code */
|
||||
m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python
|
||||
|
||||
What's going on here? When ``get_data()`` is called from Python, the return
|
||||
value (a native C++ type) must be wrapped to turn it into a usable Python type.
|
||||
In this case, the default return value policy (:enum:`return_value_policy::automatic`)
|
||||
causes pybind11 to assume ownership of the static ``_data`` instance.
|
||||
|
||||
When Python's garbage collector eventually deletes the Python
|
||||
wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator
|
||||
delete()``) due to the implied ownership. At this point, the entire application
|
||||
will come crashing down, though errors could also be more subtle and involve
|
||||
silent data corruption.
|
||||
|
||||
In the above example, the policy :enum:`return_value_policy::reference` should have
|
||||
been specified so that the global data instance is only *referenced* without any
|
||||
implied transfer of ownership, i.e.:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("get_data", &get_data, return_value_policy::reference);
|
||||
|
||||
On the other hand, this is not the right policy for many other situations,
|
||||
where ignoring ownership could lead to resource leaks.
|
||||
As a developer using pybind11, it's important to be familiar with the different
|
||||
return value policies, including which situation calls for which one of them.
|
||||
The following table provides an overview of available policies:
|
||||
|
||||
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
|
||||
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| Return value policy | Description |
|
||||
+==================================================+============================================================================+
|
||||
| :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take |
|
||||
| | ownership. Python will call the destructor and delete operator when the |
|
||||
| | object's reference count reaches zero. Undefined behavior ensues when the |
|
||||
| | C++ side does the same, or when the data was not dynamically allocated. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. |
|
||||
| | This policy is comparably safe because the lifetimes of the two instances |
|
||||
| | are decoupled. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::move` | Use ``std::move`` to move the return value contents into a new instance |
|
||||
| | that will be owned by Python. This policy is comparably safe because the |
|
||||
| | lifetimes of the two instances (move source and destination) are decoupled.|
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::reference` | Reference an existing object, but do not take ownership. The C++ side is |
|
||||
| | responsible for managing the object's lifetime and deallocating it when |
|
||||
| | it is no longer used. Warning: undefined behavior will ensue when the C++ |
|
||||
| | side deletes an object that is still referenced and used by Python. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::reference_internal` | Indicates that the lifetime of the return value is tied to the lifetime |
|
||||
| | of a parent object, namely the implicit ``this``, or ``self`` argument of |
|
||||
| | the called method or property. Internally, this policy works just like |
|
||||
| | :enum:`return_value_policy::reference` but additionally applies a |
|
||||
| | ``keep_alive<0, 1>`` *call policy* (described in the next section) that |
|
||||
| | prevents the parent object from being garbage collected as long as the |
|
||||
| | return value is referenced by Python. This is the default policy for |
|
||||
| | property getters created via ``def_property``, ``def_readwrite``, etc. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::automatic` | **Default policy.** This policy falls back to the policy |
|
||||
| | :enum:`return_value_policy::take_ownership` when the return value is a |
|
||||
| | pointer. Otherwise, it uses :enum:`return_value_policy::move` or |
|
||||
| | :enum:`return_value_policy::copy` for rvalue and lvalue references, |
|
||||
| | respectively. See above for a description of what all of these different |
|
||||
| | policies do. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
|
||||
| | return value is a pointer. This is the default conversion policy for |
|
||||
| | function arguments when calling Python functions manually from C++ code |
|
||||
| | (i.e. via handle::operator()). You probably won't need to use this. |
|
||||
+--------------------------------------------------+----------------------------------------------------------------------------+
|
||||
|
||||
Return value policies can also be applied to properties:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class_<MyClass>(m, "MyClass")
|
||||
.def_property("data", &MyClass::getData, &MyClass::setData,
|
||||
py::return_value_policy::copy);
|
||||
|
||||
Technically, the code above applies the policy to both the getter and the
|
||||
setter function, however, the setter doesn't really care about *return*
|
||||
value policies which makes this a convenient terse syntax. Alternatively,
|
||||
targeted arguments can be passed through the :class:`cpp_function` constructor:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class_<MyClass>(m, "MyClass")
|
||||
.def_property("data"
|
||||
py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
|
||||
py::cpp_function(&MyClass::setData)
|
||||
);
|
||||
|
||||
.. warning::
|
||||
|
||||
Code with invalid return value policies might access uninitialized memory or
|
||||
free data structures multiple times, which can lead to hard-to-debug
|
||||
non-determinism and segmentation faults, hence it is worth spending the
|
||||
time to understand all the different options in the table above.
|
||||
|
||||
.. note::
|
||||
|
||||
One important aspect of the above policies is that they only apply to
|
||||
instances which pybind11 has *not* seen before, in which case the policy
|
||||
clarifies essential questions about the return value's lifetime and
|
||||
ownership. When pybind11 knows the instance already (as identified by its
|
||||
type and address in memory), it will return the existing Python object
|
||||
wrapper rather than creating a new copy.
|
||||
|
||||
.. note::
|
||||
|
||||
The next section on :ref:`call_policies` discusses *call policies* that can be
|
||||
specified *in addition* to a return value policy from the list above. Call
|
||||
policies indicate reference relationships that can involve both return values
|
||||
and parameters of functions.
|
||||
|
||||
.. note::
|
||||
|
||||
As an alternative to elaborate call policies and lifetime management logic,
|
||||
consider using smart pointers (see the section on :ref:`smart_pointers` for
|
||||
details). Smart pointers can tell whether an object is still referenced from
|
||||
C++ or Python, which generally eliminates the kinds of inconsistencies that
|
||||
can lead to crashes or undefined behavior. For functions returning smart
|
||||
pointers, it is not necessary to specify a return value policy.
|
||||
|
||||
.. _call_policies:
|
||||
|
||||
Additional call policies
|
||||
========================
|
||||
|
||||
In addition to the above return value policies, further *call policies* can be
|
||||
specified to indicate dependencies between parameters or ensure a certain state
|
||||
for the function call.
|
||||
|
||||
Keep alive
|
||||
----------
|
||||
|
||||
In general, this policy is required when the C++ object is any kind of container
|
||||
and another object is being added to the container. ``keep_alive<Nurse, Patient>``
|
||||
indicates that the argument with index ``Patient`` should be kept alive at least
|
||||
until the argument with index ``Nurse`` is freed by the garbage collector. Argument
|
||||
indices start at one, while zero refers to the return value. For methods, index
|
||||
``1`` refers to the implicit ``this`` pointer, while regular arguments begin at
|
||||
index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse``
|
||||
with value ``None`` is detected at runtime, the call policy does nothing.
|
||||
|
||||
When the nurse is not a pybind11-registered type, the implementation internally
|
||||
relies on the ability to create a *weak reference* to the nurse object. When
|
||||
the nurse object is not a pybind11-registered type and does not support weak
|
||||
references, an exception will be thrown.
|
||||
|
||||
Consider the following example: here, the binding code for a list append
|
||||
operation ties the lifetime of the newly added element to the underlying
|
||||
container:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<List>(m, "List")
|
||||
.def("append", &List::append, py::keep_alive<1, 2>());
|
||||
|
||||
For consistency, the argument indexing is identical for constructors. Index
|
||||
``1`` still refers to the implicit ``this`` pointer, i.e. the object which is
|
||||
being constructed. Index ``0`` refers to the return type which is presumed to
|
||||
be ``void`` when a constructor is viewed like a function. The following example
|
||||
ties the lifetime of the constructor element to the constructed object:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Nurse>(m, "Nurse")
|
||||
.def(py::init<Patient &>(), py::keep_alive<1, 2>());
|
||||
|
||||
.. note::
|
||||
|
||||
``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse,
|
||||
Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient ==
|
||||
0) policies from Boost.Python.
|
||||
|
||||
Call guard
|
||||
----------
|
||||
|
||||
The ``call_guard<T>`` policy allows any scope guard type ``T`` to be placed
|
||||
around the function call. For example, this definition:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("foo", foo, py::call_guard<T>());
|
||||
|
||||
is equivalent to the following pseudocode:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("foo", [](args...) {
|
||||
T scope_guard;
|
||||
return foo(args...); // forwarded arguments
|
||||
});
|
||||
|
||||
The only requirement is that ``T`` is default-constructible, but otherwise any
|
||||
scope guard will work. This is very useful in combination with `gil_scoped_release`.
|
||||
See :ref:`gil`.
|
||||
|
||||
Multiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The
|
||||
constructor order is left to right and destruction happens in reverse.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_call_policies.cpp` contains a complete example
|
||||
that demonstrates using `keep_alive` and `call_guard` in more detail.
|
||||
|
||||
.. _python_objects_as_args:
|
||||
|
||||
Python objects as arguments
|
||||
===========================
|
||||
|
||||
pybind11 exposes all major Python types using thin C++ wrapper classes. These
|
||||
wrapper classes can also be used as parameters of functions in bindings, which
|
||||
makes it possible to directly work with native Python types on the C++ side.
|
||||
For instance, the following statement iterates over a Python ``dict``:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void print_dict(py::dict dict) {
|
||||
/* Easily interact with Python types */
|
||||
for (auto item : dict)
|
||||
std::cout << "key=" << std::string(py::str(item.first)) << ", "
|
||||
<< "value=" << std::string(py::str(item.second)) << std::endl;
|
||||
}
|
||||
|
||||
It can be exported:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("print_dict", &print_dict);
|
||||
|
||||
And used in Python as usual:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print_dict({'foo': 123, 'bar': 'hello'})
|
||||
key=foo, value=123
|
||||
key=bar, value=hello
|
||||
|
||||
For more information on using Python objects in C++, see :doc:`/advanced/pycpp/index`.
|
||||
|
||||
Accepting \*args and \*\*kwargs
|
||||
===============================
|
||||
|
||||
Python provides a useful mechanism to define functions that accept arbitrary
|
||||
numbers of arguments and keyword arguments:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def generic(*args, **kwargs):
|
||||
... # do something with args and kwargs
|
||||
|
||||
Such functions can also be created using pybind11:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void generic(py::args args, py::kwargs kwargs) {
|
||||
/// .. do something with args
|
||||
if (kwargs)
|
||||
/// .. do something with kwargs
|
||||
}
|
||||
|
||||
/// Binding code
|
||||
m.def("generic", &generic);
|
||||
|
||||
The class ``py::args`` derives from ``py::tuple`` and ``py::kwargs`` derives
|
||||
from ``py::dict``.
|
||||
|
||||
You may also use just one or the other, and may combine these with other
|
||||
arguments as long as the ``py::args`` and ``py::kwargs`` arguments are the last
|
||||
arguments accepted by the function.
|
||||
|
||||
Please refer to the other examples for details on how to iterate over these,
|
||||
and on how to cast their entries into C++ objects. A demonstration is also
|
||||
available in ``tests/test_kwargs_and_defaults.cpp``.
|
||||
|
||||
.. note::
|
||||
|
||||
When combining \*args or \*\*kwargs with :ref:`keyword_args` you should
|
||||
*not* include ``py::arg`` tags for the ``py::args`` and ``py::kwargs``
|
||||
arguments.
|
||||
|
||||
Default arguments revisited
|
||||
===========================
|
||||
|
||||
The section on :ref:`default_args` previously discussed basic usage of default
|
||||
arguments using pybind11. One noteworthy aspect of their implementation is that
|
||||
default arguments are converted to Python objects right at declaration time.
|
||||
Consider the following example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<MyClass>("MyClass")
|
||||
.def("myFunction", py::arg("arg") = SomeType(123));
|
||||
|
||||
In this case, pybind11 must already be set up to deal with values of the type
|
||||
``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an
|
||||
exception will be thrown.
|
||||
|
||||
Another aspect worth highlighting is that the "preview" of the default argument
|
||||
in the function signature is generated using the object's ``__repr__`` method.
|
||||
If not available, the signature may not be very helpful, e.g.:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
FUNCTIONS
|
||||
...
|
||||
| myFunction(...)
|
||||
| Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> NoneType
|
||||
...
|
||||
|
||||
The first way of addressing this is by defining ``SomeType.__repr__``.
|
||||
Alternatively, it is possible to specify the human-readable preview of the
|
||||
default argument manually using the ``arg_v`` notation:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<MyClass>("MyClass")
|
||||
.def("myFunction", py::arg_v("arg", SomeType(123), "SomeType(123)"));
|
||||
|
||||
Sometimes it may be necessary to pass a null pointer value as a default
|
||||
argument. In this case, remember to cast it to the underlying type in question,
|
||||
like so:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<MyClass>("MyClass")
|
||||
.def("myFunction", py::arg("arg") = (SomeType *) nullptr);
|
||||
|
||||
.. _nonconverting_arguments:
|
||||
|
||||
Non-converting arguments
|
||||
========================
|
||||
|
||||
Certain argument types may support conversion from one type to another. Some
|
||||
examples of conversions are:
|
||||
|
||||
* :ref:`implicit_conversions` declared using ``py::implicitly_convertible<A,B>()``
|
||||
* Calling a method accepting a double with an integer argument
|
||||
* Calling a ``std::complex<float>`` argument with a non-complex python type
|
||||
(for example, with a float). (Requires the optional ``pybind11/complex.h``
|
||||
header).
|
||||
* Calling a function taking an Eigen matrix reference with a numpy array of the
|
||||
wrong type or of an incompatible data layout. (Requires the optional
|
||||
``pybind11/eigen.h`` header).
|
||||
|
||||
This behaviour is sometimes undesirable: the binding code may prefer to raise
|
||||
an error rather than convert the argument. This behaviour can be obtained
|
||||
through ``py::arg`` by calling the ``.noconvert()`` method of the ``py::arg``
|
||||
object, such as:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("floats_only", [](double f) { return 0.5 * f; }, py::arg("f").noconvert());
|
||||
m.def("floats_preferred", [](double f) { return 0.5 * f; }, py::arg("f"));
|
||||
|
||||
Attempting the call the second function (the one without ``.noconvert()``) with
|
||||
an integer will succeed, but attempting to call the ``.noconvert()`` version
|
||||
will fail with a ``TypeError``:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> floats_preferred(4)
|
||||
2.0
|
||||
>>> floats_only(4)
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
TypeError: floats_only(): incompatible function arguments. The following argument types are supported:
|
||||
1. (f: float) -> float
|
||||
|
||||
Invoked with: 4
|
||||
|
||||
You may, of course, combine this with the :var:`_a` shorthand notation (see
|
||||
:ref:`keyword_args`) and/or :ref:`default_args`. It is also permitted to omit
|
||||
the argument name by using the ``py::arg()`` constructor without an argument
|
||||
name, i.e. by specifying ``py::arg().noconvert()``.
|
||||
|
||||
.. note::
|
||||
|
||||
When specifying ``py::arg`` options it is necessary to provide the same
|
||||
number of options as the bound function has arguments. Thus if you want to
|
||||
enable no-convert behaviour for just one of several arguments, you will
|
||||
need to specify a ``py::arg()`` annotation for each argument with the
|
||||
no-convert argument modified to ``py::arg().noconvert()``.
|
||||
|
||||
.. _none_arguments:
|
||||
|
||||
Allow/Prohibiting None arguments
|
||||
================================
|
||||
|
||||
When a C++ type registered with :class:`py::class_` is passed as an argument to
|
||||
a function taking the instance as pointer or shared holder (e.g. ``shared_ptr``
|
||||
or a custom, copyable holder as described in :ref:`smart_pointers`), pybind
|
||||
allows ``None`` to be passed from Python which results in calling the C++
|
||||
function with ``nullptr`` (or an empty holder) for the argument.
|
||||
|
||||
To explicitly enable or disable this behaviour, using the
|
||||
``.none`` method of the :class:`py::arg` object:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Dog>(m, "Dog").def(py::init<>());
|
||||
py::class_<Cat>(m, "Cat").def(py::init<>());
|
||||
m.def("bark", [](Dog *dog) -> std::string {
|
||||
if (dog) return "woof!"; /* Called with a Dog instance */
|
||||
else return "(no dog)"; /* Called with None, dog == nullptr */
|
||||
}, py::arg("dog").none(true));
|
||||
m.def("meow", [](Cat *cat) -> std::string {
|
||||
// Can't be called with None argument
|
||||
return "meow";
|
||||
}, py::arg("cat").none(false));
|
||||
|
||||
With the above, the Python call ``bark(None)`` will return the string ``"(no
|
||||
dog)"``, while attempting to call ``meow(None)`` will raise a ``TypeError``:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> from animals import Dog, Cat, bark, meow
|
||||
>>> bark(Dog())
|
||||
'woof!'
|
||||
>>> meow(Cat())
|
||||
'meow'
|
||||
>>> bark(None)
|
||||
'(no dog)'
|
||||
>>> meow(None)
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
TypeError: meow(): incompatible function arguments. The following argument types are supported:
|
||||
1. (cat: animals.Cat) -> str
|
||||
|
||||
Invoked with: None
|
||||
|
||||
The default behaviour when the tag is unspecified is to allow ``None``.
|
||||
|
||||
.. note::
|
||||
|
||||
Even when ``.none(true)`` is specified for an argument, ``None`` will be converted to a
|
||||
``nullptr`` *only* for custom and :ref:`opaque <opaque>` types. Pointers to built-in types
|
||||
(``double *``, ``int *``, ...) and STL types (``std::vector<T> *``, ...; if ``pybind11/stl.h``
|
||||
is included) are copied when converted to C++ (see :doc:`/advanced/cast/overview`) and will
|
||||
not allow ``None`` as argument. To pass optional argument of these copied types consider
|
||||
using ``std::optional<T>``
|
||||
|
||||
Overload resolution order
|
||||
=========================
|
||||
|
||||
When a function or method with multiple overloads is called from Python,
|
||||
pybind11 determines which overload to call in two passes. The first pass
|
||||
attempts to call each overload without allowing argument conversion (as if
|
||||
every argument had been specified as ``py::arg().noconvert()`` as described
|
||||
above).
|
||||
|
||||
If no overload succeeds in the no-conversion first pass, a second pass is
|
||||
attempted in which argument conversion is allowed (except where prohibited via
|
||||
an explicit ``py::arg().noconvert()`` attribute in the function definition).
|
||||
|
||||
If the second pass also fails a ``TypeError`` is raised.
|
||||
|
||||
Within each pass, overloads are tried in the order they were registered with
|
||||
pybind11.
|
||||
|
||||
What this means in practice is that pybind11 will prefer any overload that does
|
||||
not require conversion of arguments to an overload that does, but otherwise prefers
|
||||
earlier-defined overloads to later-defined ones.
|
||||
|
||||
.. note::
|
||||
|
||||
pybind11 does *not* further prioritize based on the number/pattern of
|
||||
overloaded arguments. That is, pybind11 does not prioritize a function
|
||||
requiring one conversion over one requiring three, but only prioritizes
|
||||
overloads requiring no conversion at all to overloads that require
|
||||
conversion of at least one argument.
|
306
lib/bip158/lib/pybind11/docs/advanced/misc.rst
Normal file
306
lib/bip158/lib/pybind11/docs/advanced/misc.rst
Normal file
|
@ -0,0 +1,306 @@
|
|||
Miscellaneous
|
||||
#############
|
||||
|
||||
.. _macro_notes:
|
||||
|
||||
General notes regarding convenience macros
|
||||
==========================================
|
||||
|
||||
pybind11 provides a few convenience macros such as
|
||||
:func:`PYBIND11_DECLARE_HOLDER_TYPE` and ``PYBIND11_OVERLOAD_*``. Since these
|
||||
are "just" macros that are evaluated in the preprocessor (which has no concept
|
||||
of types), they *will* get confused by commas in a template argument; for
|
||||
example, consider:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_OVERLOAD(MyReturnType<T1, T2>, Class<T3, T4>, func)
|
||||
|
||||
The limitation of the C preprocessor interprets this as five arguments (with new
|
||||
arguments beginning after each comma) rather than three. To get around this,
|
||||
there are two alternatives: you can use a type alias, or you can wrap the type
|
||||
using the ``PYBIND11_TYPE`` macro:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Version 1: using a type alias
|
||||
using ReturnType = MyReturnType<T1, T2>;
|
||||
using ClassType = Class<T3, T4>;
|
||||
PYBIND11_OVERLOAD(ReturnType, ClassType, func);
|
||||
|
||||
// Version 2: using the PYBIND11_TYPE macro:
|
||||
PYBIND11_OVERLOAD(PYBIND11_TYPE(MyReturnType<T1, T2>),
|
||||
PYBIND11_TYPE(Class<T3, T4>), func)
|
||||
|
||||
The ``PYBIND11_MAKE_OPAQUE`` macro does *not* require the above workarounds.
|
||||
|
||||
.. _gil:
|
||||
|
||||
Global Interpreter Lock (GIL)
|
||||
=============================
|
||||
|
||||
When calling a C++ function from Python, the GIL is always held.
|
||||
The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be
|
||||
used to acquire and release the global interpreter lock in the body of a C++
|
||||
function call. In this way, long-running C++ code can be parallelized using
|
||||
multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this
|
||||
could be realized as follows (important changes highlighted):
|
||||
|
||||
.. code-block:: cpp
|
||||
:emphasize-lines: 8,9,31,32
|
||||
|
||||
class PyAnimal : public Animal {
|
||||
public:
|
||||
/* Inherit the constructors */
|
||||
using Animal::Animal;
|
||||
|
||||
/* Trampoline (need one for each virtual function) */
|
||||
std::string go(int n_times) {
|
||||
/* Acquire GIL before calling Python code */
|
||||
py::gil_scoped_acquire acquire;
|
||||
|
||||
PYBIND11_OVERLOAD_PURE(
|
||||
std::string, /* Return type */
|
||||
Animal, /* Parent class */
|
||||
go, /* Name of function */
|
||||
n_times /* Argument(s) */
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
py::class_<Animal, PyAnimal> animal(m, "Animal");
|
||||
animal
|
||||
.def(py::init<>())
|
||||
.def("go", &Animal::go);
|
||||
|
||||
py::class_<Dog>(m, "Dog", animal)
|
||||
.def(py::init<>());
|
||||
|
||||
m.def("call_go", [](Animal *animal) -> std::string {
|
||||
/* Release GIL before calling into (potentially long-running) C++ code */
|
||||
py::gil_scoped_release release;
|
||||
return call_go(animal);
|
||||
});
|
||||
}
|
||||
|
||||
The ``call_go`` wrapper can also be simplified using the `call_guard` policy
|
||||
(see :ref:`call_policies`) which yields the same result:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("call_go", &call_go, py::call_guard<py::gil_scoped_release>());
|
||||
|
||||
|
||||
Binding sequence data types, iterators, the slicing protocol, etc.
|
||||
==================================================================
|
||||
|
||||
Please refer to the supplemental example for details.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_sequences_and_iterators.cpp` contains a
|
||||
complete example that shows how to bind a sequence data type, including
|
||||
length queries (``__len__``), iterators (``__iter__``), the slicing
|
||||
protocol and other kinds of useful operations.
|
||||
|
||||
|
||||
Partitioning code over multiple extension modules
|
||||
=================================================
|
||||
|
||||
It's straightforward to split binding code over multiple extension modules,
|
||||
while referencing types that are declared elsewhere. Everything "just" works
|
||||
without any special precautions. One exception to this rule occurs when
|
||||
extending a type declared in another extension module. Recall the basic example
|
||||
from Section :ref:`inheritance`.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet> pet(m, "Pet");
|
||||
pet.def(py::init<const std::string &>())
|
||||
.def_readwrite("name", &Pet::name);
|
||||
|
||||
py::class_<Dog>(m, "Dog", pet /* <- specify parent */)
|
||||
.def(py::init<const std::string &>())
|
||||
.def("bark", &Dog::bark);
|
||||
|
||||
Suppose now that ``Pet`` bindings are defined in a module named ``basic``,
|
||||
whereas the ``Dog`` bindings are defined somewhere else. The challenge is of
|
||||
course that the variable ``pet`` is not available anymore though it is needed
|
||||
to indicate the inheritance relationship to the constructor of ``class_<Dog>``.
|
||||
However, it can be acquired as follows:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::object pet = (py::object) py::module::import("basic").attr("Pet");
|
||||
|
||||
py::class_<Dog>(m, "Dog", pet)
|
||||
.def(py::init<const std::string &>())
|
||||
.def("bark", &Dog::bark);
|
||||
|
||||
Alternatively, you can specify the base class as a template parameter option to
|
||||
``class_``, which performs an automated lookup of the corresponding Python
|
||||
type. Like the above code, however, this also requires invoking the ``import``
|
||||
function once to ensure that the pybind11 binding code of the module ``basic``
|
||||
has been executed:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::module::import("basic");
|
||||
|
||||
py::class_<Dog, Pet>(m, "Dog")
|
||||
.def(py::init<const std::string &>())
|
||||
.def("bark", &Dog::bark);
|
||||
|
||||
Naturally, both methods will fail when there are cyclic dependencies.
|
||||
|
||||
Note that pybind11 code compiled with hidden-by-default symbol visibility (e.g.
|
||||
via the command line flag ``-fvisibility=hidden`` on GCC/Clang), which is
|
||||
required for proper pybind11 functionality, can interfere with the ability to
|
||||
access types defined in another extension module. Working around this requires
|
||||
manually exporting types that are accessed by multiple extension modules;
|
||||
pybind11 provides a macro to do just this:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class PYBIND11_EXPORT Dog : public Animal {
|
||||
...
|
||||
};
|
||||
|
||||
Note also that it is possible (although would rarely be required) to share arbitrary
|
||||
C++ objects between extension modules at runtime. Internal library data is shared
|
||||
between modules using capsule machinery [#f6]_ which can be also utilized for
|
||||
storing, modifying and accessing user-defined data. Note that an extension module
|
||||
will "see" other extensions' data if and only if they were built with the same
|
||||
pybind11 version. Consider the following example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
auto data = (MyData *) py::get_shared_data("mydata");
|
||||
if (!data)
|
||||
data = (MyData *) py::set_shared_data("mydata", new MyData(42));
|
||||
|
||||
If the above snippet was used in several separately compiled extension modules,
|
||||
the first one to be imported would create a ``MyData`` instance and associate
|
||||
a ``"mydata"`` key with a pointer to it. Extensions that are imported later
|
||||
would be then able to access the data behind the same pointer.
|
||||
|
||||
.. [#f6] https://docs.python.org/3/extending/extending.html#using-capsules
|
||||
|
||||
Module Destructors
|
||||
==================
|
||||
|
||||
pybind11 does not provide an explicit mechanism to invoke cleanup code at
|
||||
module destruction time. In rare cases where such functionality is required, it
|
||||
is possible to emulate it using Python capsules or weak references with a
|
||||
destruction callback.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
auto cleanup_callback = []() {
|
||||
// perform cleanup here -- this function is called with the GIL held
|
||||
};
|
||||
|
||||
m.add_object("_cleanup", py::capsule(cleanup_callback));
|
||||
|
||||
This approach has the potential downside that instances of classes exposed
|
||||
within the module may still be alive when the cleanup callback is invoked
|
||||
(whether this is acceptable will generally depend on the application).
|
||||
|
||||
Alternatively, the capsule may also be stashed within a type object, which
|
||||
ensures that it not called before all instances of that type have been
|
||||
collected:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
auto cleanup_callback = []() { /* ... */ };
|
||||
m.attr("BaseClass").attr("_cleanup") = py::capsule(cleanup_callback);
|
||||
|
||||
Both approaches also expose a potentially dangerous ``_cleanup`` attribute in
|
||||
Python, which may be undesirable from an API standpoint (a premature explicit
|
||||
call from Python might lead to undefined behavior). Yet another approach that
|
||||
avoids this issue involves weak reference with a cleanup callback:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Register a callback function that is invoked when the BaseClass object is colelcted
|
||||
py::cpp_function cleanup_callback(
|
||||
[](py::handle weakref) {
|
||||
// perform cleanup here -- this function is called with the GIL held
|
||||
|
||||
weakref.dec_ref(); // release weak reference
|
||||
}
|
||||
);
|
||||
|
||||
// Create a weak reference with a cleanup callback and initially leak it
|
||||
(void) py::weakref(m.attr("BaseClass"), cleanup_callback).release();
|
||||
|
||||
.. note::
|
||||
|
||||
PyPy (at least version 5.9) does not garbage collect objects when the
|
||||
interpreter exits. An alternative approach (which also works on CPython) is to use
|
||||
the :py:mod:`atexit` module [#f7]_, for example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
auto atexit = py::module::import("atexit");
|
||||
atexit.attr("register")(py::cpp_function([]() {
|
||||
// perform cleanup here -- this function is called with the GIL held
|
||||
}));
|
||||
|
||||
.. [#f7] https://docs.python.org/3/library/atexit.html
|
||||
|
||||
|
||||
Generating documentation using Sphinx
|
||||
=====================================
|
||||
|
||||
Sphinx [#f4]_ has the ability to inspect the signatures and documentation
|
||||
strings in pybind11-based extension modules to automatically generate beautiful
|
||||
documentation in a variety formats. The python_example repository [#f5]_ contains a
|
||||
simple example repository which uses this approach.
|
||||
|
||||
There are two potential gotchas when using this approach: first, make sure that
|
||||
the resulting strings do not contain any :kbd:`TAB` characters, which break the
|
||||
docstring parsing routines. You may want to use C++11 raw string literals,
|
||||
which are convenient for multi-line comments. Conveniently, any excess
|
||||
indentation will be automatically be removed by Sphinx. However, for this to
|
||||
work, it is important that all lines are indented consistently, i.e.:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// ok
|
||||
m.def("foo", &foo, R"mydelimiter(
|
||||
The foo function
|
||||
|
||||
Parameters
|
||||
----------
|
||||
)mydelimiter");
|
||||
|
||||
// *not ok*
|
||||
m.def("foo", &foo, R"mydelimiter(The foo function
|
||||
|
||||
Parameters
|
||||
----------
|
||||
)mydelimiter");
|
||||
|
||||
By default, pybind11 automatically generates and prepends a signature to the docstring of a function
|
||||
registered with ``module::def()`` and ``class_::def()``. Sometimes this
|
||||
behavior is not desirable, because you want to provide your own signature or remove
|
||||
the docstring completely to exclude the function from the Sphinx documentation.
|
||||
The class ``options`` allows you to selectively suppress auto-generated signatures:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
py::options options;
|
||||
options.disable_function_signatures();
|
||||
|
||||
m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers");
|
||||
}
|
||||
|
||||
Note that changes to the settings affect only function bindings created during the
|
||||
lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function,
|
||||
the default settings are restored to prevent unwanted side effects.
|
||||
|
||||
.. [#f4] http://www.sphinx-doc.org
|
||||
.. [#f5] http://github.com/pybind/python_example
|
13
lib/bip158/lib/pybind11/docs/advanced/pycpp/index.rst
Normal file
13
lib/bip158/lib/pybind11/docs/advanced/pycpp/index.rst
Normal file
|
@ -0,0 +1,13 @@
|
|||
Python C++ interface
|
||||
####################
|
||||
|
||||
pybind11 exposes Python types and functions using thin C++ wrappers, which
|
||||
makes it possible to conveniently call Python code from C++ without resorting
|
||||
to Python's C API.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
object
|
||||
numpy
|
||||
utilities
|
386
lib/bip158/lib/pybind11/docs/advanced/pycpp/numpy.rst
Normal file
386
lib/bip158/lib/pybind11/docs/advanced/pycpp/numpy.rst
Normal file
|
@ -0,0 +1,386 @@
|
|||
.. _numpy:
|
||||
|
||||
NumPy
|
||||
#####
|
||||
|
||||
Buffer protocol
|
||||
===============
|
||||
|
||||
Python supports an extremely general and convenient approach for exchanging
|
||||
data between plugin libraries. Types can expose a buffer view [#f2]_, which
|
||||
provides fast direct access to the raw internal data representation. Suppose we
|
||||
want to bind the following simplistic Matrix class:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class Matrix {
|
||||
public:
|
||||
Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) {
|
||||
m_data = new float[rows*cols];
|
||||
}
|
||||
float *data() { return m_data; }
|
||||
size_t rows() const { return m_rows; }
|
||||
size_t cols() const { return m_cols; }
|
||||
private:
|
||||
size_t m_rows, m_cols;
|
||||
float *m_data;
|
||||
};
|
||||
|
||||
The following binding code exposes the ``Matrix`` contents as a buffer object,
|
||||
making it possible to cast Matrices into NumPy arrays. It is even possible to
|
||||
completely avoid copy operations with Python expressions like
|
||||
``np.array(matrix_instance, copy = False)``.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
|
||||
.def_buffer([](Matrix &m) -> py::buffer_info {
|
||||
return py::buffer_info(
|
||||
m.data(), /* Pointer to buffer */
|
||||
sizeof(float), /* Size of one scalar */
|
||||
py::format_descriptor<float>::format(), /* Python struct-style format descriptor */
|
||||
2, /* Number of dimensions */
|
||||
{ m.rows(), m.cols() }, /* Buffer dimensions */
|
||||
{ sizeof(float) * m.cols(), /* Strides (in bytes) for each index */
|
||||
sizeof(float) }
|
||||
);
|
||||
});
|
||||
|
||||
Supporting the buffer protocol in a new type involves specifying the special
|
||||
``py::buffer_protocol()`` tag in the ``py::class_`` constructor and calling the
|
||||
``def_buffer()`` method with a lambda function that creates a
|
||||
``py::buffer_info`` description record on demand describing a given matrix
|
||||
instance. The contents of ``py::buffer_info`` mirror the Python buffer protocol
|
||||
specification.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct buffer_info {
|
||||
void *ptr;
|
||||
ssize_t itemsize;
|
||||
std::string format;
|
||||
ssize_t ndim;
|
||||
std::vector<ssize_t> shape;
|
||||
std::vector<ssize_t> strides;
|
||||
};
|
||||
|
||||
To create a C++ function that can take a Python buffer object as an argument,
|
||||
simply use the type ``py::buffer`` as one of its arguments. Buffers can exist
|
||||
in a great variety of configurations, hence some safety checks are usually
|
||||
necessary in the function body. Below, you can see an basic example on how to
|
||||
define a custom constructor for the Eigen double precision matrix
|
||||
(``Eigen::MatrixXd``) type, which supports initialization from compatible
|
||||
buffer objects (e.g. a NumPy matrix).
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
/* Bind MatrixXd (or some other Eigen type) to Python */
|
||||
typedef Eigen::MatrixXd Matrix;
|
||||
|
||||
typedef Matrix::Scalar Scalar;
|
||||
constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
|
||||
|
||||
py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
|
||||
.def("__init__", [](Matrix &m, py::buffer b) {
|
||||
typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
|
||||
|
||||
/* Request a buffer descriptor from Python */
|
||||
py::buffer_info info = b.request();
|
||||
|
||||
/* Some sanity checks ... */
|
||||
if (info.format != py::format_descriptor<Scalar>::format())
|
||||
throw std::runtime_error("Incompatible format: expected a double array!");
|
||||
|
||||
if (info.ndim != 2)
|
||||
throw std::runtime_error("Incompatible buffer dimension!");
|
||||
|
||||
auto strides = Strides(
|
||||
info.strides[rowMajor ? 0 : 1] / (py::ssize_t)sizeof(Scalar),
|
||||
info.strides[rowMajor ? 1 : 0] / (py::ssize_t)sizeof(Scalar));
|
||||
|
||||
auto map = Eigen::Map<Matrix, 0, Strides>(
|
||||
static_cast<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides);
|
||||
|
||||
new (&m) Matrix(map);
|
||||
});
|
||||
|
||||
For reference, the ``def_buffer()`` call for this Eigen data type should look
|
||||
as follows:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
.def_buffer([](Matrix &m) -> py::buffer_info {
|
||||
return py::buffer_info(
|
||||
m.data(), /* Pointer to buffer */
|
||||
sizeof(Scalar), /* Size of one scalar */
|
||||
py::format_descriptor<Scalar>::format(), /* Python struct-style format descriptor */
|
||||
2, /* Number of dimensions */
|
||||
{ m.rows(), m.cols() }, /* Buffer dimensions */
|
||||
{ sizeof(Scalar) * (rowMajor ? m.cols() : 1),
|
||||
sizeof(Scalar) * (rowMajor ? 1 : m.rows()) }
|
||||
/* Strides (in bytes) for each index */
|
||||
);
|
||||
})
|
||||
|
||||
For a much easier approach of binding Eigen types (although with some
|
||||
limitations), refer to the section on :doc:`/advanced/cast/eigen`.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_buffers.cpp` contains a complete example
|
||||
that demonstrates using the buffer protocol with pybind11 in more detail.
|
||||
|
||||
.. [#f2] http://docs.python.org/3/c-api/buffer.html
|
||||
|
||||
Arrays
|
||||
======
|
||||
|
||||
By exchanging ``py::buffer`` with ``py::array`` in the above snippet, we can
|
||||
restrict the function so that it only accepts NumPy arrays (rather than any
|
||||
type of Python object satisfying the buffer protocol).
|
||||
|
||||
In many situations, we want to define a function which only accepts a NumPy
|
||||
array of a certain data type. This is possible via the ``py::array_t<T>``
|
||||
template. For instance, the following function requires the argument to be a
|
||||
NumPy array containing double precision values.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void f(py::array_t<double> array);
|
||||
|
||||
When it is invoked with a different type (e.g. an integer or a list of
|
||||
integers), the binding code will attempt to cast the input into a NumPy array
|
||||
of the requested type. Note that this feature requires the
|
||||
:file:`pybind11/numpy.h` header to be included.
|
||||
|
||||
Data in NumPy arrays is not guaranteed to packed in a dense manner;
|
||||
furthermore, entries can be separated by arbitrary column and row strides.
|
||||
Sometimes, it can be useful to require a function to only accept dense arrays
|
||||
using either the C (row-major) or Fortran (column-major) ordering. This can be
|
||||
accomplished via a second template argument with values ``py::array::c_style``
|
||||
or ``py::array::f_style``.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void f(py::array_t<double, py::array::c_style | py::array::forcecast> array);
|
||||
|
||||
The ``py::array::forcecast`` argument is the default value of the second
|
||||
template parameter, and it ensures that non-conforming arguments are converted
|
||||
into an array satisfying the specified requirements instead of trying the next
|
||||
function overload.
|
||||
|
||||
Structured types
|
||||
================
|
||||
|
||||
In order for ``py::array_t`` to work with structured (record) types, we first
|
||||
need to register the memory layout of the type. This can be done via
|
||||
``PYBIND11_NUMPY_DTYPE`` macro, called in the plugin definition code, which
|
||||
expects the type followed by field names:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct A {
|
||||
int x;
|
||||
double y;
|
||||
};
|
||||
|
||||
struct B {
|
||||
int z;
|
||||
A a;
|
||||
};
|
||||
|
||||
// ...
|
||||
PYBIND11_MODULE(test, m) {
|
||||
// ...
|
||||
|
||||
PYBIND11_NUMPY_DTYPE(A, x, y);
|
||||
PYBIND11_NUMPY_DTYPE(B, z, a);
|
||||
/* now both A and B can be used as template arguments to py::array_t */
|
||||
}
|
||||
|
||||
The structure should consist of fundamental arithmetic types, ``std::complex``,
|
||||
previously registered substructures, and arrays of any of the above. Both C++
|
||||
arrays and ``std::array`` are supported. While there is a static assertion to
|
||||
prevent many types of unsupported structures, it is still the user's
|
||||
responsibility to use only "plain" structures that can be safely manipulated as
|
||||
raw memory without violating invariants.
|
||||
|
||||
Vectorizing functions
|
||||
=====================
|
||||
|
||||
Suppose we want to bind a function with the following signature to Python so
|
||||
that it can process arbitrary NumPy array arguments (vectors, matrices, general
|
||||
N-D arrays) in addition to its normal arguments:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
double my_func(int x, float y, double z);
|
||||
|
||||
After including the ``pybind11/numpy.h`` header, this is extremely simple:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("vectorized_func", py::vectorize(my_func));
|
||||
|
||||
Invoking the function like below causes 4 calls to be made to ``my_func`` with
|
||||
each of the array elements. The significant advantage of this compared to
|
||||
solutions like ``numpy.vectorize()`` is that the loop over the elements runs
|
||||
entirely on the C++ side and can be crunched down into a tight, optimized loop
|
||||
by the compiler. The result is returned as a NumPy array of type
|
||||
``numpy.dtype.float64``.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> x = np.array([[1, 3],[5, 7]])
|
||||
>>> y = np.array([[2, 4],[6, 8]])
|
||||
>>> z = 3
|
||||
>>> result = vectorized_func(x, y, z)
|
||||
|
||||
The scalar argument ``z`` is transparently replicated 4 times. The input
|
||||
arrays ``x`` and ``y`` are automatically converted into the right types (they
|
||||
are of type ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and
|
||||
``numpy.dtype.float32``, respectively).
|
||||
|
||||
.. note::
|
||||
|
||||
Only arithmetic, complex, and POD types passed by value or by ``const &``
|
||||
reference are vectorized; all other arguments are passed through as-is.
|
||||
Functions taking rvalue reference arguments cannot be vectorized.
|
||||
|
||||
In cases where the computation is too complicated to be reduced to
|
||||
``vectorize``, it will be necessary to create and access the buffer contents
|
||||
manually. The following snippet contains a complete example that shows how this
|
||||
works (the code is somewhat contrived, since it could have been done more
|
||||
simply using ``vectorize``).
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/pybind11.h>
|
||||
#include <pybind11/numpy.h>
|
||||
|
||||
namespace py = pybind11;
|
||||
|
||||
py::array_t<double> add_arrays(py::array_t<double> input1, py::array_t<double> input2) {
|
||||
py::buffer_info buf1 = input1.request(), buf2 = input2.request();
|
||||
|
||||
if (buf1.ndim != 1 || buf2.ndim != 1)
|
||||
throw std::runtime_error("Number of dimensions must be one");
|
||||
|
||||
if (buf1.size != buf2.size)
|
||||
throw std::runtime_error("Input shapes must match");
|
||||
|
||||
/* No pointer is passed, so NumPy will allocate the buffer */
|
||||
auto result = py::array_t<double>(buf1.size);
|
||||
|
||||
py::buffer_info buf3 = result.request();
|
||||
|
||||
double *ptr1 = (double *) buf1.ptr,
|
||||
*ptr2 = (double *) buf2.ptr,
|
||||
*ptr3 = (double *) buf3.ptr;
|
||||
|
||||
for (size_t idx = 0; idx < buf1.shape[0]; idx++)
|
||||
ptr3[idx] = ptr1[idx] + ptr2[idx];
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
PYBIND11_MODULE(test, m) {
|
||||
m.def("add_arrays", &add_arrays, "Add two NumPy arrays");
|
||||
}
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_numpy_vectorize.cpp` contains a complete
|
||||
example that demonstrates using :func:`vectorize` in more detail.
|
||||
|
||||
Direct access
|
||||
=============
|
||||
|
||||
For performance reasons, particularly when dealing with very large arrays, it
|
||||
is often desirable to directly access array elements without internal checking
|
||||
of dimensions and bounds on every access when indices are known to be already
|
||||
valid. To avoid such checks, the ``array`` class and ``array_t<T>`` template
|
||||
class offer an unchecked proxy object that can be used for this unchecked
|
||||
access through the ``unchecked<N>`` and ``mutable_unchecked<N>`` methods,
|
||||
where ``N`` gives the required dimensionality of the array:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("sum_3d", [](py::array_t<double> x) {
|
||||
auto r = x.unchecked<3>(); // x must have ndim = 3; can be non-writeable
|
||||
double sum = 0;
|
||||
for (ssize_t i = 0; i < r.shape(0); i++)
|
||||
for (ssize_t j = 0; j < r.shape(1); j++)
|
||||
for (ssize_t k = 0; k < r.shape(2); k++)
|
||||
sum += r(i, j, k);
|
||||
return sum;
|
||||
});
|
||||
m.def("increment_3d", [](py::array_t<double> x) {
|
||||
auto r = x.mutable_unchecked<3>(); // Will throw if ndim != 3 or flags.writeable is false
|
||||
for (ssize_t i = 0; i < r.shape(0); i++)
|
||||
for (ssize_t j = 0; j < r.shape(1); j++)
|
||||
for (ssize_t k = 0; k < r.shape(2); k++)
|
||||
r(i, j, k) += 1.0;
|
||||
}, py::arg().noconvert());
|
||||
|
||||
To obtain the proxy from an ``array`` object, you must specify both the data
|
||||
type and number of dimensions as template arguments, such as ``auto r =
|
||||
myarray.mutable_unchecked<float, 2>()``.
|
||||
|
||||
If the number of dimensions is not known at compile time, you can omit the
|
||||
dimensions template parameter (i.e. calling ``arr_t.unchecked()`` or
|
||||
``arr.unchecked<T>()``. This will give you a proxy object that works in the
|
||||
same way, but results in less optimizable code and thus a small efficiency
|
||||
loss in tight loops.
|
||||
|
||||
Note that the returned proxy object directly references the array's data, and
|
||||
only reads its shape, strides, and writeable flag when constructed. You must
|
||||
take care to ensure that the referenced array is not destroyed or reshaped for
|
||||
the duration of the returned object, typically by limiting the scope of the
|
||||
returned instance.
|
||||
|
||||
The returned proxy object supports some of the same methods as ``py::array`` so
|
||||
that it can be used as a drop-in replacement for some existing, index-checked
|
||||
uses of ``py::array``:
|
||||
|
||||
- ``r.ndim()`` returns the number of dimensions
|
||||
|
||||
- ``r.data(1, 2, ...)`` and ``r.mutable_data(1, 2, ...)``` returns a pointer to
|
||||
the ``const T`` or ``T`` data, respectively, at the given indices. The
|
||||
latter is only available to proxies obtained via ``a.mutable_unchecked()``.
|
||||
|
||||
- ``itemsize()`` returns the size of an item in bytes, i.e. ``sizeof(T)``.
|
||||
|
||||
- ``ndim()`` returns the number of dimensions.
|
||||
|
||||
- ``shape(n)`` returns the size of dimension ``n``
|
||||
|
||||
- ``size()`` returns the total number of elements (i.e. the product of the shapes).
|
||||
|
||||
- ``nbytes()`` returns the number of bytes used by the referenced elements
|
||||
(i.e. ``itemsize()`` times ``size()``).
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_numpy_array.cpp` contains additional examples
|
||||
demonstrating the use of this feature.
|
||||
|
||||
Ellipsis
|
||||
========
|
||||
|
||||
Python 3 provides a convenient ``...`` ellipsis notation that is often used to
|
||||
slice multidimensional arrays. For instance, the following snippet extracts the
|
||||
middle dimensions of a tensor with the first and last index set to zero.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
a = # a NumPy array
|
||||
b = a[0, ..., 0]
|
||||
|
||||
The function ``py::ellipsis()`` function can be used to perform the same
|
||||
operation on the C++ side:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::array a = /* A NumPy array */;
|
||||
py::array b = a[py::make_tuple(0, py::ellipsis(), 0)];
|
170
lib/bip158/lib/pybind11/docs/advanced/pycpp/object.rst
Normal file
170
lib/bip158/lib/pybind11/docs/advanced/pycpp/object.rst
Normal file
|
@ -0,0 +1,170 @@
|
|||
Python types
|
||||
############
|
||||
|
||||
Available wrappers
|
||||
==================
|
||||
|
||||
All major Python types are available as thin C++ wrapper classes. These
|
||||
can also be used as function parameters -- see :ref:`python_objects_as_args`.
|
||||
|
||||
Available types include :class:`handle`, :class:`object`, :class:`bool_`,
|
||||
:class:`int_`, :class:`float_`, :class:`str`, :class:`bytes`, :class:`tuple`,
|
||||
:class:`list`, :class:`dict`, :class:`slice`, :class:`none`, :class:`capsule`,
|
||||
:class:`iterable`, :class:`iterator`, :class:`function`, :class:`buffer`,
|
||||
:class:`array`, and :class:`array_t`.
|
||||
|
||||
Casting back and forth
|
||||
======================
|
||||
|
||||
In this kind of mixed code, it is often necessary to convert arbitrary C++
|
||||
types to Python, which can be done using :func:`py::cast`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
MyClass *cls = ..;
|
||||
py::object obj = py::cast(cls);
|
||||
|
||||
The reverse direction uses the following syntax:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::object obj = ...;
|
||||
MyClass *cls = obj.cast<MyClass *>();
|
||||
|
||||
When conversion fails, both directions throw the exception :class:`cast_error`.
|
||||
|
||||
.. _python_libs:
|
||||
|
||||
Accessing Python libraries from C++
|
||||
===================================
|
||||
|
||||
It is also possible to import objects defined in the Python standard
|
||||
library or available in the current Python environment (``sys.path``) and work
|
||||
with these in C++.
|
||||
|
||||
This example obtains a reference to the Python ``Decimal`` class.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Equivalent to "from decimal import Decimal"
|
||||
py::object Decimal = py::module::import("decimal").attr("Decimal");
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Try to import scipy
|
||||
py::object scipy = py::module::import("scipy");
|
||||
return scipy.attr("__version__");
|
||||
|
||||
.. _calling_python_functions:
|
||||
|
||||
Calling Python functions
|
||||
========================
|
||||
|
||||
It is also possible to call Python classes, functions and methods
|
||||
via ``operator()``.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Construct a Python object of class Decimal
|
||||
py::object pi = Decimal("3.14159");
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Use Python to make our directories
|
||||
py::object os = py::module::import("os");
|
||||
py::object makedirs = os.attr("makedirs");
|
||||
makedirs("/tmp/path/to/somewhere");
|
||||
|
||||
One can convert the result obtained from Python to a pure C++ version
|
||||
if a ``py::class_`` or type conversion is defined.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::function f = <...>;
|
||||
py::object result_py = f(1234, "hello", some_instance);
|
||||
MyClass &result = result_py.cast<MyClass>();
|
||||
|
||||
.. _calling_python_methods:
|
||||
|
||||
Calling Python methods
|
||||
========================
|
||||
|
||||
To call an object's method, one can again use ``.attr`` to obtain access to the
|
||||
Python method.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Calculate e^π in decimal
|
||||
py::object exp_pi = pi.attr("exp")();
|
||||
py::print(py::str(exp_pi));
|
||||
|
||||
In the example above ``pi.attr("exp")`` is a *bound method*: it will always call
|
||||
the method for that same instance of the class. Alternately one can create an
|
||||
*unbound method* via the Python class (instead of instance) and pass the ``self``
|
||||
object explicitly, followed by other arguments.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::object decimal_exp = Decimal.attr("exp");
|
||||
|
||||
// Compute the e^n for n=0..4
|
||||
for (int n = 0; n < 5; n++) {
|
||||
py::print(decimal_exp(Decimal(n));
|
||||
}
|
||||
|
||||
Keyword arguments
|
||||
=================
|
||||
|
||||
Keyword arguments are also supported. In Python, there is the usual call syntax:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def f(number, say, to):
|
||||
... # function code
|
||||
|
||||
f(1234, say="hello", to=some_instance) # keyword call in Python
|
||||
|
||||
In C++, the same call can be made using:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
using namespace pybind11::literals; // to bring in the `_a` literal
|
||||
f(1234, "say"_a="hello", "to"_a=some_instance); // keyword call in C++
|
||||
|
||||
Unpacking arguments
|
||||
===================
|
||||
|
||||
Unpacking of ``*args`` and ``**kwargs`` is also possible and can be mixed with
|
||||
other arguments:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// * unpacking
|
||||
py::tuple args = py::make_tuple(1234, "hello", some_instance);
|
||||
f(*args);
|
||||
|
||||
// ** unpacking
|
||||
py::dict kwargs = py::dict("number"_a=1234, "say"_a="hello", "to"_a=some_instance);
|
||||
f(**kwargs);
|
||||
|
||||
// mixed keywords, * and ** unpacking
|
||||
py::tuple args = py::make_tuple(1234);
|
||||
py::dict kwargs = py::dict("to"_a=some_instance);
|
||||
f(*args, "say"_a="hello", **kwargs);
|
||||
|
||||
Generalized unpacking according to PEP448_ is also supported:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::dict kwargs1 = py::dict("number"_a=1234);
|
||||
py::dict kwargs2 = py::dict("to"_a=some_instance);
|
||||
f(**kwargs1, "say"_a="hello", **kwargs2);
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_pytypes.cpp` contains a complete
|
||||
example that demonstrates passing native Python types in more detail. The
|
||||
file :file:`tests/test_callbacks.cpp` presents a few examples of calling
|
||||
Python functions from C++, including keywords arguments and unpacking.
|
||||
|
||||
.. _PEP448: https://www.python.org/dev/peps/pep-0448/
|
144
lib/bip158/lib/pybind11/docs/advanced/pycpp/utilities.rst
Normal file
144
lib/bip158/lib/pybind11/docs/advanced/pycpp/utilities.rst
Normal file
|
@ -0,0 +1,144 @@
|
|||
Utilities
|
||||
#########
|
||||
|
||||
Using Python's print function in C++
|
||||
====================================
|
||||
|
||||
The usual way to write output in C++ is using ``std::cout`` while in Python one
|
||||
would use ``print``. Since these methods use different buffers, mixing them can
|
||||
lead to output order issues. To resolve this, pybind11 modules can use the
|
||||
:func:`py::print` function which writes to Python's ``sys.stdout`` for consistency.
|
||||
|
||||
Python's ``print`` function is replicated in the C++ API including optional
|
||||
keyword arguments ``sep``, ``end``, ``file``, ``flush``. Everything works as
|
||||
expected in Python:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::print(1, 2.0, "three"); // 1 2.0 three
|
||||
py::print(1, 2.0, "three", "sep"_a="-"); // 1-2.0-three
|
||||
|
||||
auto args = py::make_tuple("unpacked", true);
|
||||
py::print("->", *args, "end"_a="<-"); // -> unpacked True <-
|
||||
|
||||
.. _ostream_redirect:
|
||||
|
||||
Capturing standard output from ostream
|
||||
======================================
|
||||
|
||||
Often, a library will use the streams ``std::cout`` and ``std::cerr`` to print,
|
||||
but this does not play well with Python's standard ``sys.stdout`` and ``sys.stderr``
|
||||
redirection. Replacing a library's printing with `py::print <print>` may not
|
||||
be feasible. This can be fixed using a guard around the library function that
|
||||
redirects output to the corresponding Python streams:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/iostream.h>
|
||||
|
||||
...
|
||||
|
||||
// Add a scoped redirect for your noisy code
|
||||
m.def("noisy_func", []() {
|
||||
py::scoped_ostream_redirect stream(
|
||||
std::cout, // std::ostream&
|
||||
py::module::import("sys").attr("stdout") // Python output
|
||||
);
|
||||
call_noisy_func();
|
||||
});
|
||||
|
||||
This method respects flushes on the output streams and will flush if needed
|
||||
when the scoped guard is destroyed. This allows the output to be redirected in
|
||||
real time, such as to a Jupyter notebook. The two arguments, the C++ stream and
|
||||
the Python output, are optional, and default to standard output if not given. An
|
||||
extra type, `py::scoped_estream_redirect <scoped_estream_redirect>`, is identical
|
||||
except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful with
|
||||
`py::call_guard`, which allows multiple items, but uses the default constructor:
|
||||
|
||||
.. code-block:: py
|
||||
|
||||
// Alternative: Call single function using call guard
|
||||
m.def("noisy_func", &call_noisy_function,
|
||||
py::call_guard<py::scoped_ostream_redirect,
|
||||
py::scoped_estream_redirect>());
|
||||
|
||||
The redirection can also be done in Python with the addition of a context
|
||||
manager, using the `py::add_ostream_redirect() <add_ostream_redirect>` function:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::add_ostream_redirect(m, "ostream_redirect");
|
||||
|
||||
The name in Python defaults to ``ostream_redirect`` if no name is passed. This
|
||||
creates the following context manager in Python:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with ostream_redirect(stdout=True, stderr=True):
|
||||
noisy_function()
|
||||
|
||||
It defaults to redirecting both streams, though you can use the keyword
|
||||
arguments to disable one of the streams if needed.
|
||||
|
||||
.. note::
|
||||
|
||||
The above methods will not redirect C-level output to file descriptors, such
|
||||
as ``fprintf``. For those cases, you'll need to redirect the file
|
||||
descriptors either directly in C or with Python's ``os.dup2`` function
|
||||
in an operating-system dependent way.
|
||||
|
||||
.. _eval:
|
||||
|
||||
Evaluating Python expressions from strings and files
|
||||
====================================================
|
||||
|
||||
pybind11 provides the `eval`, `exec` and `eval_file` functions to evaluate
|
||||
Python expressions and statements. The following example illustrates how they
|
||||
can be used.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// At beginning of file
|
||||
#include <pybind11/eval.h>
|
||||
|
||||
...
|
||||
|
||||
// Evaluate in scope of main module
|
||||
py::object scope = py::module::import("__main__").attr("__dict__");
|
||||
|
||||
// Evaluate an isolated expression
|
||||
int result = py::eval("my_variable + 10", scope).cast<int>();
|
||||
|
||||
// Evaluate a sequence of statements
|
||||
py::exec(
|
||||
"print('Hello')\n"
|
||||
"print('world!');",
|
||||
scope);
|
||||
|
||||
// Evaluate the statements in an separate Python file on disk
|
||||
py::eval_file("script.py", scope);
|
||||
|
||||
C++11 raw string literals are also supported and quite handy for this purpose.
|
||||
The only requirement is that the first statement must be on a new line following
|
||||
the raw string delimiter ``R"(``, ensuring all lines have common leading indent:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::exec(R"(
|
||||
x = get_answer()
|
||||
if x == 42:
|
||||
print('Hello World!')
|
||||
else:
|
||||
print('Bye!')
|
||||
)", scope
|
||||
);
|
||||
|
||||
.. note::
|
||||
|
||||
`eval` and `eval_file` accept a template parameter that describes how the
|
||||
string/file should be interpreted. Possible choices include ``eval_expr``
|
||||
(isolated expression), ``eval_single_statement`` (a single statement, return
|
||||
value is always ``none``), and ``eval_statements`` (sequence of statements,
|
||||
return value is always ``none``). `eval` defaults to ``eval_expr``,
|
||||
`eval_file` defaults to ``eval_statements`` and `exec` is just a shortcut
|
||||
for ``eval<eval_statements>``.
|
173
lib/bip158/lib/pybind11/docs/advanced/smart_ptrs.rst
Normal file
173
lib/bip158/lib/pybind11/docs/advanced/smart_ptrs.rst
Normal file
|
@ -0,0 +1,173 @@
|
|||
Smart pointers
|
||||
##############
|
||||
|
||||
std::unique_ptr
|
||||
===============
|
||||
|
||||
Given a class ``Example`` with Python bindings, it's possible to return
|
||||
instances wrapped in C++11 unique pointers, like so
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
std::unique_ptr<Example> create_example() { return std::unique_ptr<Example>(new Example()); }
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("create_example", &create_example);
|
||||
|
||||
In other words, there is nothing special that needs to be done. While returning
|
||||
unique pointers in this way is allowed, it is *illegal* to use them as function
|
||||
arguments. For instance, the following function signature cannot be processed
|
||||
by pybind11.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void do_something_with_example(std::unique_ptr<Example> ex) { ... }
|
||||
|
||||
The above signature would imply that Python needs to give up ownership of an
|
||||
object that is passed to this function, which is generally not possible (for
|
||||
instance, the object might be referenced elsewhere).
|
||||
|
||||
std::shared_ptr
|
||||
===============
|
||||
|
||||
The binding generator for classes, :class:`class_`, can be passed a template
|
||||
type that denotes a special *holder* type that is used to manage references to
|
||||
the object. If no such holder type template argument is given, the default for
|
||||
a type named ``Type`` is ``std::unique_ptr<Type>``, which means that the object
|
||||
is deallocated when Python's reference count goes to zero.
|
||||
|
||||
It is possible to switch to other types of reference counting wrappers or smart
|
||||
pointers, which is useful in codebases that rely on them. For instance, the
|
||||
following snippet causes ``std::shared_ptr`` to be used instead.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
|
||||
|
||||
Note that any particular class can only be associated with a single holder type.
|
||||
|
||||
One potential stumbling block when using holder types is that they need to be
|
||||
applied consistently. Can you guess what's broken about the following binding
|
||||
code?
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class Child { };
|
||||
|
||||
class Parent {
|
||||
public:
|
||||
Parent() : child(std::make_shared<Child>()) { }
|
||||
Child *get_child() { return child.get(); } /* Hint: ** DON'T DO THIS ** */
|
||||
private:
|
||||
std::shared_ptr<Child> child;
|
||||
};
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
py::class_<Child, std::shared_ptr<Child>>(m, "Child");
|
||||
|
||||
py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
|
||||
.def(py::init<>())
|
||||
.def("get_child", &Parent::get_child);
|
||||
}
|
||||
|
||||
The following Python code will cause undefined behavior (and likely a
|
||||
segmentation fault).
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from example import Parent
|
||||
print(Parent().get_child())
|
||||
|
||||
The problem is that ``Parent::get_child()`` returns a pointer to an instance of
|
||||
``Child``, but the fact that this instance is already managed by
|
||||
``std::shared_ptr<...>`` is lost when passing raw pointers. In this case,
|
||||
pybind11 will create a second independent ``std::shared_ptr<...>`` that also
|
||||
claims ownership of the pointer. In the end, the object will be freed **twice**
|
||||
since these shared pointers have no way of knowing about each other.
|
||||
|
||||
There are two ways to resolve this issue:
|
||||
|
||||
1. For types that are managed by a smart pointer class, never use raw pointers
|
||||
in function arguments or return values. In other words: always consistently
|
||||
wrap pointers into their designated holder types (such as
|
||||
``std::shared_ptr<...>``). In this case, the signature of ``get_child()``
|
||||
should be modified as follows:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
std::shared_ptr<Child> get_child() { return child; }
|
||||
|
||||
2. Adjust the definition of ``Child`` by specifying
|
||||
``std::enable_shared_from_this<T>`` (see cppreference_ for details) as a
|
||||
base class. This adds a small bit of information to ``Child`` that allows
|
||||
pybind11 to realize that there is already an existing
|
||||
``std::shared_ptr<...>`` and communicate with it. In this case, the
|
||||
declaration of ``Child`` should look as follows:
|
||||
|
||||
.. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class Child : public std::enable_shared_from_this<Child> { };
|
||||
|
||||
.. _smart_pointers:
|
||||
|
||||
Custom smart pointers
|
||||
=====================
|
||||
|
||||
pybind11 supports ``std::unique_ptr`` and ``std::shared_ptr`` right out of the
|
||||
box. For any other custom smart pointer, transparent conversions can be enabled
|
||||
using a macro invocation similar to the following. It must be declared at the
|
||||
top namespace level before any binding code:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>);
|
||||
|
||||
The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
|
||||
placeholder name that is used as a template parameter of the second argument.
|
||||
Thus, feel free to use any identifier, but use it consistently on both sides;
|
||||
also, don't use the name of a type that already exists in your codebase.
|
||||
|
||||
The macro also accepts a third optional boolean parameter that is set to false
|
||||
by default. Specify
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>, true);
|
||||
|
||||
if ``SmartPtr<T>`` can always be initialized from a ``T*`` pointer without the
|
||||
risk of inconsistencies (such as multiple independent ``SmartPtr`` instances
|
||||
believing that they are the sole owner of the ``T*`` pointer). A common
|
||||
situation where ``true`` should be passed is when the ``T`` instances use
|
||||
*intrusive* reference counting.
|
||||
|
||||
Please take a look at the :ref:`macro_notes` before using this feature.
|
||||
|
||||
By default, pybind11 assumes that your custom smart pointer has a standard
|
||||
interface, i.e. provides a ``.get()`` member function to access the underlying
|
||||
raw pointer. If this is not the case, pybind11's ``holder_helper`` must be
|
||||
specialized:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Always needed for custom holder types
|
||||
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>);
|
||||
|
||||
// Only needed if the type's `.get()` goes by another name
|
||||
namespace pybind11 { namespace detail {
|
||||
template <typename T>
|
||||
struct holder_helper<SmartPtr<T>> { // <-- specialization
|
||||
static const T *get(const SmartPtr<T> &p) { return p.getPointer(); }
|
||||
};
|
||||
}}
|
||||
|
||||
The above specialization informs pybind11 that the custom ``SmartPtr`` class
|
||||
provides ``.get()`` functionality via ``.getPointer()``.
|
||||
|
||||
.. seealso::
|
||||
|
||||
The file :file:`tests/test_smart_ptr.cpp` contains a complete example
|
||||
that demonstrates how to work with custom reference-counting holder types
|
||||
in more detail.
|
293
lib/bip158/lib/pybind11/docs/basics.rst
Normal file
293
lib/bip158/lib/pybind11/docs/basics.rst
Normal file
|
@ -0,0 +1,293 @@
|
|||
.. _basics:
|
||||
|
||||
First steps
|
||||
###########
|
||||
|
||||
This sections demonstrates the basic features of pybind11. Before getting
|
||||
started, make sure that development environment is set up to compile the
|
||||
included set of test cases.
|
||||
|
||||
|
||||
Compiling the test cases
|
||||
========================
|
||||
|
||||
Linux/MacOS
|
||||
-----------
|
||||
|
||||
On Linux you'll need to install the **python-dev** or **python3-dev** packages as
|
||||
well as **cmake**. On Mac OS, the included python version works out of the box,
|
||||
but **cmake** must still be installed.
|
||||
|
||||
After installing the prerequisites, run
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make check -j 4
|
||||
|
||||
The last line will both compile and run the tests.
|
||||
|
||||
Windows
|
||||
-------
|
||||
|
||||
On Windows, only **Visual Studio 2015** and newer are supported since pybind11 relies
|
||||
on various C++11 language features that break older versions of Visual Studio.
|
||||
|
||||
To compile and run the tests:
|
||||
|
||||
.. code-block:: batch
|
||||
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
cmake --build . --config Release --target check
|
||||
|
||||
This will create a Visual Studio project, compile and run the target, all from the
|
||||
command line.
|
||||
|
||||
.. Note::
|
||||
|
||||
If all tests fail, make sure that the Python binary and the testcases are compiled
|
||||
for the same processor type and bitness (i.e. either **i386** or **x86_64**). You
|
||||
can specify **x86_64** as the target architecture for the generated Visual Studio
|
||||
project using ``cmake -A x64 ..``.
|
||||
|
||||
.. seealso::
|
||||
|
||||
Advanced users who are already familiar with Boost.Python may want to skip
|
||||
the tutorial and look at the test cases in the :file:`tests` directory,
|
||||
which exercise all features of pybind11.
|
||||
|
||||
Header and namespace conventions
|
||||
================================
|
||||
|
||||
For brevity, all code examples assume that the following two lines are present:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/pybind11.h>
|
||||
|
||||
namespace py = pybind11;
|
||||
|
||||
Some features may require additional headers, but those will be specified as needed.
|
||||
|
||||
.. _simple_example:
|
||||
|
||||
Creating bindings for a simple function
|
||||
=======================================
|
||||
|
||||
Let's start by creating Python bindings for an extremely simple function, which
|
||||
adds two numbers and returns their result:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
int add(int i, int j) {
|
||||
return i + j;
|
||||
}
|
||||
|
||||
For simplicity [#f1]_, we'll put both this function and the binding code into
|
||||
a file named :file:`example.cpp` with the following contents:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/pybind11.h>
|
||||
|
||||
int add(int i, int j) {
|
||||
return i + j;
|
||||
}
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
m.doc() = "pybind11 example plugin"; // optional module docstring
|
||||
|
||||
m.def("add", &add, "A function which adds two numbers");
|
||||
}
|
||||
|
||||
.. [#f1] In practice, implementation and binding code will generally be located
|
||||
in separate files.
|
||||
|
||||
The :func:`PYBIND11_MODULE` macro creates a function that will be called when an
|
||||
``import`` statement is issued from within Python. The module name (``example``)
|
||||
is given as the first macro argument (it should not be in quotes). The second
|
||||
argument (``m``) defines a variable of type :class:`py::module <module>` which
|
||||
is the main interface for creating bindings. The method :func:`module::def`
|
||||
generates binding code that exposes the ``add()`` function to Python.
|
||||
|
||||
.. note::
|
||||
|
||||
Notice how little code was needed to expose our function to Python: all
|
||||
details regarding the function's parameters and return value were
|
||||
automatically inferred using template metaprogramming. This overall
|
||||
approach and the used syntax are borrowed from Boost.Python, though the
|
||||
underlying implementation is very different.
|
||||
|
||||
pybind11 is a header-only library, hence it is not necessary to link against
|
||||
any special libraries and there are no intermediate (magic) translation steps.
|
||||
On Linux, the above example can be compiled using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
|
||||
|
||||
For more details on the required compiler flags on Linux and MacOS, see
|
||||
:ref:`building_manually`. For complete cross-platform compilation instructions,
|
||||
refer to the :ref:`compiling` page.
|
||||
|
||||
The `python_example`_ and `cmake_example`_ repositories are also a good place
|
||||
to start. They are both complete project examples with cross-platform build
|
||||
systems. The only difference between the two is that `python_example`_ uses
|
||||
Python's ``setuptools`` to build the module, while `cmake_example`_ uses CMake
|
||||
(which may be preferable for existing C++ projects).
|
||||
|
||||
.. _python_example: https://github.com/pybind/python_example
|
||||
.. _cmake_example: https://github.com/pybind/cmake_example
|
||||
|
||||
Building the above C++ code will produce a binary module file that can be
|
||||
imported to Python. Assuming that the compiled module is located in the
|
||||
current directory, the following interactive Python session shows how to
|
||||
load and execute the example:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
$ python
|
||||
Python 2.7.10 (default, Aug 22 2015, 20:33:39)
|
||||
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1)] on darwin
|
||||
Type "help", "copyright", "credits" or "license" for more information.
|
||||
>>> import example
|
||||
>>> example.add(1, 2)
|
||||
3L
|
||||
>>>
|
||||
|
||||
.. _keyword_args:
|
||||
|
||||
Keyword arguments
|
||||
=================
|
||||
|
||||
With a simple code modification, it is possible to inform Python about the
|
||||
names of the arguments ("i" and "j" in this case).
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("add", &add, "A function which adds two numbers",
|
||||
py::arg("i"), py::arg("j"));
|
||||
|
||||
:class:`arg` is one of several special tag classes which can be used to pass
|
||||
metadata into :func:`module::def`. With this modified binding code, we can now
|
||||
call the function using keyword arguments, which is a more readable alternative
|
||||
particularly for functions taking many parameters:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import example
|
||||
>>> example.add(i=1, j=2)
|
||||
3L
|
||||
|
||||
The keyword names also appear in the function signatures within the documentation.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> help(example)
|
||||
|
||||
....
|
||||
|
||||
FUNCTIONS
|
||||
add(...)
|
||||
Signature : (i: int, j: int) -> int
|
||||
|
||||
A function which adds two numbers
|
||||
|
||||
A shorter notation for named arguments is also available:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// regular notation
|
||||
m.def("add1", &add, py::arg("i"), py::arg("j"));
|
||||
// shorthand
|
||||
using namespace pybind11::literals;
|
||||
m.def("add2", &add, "i"_a, "j"_a);
|
||||
|
||||
The :var:`_a` suffix forms a C++11 literal which is equivalent to :class:`arg`.
|
||||
Note that the literal operator must first be made visible with the directive
|
||||
``using namespace pybind11::literals``. This does not bring in anything else
|
||||
from the ``pybind11`` namespace except for literals.
|
||||
|
||||
.. _default_args:
|
||||
|
||||
Default arguments
|
||||
=================
|
||||
|
||||
Suppose now that the function to be bound has default arguments, e.g.:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
int add(int i = 1, int j = 2) {
|
||||
return i + j;
|
||||
}
|
||||
|
||||
Unfortunately, pybind11 cannot automatically extract these parameters, since they
|
||||
are not part of the function's type information. However, they are simple to specify
|
||||
using an extension of :class:`arg`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("add", &add, "A function which adds two numbers",
|
||||
py::arg("i") = 1, py::arg("j") = 2);
|
||||
|
||||
The default values also appear within the documentation.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> help(example)
|
||||
|
||||
....
|
||||
|
||||
FUNCTIONS
|
||||
add(...)
|
||||
Signature : (i: int = 1, j: int = 2) -> int
|
||||
|
||||
A function which adds two numbers
|
||||
|
||||
The shorthand notation is also available for default arguments:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// regular notation
|
||||
m.def("add1", &add, py::arg("i") = 1, py::arg("j") = 2);
|
||||
// shorthand
|
||||
m.def("add2", &add, "i"_a=1, "j"_a=2);
|
||||
|
||||
Exporting variables
|
||||
===================
|
||||
|
||||
To expose a value from C++, use the ``attr`` function to register it in a
|
||||
module as shown below. Built-in types and general objects (more on that later)
|
||||
are automatically converted when assigned as attributes, and can be explicitly
|
||||
converted using the function ``py::cast``.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
m.attr("the_answer") = 42;
|
||||
py::object world = py::cast("World");
|
||||
m.attr("what") = world;
|
||||
}
|
||||
|
||||
These are then accessible from Python:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import example
|
||||
>>> example.the_answer
|
||||
42
|
||||
>>> example.what
|
||||
'World'
|
||||
|
||||
.. _supported_types:
|
||||
|
||||
Supported data types
|
||||
====================
|
||||
|
||||
A large number of data types are supported out of the box and can be used
|
||||
seamlessly as functions arguments, return values or with ``py::cast`` in general.
|
||||
For a full overview, see the :doc:`advanced/cast/index` section.
|
88
lib/bip158/lib/pybind11/docs/benchmark.py
Normal file
88
lib/bip158/lib/pybind11/docs/benchmark.py
Normal file
|
@ -0,0 +1,88 @@
|
|||
import random
|
||||
import os
|
||||
import time
|
||||
import datetime as dt
|
||||
|
||||
nfns = 4 # Functions per class
|
||||
nargs = 4 # Arguments per function
|
||||
|
||||
|
||||
def generate_dummy_code_pybind11(nclasses=10):
|
||||
decl = ""
|
||||
bindings = ""
|
||||
|
||||
for cl in range(nclasses):
|
||||
decl += "class cl%03i;\n" % cl
|
||||
decl += '\n'
|
||||
|
||||
for cl in range(nclasses):
|
||||
decl += "class cl%03i {\n" % cl
|
||||
decl += "public:\n"
|
||||
bindings += ' py::class_<cl%03i>(m, "cl%03i")\n' % (cl, cl)
|
||||
for fn in range(nfns):
|
||||
ret = random.randint(0, nclasses - 1)
|
||||
params = [random.randint(0, nclasses - 1) for i in range(nargs)]
|
||||
decl += " cl%03i *fn_%03i(" % (ret, fn)
|
||||
decl += ", ".join("cl%03i *" % p for p in params)
|
||||
decl += ");\n"
|
||||
bindings += ' .def("fn_%03i", &cl%03i::fn_%03i)\n' % \
|
||||
(fn, cl, fn)
|
||||
decl += "};\n\n"
|
||||
bindings += ' ;\n'
|
||||
|
||||
result = "#include <pybind11/pybind11.h>\n\n"
|
||||
result += "namespace py = pybind11;\n\n"
|
||||
result += decl + '\n'
|
||||
result += "PYBIND11_MODULE(example, m) {\n"
|
||||
result += bindings
|
||||
result += "}"
|
||||
return result
|
||||
|
||||
|
||||
def generate_dummy_code_boost(nclasses=10):
|
||||
decl = ""
|
||||
bindings = ""
|
||||
|
||||
for cl in range(nclasses):
|
||||
decl += "class cl%03i;\n" % cl
|
||||
decl += '\n'
|
||||
|
||||
for cl in range(nclasses):
|
||||
decl += "class cl%03i {\n" % cl
|
||||
decl += "public:\n"
|
||||
bindings += ' py::class_<cl%03i>("cl%03i")\n' % (cl, cl)
|
||||
for fn in range(nfns):
|
||||
ret = random.randint(0, nclasses - 1)
|
||||
params = [random.randint(0, nclasses - 1) for i in range(nargs)]
|
||||
decl += " cl%03i *fn_%03i(" % (ret, fn)
|
||||
decl += ", ".join("cl%03i *" % p for p in params)
|
||||
decl += ");\n"
|
||||
bindings += ' .def("fn_%03i", &cl%03i::fn_%03i, py::return_value_policy<py::manage_new_object>())\n' % \
|
||||
(fn, cl, fn)
|
||||
decl += "};\n\n"
|
||||
bindings += ' ;\n'
|
||||
|
||||
result = "#include <boost/python.hpp>\n\n"
|
||||
result += "namespace py = boost::python;\n\n"
|
||||
result += decl + '\n'
|
||||
result += "BOOST_PYTHON_MODULE(example) {\n"
|
||||
result += bindings
|
||||
result += "}"
|
||||
return result
|
||||
|
||||
|
||||
for codegen in [generate_dummy_code_pybind11, generate_dummy_code_boost]:
|
||||
print ("{")
|
||||
for i in range(0, 10):
|
||||
nclasses = 2 ** i
|
||||
with open("test.cpp", "w") as f:
|
||||
f.write(codegen(nclasses))
|
||||
n1 = dt.datetime.now()
|
||||
os.system("g++ -Os -shared -rdynamic -undefined dynamic_lookup "
|
||||
"-fvisibility=hidden -std=c++14 test.cpp -I include "
|
||||
"-I /System/Library/Frameworks/Python.framework/Headers -o test.so")
|
||||
n2 = dt.datetime.now()
|
||||
elapsed = (n2 - n1).total_seconds()
|
||||
size = os.stat('test.so').st_size
|
||||
print(" {%i, %f, %i}," % (nclasses * nfns, elapsed, size))
|
||||
print ("}")
|
97
lib/bip158/lib/pybind11/docs/benchmark.rst
Normal file
97
lib/bip158/lib/pybind11/docs/benchmark.rst
Normal file
|
@ -0,0 +1,97 @@
|
|||
Benchmark
|
||||
=========
|
||||
|
||||
The following is the result of a synthetic benchmark comparing both compilation
|
||||
time and module size of pybind11 against Boost.Python. A detailed report about a
|
||||
Boost.Python to pybind11 conversion of a real project is available here: [#f1]_.
|
||||
|
||||
.. [#f1] http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf
|
||||
|
||||
Setup
|
||||
-----
|
||||
|
||||
A python script (see the ``docs/benchmark.py`` file) was used to generate a set
|
||||
of files with dummy classes whose count increases for each successive benchmark
|
||||
(between 1 and 2048 classes in powers of two). Each class has four methods with
|
||||
a randomly generated signature with a return value and four arguments. (There
|
||||
was no particular reason for this setup other than the desire to generate many
|
||||
unique function signatures whose count could be controlled in a simple way.)
|
||||
|
||||
Here is an example of the binding code for one class:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
...
|
||||
class cl034 {
|
||||
public:
|
||||
cl279 *fn_000(cl084 *, cl057 *, cl065 *, cl042 *);
|
||||
cl025 *fn_001(cl098 *, cl262 *, cl414 *, cl121 *);
|
||||
cl085 *fn_002(cl445 *, cl297 *, cl145 *, cl421 *);
|
||||
cl470 *fn_003(cl200 *, cl323 *, cl332 *, cl492 *);
|
||||
};
|
||||
...
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
...
|
||||
py::class_<cl034>(m, "cl034")
|
||||
.def("fn_000", &cl034::fn_000)
|
||||
.def("fn_001", &cl034::fn_001)
|
||||
.def("fn_002", &cl034::fn_002)
|
||||
.def("fn_003", &cl034::fn_003)
|
||||
...
|
||||
}
|
||||
|
||||
The Boost.Python version looks almost identical except that a return value
|
||||
policy had to be specified as an argument to ``def()``. For both libraries,
|
||||
compilation was done with
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
Apple LLVM version 7.0.2 (clang-700.1.81)
|
||||
|
||||
and the following compilation flags
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++14
|
||||
|
||||
Compilation time
|
||||
----------------
|
||||
|
||||
The following log-log plot shows how the compilation time grows for an
|
||||
increasing number of class and function declarations. pybind11 includes many
|
||||
fewer headers, which initially leads to shorter compilation times, but the
|
||||
performance is ultimately fairly similar (pybind11 is 19.8 seconds faster for
|
||||
the largest largest file with 2048 classes and a total of 8192 methods -- a
|
||||
modest **1.2x** speedup relative to Boost.Python, which required 116.35
|
||||
seconds).
|
||||
|
||||
.. only:: not latex
|
||||
|
||||
.. image:: pybind11_vs_boost_python1.svg
|
||||
|
||||
.. only:: latex
|
||||
|
||||
.. image:: pybind11_vs_boost_python1.png
|
||||
|
||||
Module size
|
||||
-----------
|
||||
|
||||
Differences between the two libraries become much more pronounced when
|
||||
considering the file size of the generated Python plugin: for the largest file,
|
||||
the binary generated by Boost.Python required 16.8 MiB, which was **2.17
|
||||
times** / **9.1 megabytes** larger than the output generated by pybind11. For
|
||||
very small inputs, Boost.Python has an edge in the plot below -- however, note
|
||||
that it stores many definitions in an external library, whose size was not
|
||||
included here, hence the comparison is slightly shifted in Boost.Python's
|
||||
favor.
|
||||
|
||||
.. only:: not latex
|
||||
|
||||
.. image:: pybind11_vs_boost_python2.svg
|
||||
|
||||
.. only:: latex
|
||||
|
||||
.. image:: pybind11_vs_boost_python2.png
|
||||
|
||||
|
1232
lib/bip158/lib/pybind11/docs/changelog.rst
Normal file
1232
lib/bip158/lib/pybind11/docs/changelog.rst
Normal file
File diff suppressed because it is too large
Load diff
532
lib/bip158/lib/pybind11/docs/classes.rst
Normal file
532
lib/bip158/lib/pybind11/docs/classes.rst
Normal file
|
@ -0,0 +1,532 @@
|
|||
.. _classes:
|
||||
|
||||
Object-oriented code
|
||||
####################
|
||||
|
||||
Creating bindings for a custom type
|
||||
===================================
|
||||
|
||||
Let's now look at a more complex example where we'll create bindings for a
|
||||
custom C++ data structure named ``Pet``. Its definition is given below:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct Pet {
|
||||
Pet(const std::string &name) : name(name) { }
|
||||
void setName(const std::string &name_) { name = name_; }
|
||||
const std::string &getName() const { return name; }
|
||||
|
||||
std::string name;
|
||||
};
|
||||
|
||||
The binding code for ``Pet`` looks as follows:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
#include <pybind11/pybind11.h>
|
||||
|
||||
namespace py = pybind11;
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &>())
|
||||
.def("setName", &Pet::setName)
|
||||
.def("getName", &Pet::getName);
|
||||
}
|
||||
|
||||
:class:`class_` creates bindings for a C++ *class* or *struct*-style data
|
||||
structure. :func:`init` is a convenience function that takes the types of a
|
||||
constructor's parameters as template arguments and wraps the corresponding
|
||||
constructor (see the :ref:`custom_constructors` section for details). An
|
||||
interactive Python session demonstrating this example is shown below:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
% python
|
||||
>>> import example
|
||||
>>> p = example.Pet('Molly')
|
||||
>>> print(p)
|
||||
<example.Pet object at 0x10cd98060>
|
||||
>>> p.getName()
|
||||
u'Molly'
|
||||
>>> p.setName('Charly')
|
||||
>>> p.getName()
|
||||
u'Charly'
|
||||
|
||||
.. seealso::
|
||||
|
||||
Static member functions can be bound in the same way using
|
||||
:func:`class_::def_static`.
|
||||
|
||||
Keyword and default arguments
|
||||
=============================
|
||||
It is possible to specify keyword and default arguments using the syntax
|
||||
discussed in the previous chapter. Refer to the sections :ref:`keyword_args`
|
||||
and :ref:`default_args` for details.
|
||||
|
||||
Binding lambda functions
|
||||
========================
|
||||
|
||||
Note how ``print(p)`` produced a rather useless summary of our data structure in the example above:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print(p)
|
||||
<example.Pet object at 0x10cd98060>
|
||||
|
||||
To address this, we could bind an utility function that returns a human-readable
|
||||
summary to the special method slot named ``__repr__``. Unfortunately, there is no
|
||||
suitable functionality in the ``Pet`` data structure, and it would be nice if
|
||||
we did not have to change it. This can easily be accomplished by binding a
|
||||
Lambda function instead:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &>())
|
||||
.def("setName", &Pet::setName)
|
||||
.def("getName", &Pet::getName)
|
||||
.def("__repr__",
|
||||
[](const Pet &a) {
|
||||
return "<example.Pet named '" + a.name + "'>";
|
||||
}
|
||||
);
|
||||
|
||||
Both stateless [#f1]_ and stateful lambda closures are supported by pybind11.
|
||||
With the above change, the same Python code now produces the following output:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print(p)
|
||||
<example.Pet named 'Molly'>
|
||||
|
||||
.. [#f1] Stateless closures are those with an empty pair of brackets ``[]`` as the capture object.
|
||||
|
||||
.. _properties:
|
||||
|
||||
Instance and static fields
|
||||
==========================
|
||||
|
||||
We can also directly expose the ``name`` field using the
|
||||
:func:`class_::def_readwrite` method. A similar :func:`class_::def_readonly`
|
||||
method also exists for ``const`` fields.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &>())
|
||||
.def_readwrite("name", &Pet::name)
|
||||
// ... remainder ...
|
||||
|
||||
This makes it possible to write
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.Pet('Molly')
|
||||
>>> p.name
|
||||
u'Molly'
|
||||
>>> p.name = 'Charly'
|
||||
>>> p.name
|
||||
u'Charly'
|
||||
|
||||
Now suppose that ``Pet::name`` was a private internal variable
|
||||
that can only be accessed via setters and getters.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
class Pet {
|
||||
public:
|
||||
Pet(const std::string &name) : name(name) { }
|
||||
void setName(const std::string &name_) { name = name_; }
|
||||
const std::string &getName() const { return name; }
|
||||
private:
|
||||
std::string name;
|
||||
};
|
||||
|
||||
In this case, the method :func:`class_::def_property`
|
||||
(:func:`class_::def_property_readonly` for read-only data) can be used to
|
||||
provide a field-like interface within Python that will transparently call
|
||||
the setter and getter functions:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &>())
|
||||
.def_property("name", &Pet::getName, &Pet::setName)
|
||||
// ... remainder ...
|
||||
|
||||
Write only properties can be defined by passing ``nullptr`` as the
|
||||
input for the read function.
|
||||
|
||||
.. seealso::
|
||||
|
||||
Similar functions :func:`class_::def_readwrite_static`,
|
||||
:func:`class_::def_readonly_static` :func:`class_::def_property_static`,
|
||||
and :func:`class_::def_property_readonly_static` are provided for binding
|
||||
static variables and properties. Please also see the section on
|
||||
:ref:`static_properties` in the advanced part of the documentation.
|
||||
|
||||
Dynamic attributes
|
||||
==================
|
||||
|
||||
Native Python classes can pick up new attributes dynamically:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> class Pet:
|
||||
... name = 'Molly'
|
||||
...
|
||||
>>> p = Pet()
|
||||
>>> p.name = 'Charly' # overwrite existing
|
||||
>>> p.age = 2 # dynamically add a new attribute
|
||||
|
||||
By default, classes exported from C++ do not support this and the only writable
|
||||
attributes are the ones explicitly defined using :func:`class_::def_readwrite`
|
||||
or :func:`class_::def_property`.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<>())
|
||||
.def_readwrite("name", &Pet::name);
|
||||
|
||||
Trying to set any other attribute results in an error:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.Pet()
|
||||
>>> p.name = 'Charly' # OK, attribute defined in C++
|
||||
>>> p.age = 2 # fail
|
||||
AttributeError: 'Pet' object has no attribute 'age'
|
||||
|
||||
To enable dynamic attributes for C++ classes, the :class:`py::dynamic_attr` tag
|
||||
must be added to the :class:`py::class_` constructor:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet", py::dynamic_attr())
|
||||
.def(py::init<>())
|
||||
.def_readwrite("name", &Pet::name);
|
||||
|
||||
Now everything works as expected:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.Pet()
|
||||
>>> p.name = 'Charly' # OK, overwrite value in C++
|
||||
>>> p.age = 2 # OK, dynamically add a new attribute
|
||||
>>> p.__dict__ # just like a native Python class
|
||||
{'age': 2}
|
||||
|
||||
Note that there is a small runtime cost for a class with dynamic attributes.
|
||||
Not only because of the addition of a ``__dict__``, but also because of more
|
||||
expensive garbage collection tracking which must be activated to resolve
|
||||
possible circular references. Native Python classes incur this same cost by
|
||||
default, so this is not anything to worry about. By default, pybind11 classes
|
||||
are more efficient than native Python classes. Enabling dynamic attributes
|
||||
just brings them on par.
|
||||
|
||||
.. _inheritance:
|
||||
|
||||
Inheritance and automatic downcasting
|
||||
=====================================
|
||||
|
||||
Suppose now that the example consists of two data structures with an
|
||||
inheritance relationship:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct Pet {
|
||||
Pet(const std::string &name) : name(name) { }
|
||||
std::string name;
|
||||
};
|
||||
|
||||
struct Dog : Pet {
|
||||
Dog(const std::string &name) : Pet(name) { }
|
||||
std::string bark() const { return "woof!"; }
|
||||
};
|
||||
|
||||
There are two different ways of indicating a hierarchical relationship to
|
||||
pybind11: the first specifies the C++ base class as an extra template
|
||||
parameter of the :class:`class_`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &>())
|
||||
.def_readwrite("name", &Pet::name);
|
||||
|
||||
// Method 1: template parameter:
|
||||
py::class_<Dog, Pet /* <- specify C++ parent type */>(m, "Dog")
|
||||
.def(py::init<const std::string &>())
|
||||
.def("bark", &Dog::bark);
|
||||
|
||||
Alternatively, we can also assign a name to the previously bound ``Pet``
|
||||
:class:`class_` object and reference it when binding the ``Dog`` class:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet> pet(m, "Pet");
|
||||
pet.def(py::init<const std::string &>())
|
||||
.def_readwrite("name", &Pet::name);
|
||||
|
||||
// Method 2: pass parent class_ object:
|
||||
py::class_<Dog>(m, "Dog", pet /* <- specify Python parent type */)
|
||||
.def(py::init<const std::string &>())
|
||||
.def("bark", &Dog::bark);
|
||||
|
||||
Functionality-wise, both approaches are equivalent. Afterwards, instances will
|
||||
expose fields and methods of both types:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.Dog('Molly')
|
||||
>>> p.name
|
||||
u'Molly'
|
||||
>>> p.bark()
|
||||
u'woof!'
|
||||
|
||||
The C++ classes defined above are regular non-polymorphic types with an
|
||||
inheritance relationship. This is reflected in Python:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
// Return a base pointer to a derived instance
|
||||
m.def("pet_store", []() { return std::unique_ptr<Pet>(new Dog("Molly")); });
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.pet_store()
|
||||
>>> type(p) # `Dog` instance behind `Pet` pointer
|
||||
Pet # no pointer downcasting for regular non-polymorphic types
|
||||
>>> p.bark()
|
||||
AttributeError: 'Pet' object has no attribute 'bark'
|
||||
|
||||
The function returned a ``Dog`` instance, but because it's a non-polymorphic
|
||||
type behind a base pointer, Python only sees a ``Pet``. In C++, a type is only
|
||||
considered polymorphic if it has at least one virtual function and pybind11
|
||||
will automatically recognize this:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct PolymorphicPet {
|
||||
virtual ~PolymorphicPet() = default;
|
||||
};
|
||||
|
||||
struct PolymorphicDog : PolymorphicPet {
|
||||
std::string bark() const { return "woof!"; }
|
||||
};
|
||||
|
||||
// Same binding code
|
||||
py::class_<PolymorphicPet>(m, "PolymorphicPet");
|
||||
py::class_<PolymorphicDog, PolymorphicPet>(m, "PolymorphicDog")
|
||||
.def(py::init<>())
|
||||
.def("bark", &PolymorphicDog::bark);
|
||||
|
||||
// Again, return a base pointer to a derived instance
|
||||
m.def("pet_store2", []() { return std::unique_ptr<PolymorphicPet>(new PolymorphicDog); });
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = example.pet_store2()
|
||||
>>> type(p)
|
||||
PolymorphicDog # automatically downcast
|
||||
>>> p.bark()
|
||||
u'woof!'
|
||||
|
||||
Given a pointer to a polymorphic base, pybind11 performs automatic downcasting
|
||||
to the actual derived type. Note that this goes beyond the usual situation in
|
||||
C++: we don't just get access to the virtual functions of the base, we get the
|
||||
concrete derived type including functions and attributes that the base type may
|
||||
not even be aware of.
|
||||
|
||||
.. seealso::
|
||||
|
||||
For more information about polymorphic behavior see :ref:`overriding_virtuals`.
|
||||
|
||||
|
||||
Overloaded methods
|
||||
==================
|
||||
|
||||
Sometimes there are several overloaded C++ methods with the same name taking
|
||||
different kinds of input arguments:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct Pet {
|
||||
Pet(const std::string &name, int age) : name(name), age(age) { }
|
||||
|
||||
void set(int age_) { age = age_; }
|
||||
void set(const std::string &name_) { name = name_; }
|
||||
|
||||
std::string name;
|
||||
int age;
|
||||
};
|
||||
|
||||
Attempting to bind ``Pet::set`` will cause an error since the compiler does not
|
||||
know which method the user intended to select. We can disambiguate by casting
|
||||
them to function pointers. Binding multiple functions to the same Python name
|
||||
automatically creates a chain of function overloads that will be tried in
|
||||
sequence.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def(py::init<const std::string &, int>())
|
||||
.def("set", (void (Pet::*)(int)) &Pet::set, "Set the pet's age")
|
||||
.def("set", (void (Pet::*)(const std::string &)) &Pet::set, "Set the pet's name");
|
||||
|
||||
The overload signatures are also visible in the method's docstring:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> help(example.Pet)
|
||||
|
||||
class Pet(__builtin__.object)
|
||||
| Methods defined here:
|
||||
|
|
||||
| __init__(...)
|
||||
| Signature : (Pet, str, int) -> NoneType
|
||||
|
|
||||
| set(...)
|
||||
| 1. Signature : (Pet, int) -> NoneType
|
||||
|
|
||||
| Set the pet's age
|
||||
|
|
||||
| 2. Signature : (Pet, str) -> NoneType
|
||||
|
|
||||
| Set the pet's name
|
||||
|
||||
If you have a C++14 compatible compiler [#cpp14]_, you can use an alternative
|
||||
syntax to cast the overloaded function:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def("set", py::overload_cast<int>(&Pet::set), "Set the pet's age")
|
||||
.def("set", py::overload_cast<const std::string &>(&Pet::set), "Set the pet's name");
|
||||
|
||||
Here, ``py::overload_cast`` only requires the parameter types to be specified.
|
||||
The return type and class are deduced. This avoids the additional noise of
|
||||
``void (Pet::*)()`` as seen in the raw cast. If a function is overloaded based
|
||||
on constness, the ``py::const_`` tag should be used:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct Widget {
|
||||
int foo(int x, float y);
|
||||
int foo(int x, float y) const;
|
||||
};
|
||||
|
||||
py::class_<Widget>(m, "Widget")
|
||||
.def("foo_mutable", py::overload_cast<int, float>(&Widget::foo))
|
||||
.def("foo_const", py::overload_cast<int, float>(&Widget::foo, py::const_));
|
||||
|
||||
If you prefer the ``py::overload_cast`` syntax but have a C++11 compatible compiler only,
|
||||
you can use ``py::detail::overload_cast_impl`` with an additional set of parentheses:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
template <typename... Args>
|
||||
using overload_cast_ = pybind11::detail::overload_cast_impl<Args...>;
|
||||
|
||||
py::class_<Pet>(m, "Pet")
|
||||
.def("set", overload_cast_<int>()(&Pet::set), "Set the pet's age")
|
||||
.def("set", overload_cast_<const std::string &>()(&Pet::set), "Set the pet's name");
|
||||
|
||||
.. [#cpp14] A compiler which supports the ``-std=c++14`` flag
|
||||
or Visual Studio 2015 Update 2 and newer.
|
||||
|
||||
.. note::
|
||||
|
||||
To define multiple overloaded constructors, simply declare one after the
|
||||
other using the ``.def(py::init<...>())`` syntax. The existing machinery
|
||||
for specifying keyword and default arguments also works.
|
||||
|
||||
Enumerations and internal types
|
||||
===============================
|
||||
|
||||
Let's now suppose that the example class contains an internal enumeration type,
|
||||
e.g.:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct Pet {
|
||||
enum Kind {
|
||||
Dog = 0,
|
||||
Cat
|
||||
};
|
||||
|
||||
Pet(const std::string &name, Kind type) : name(name), type(type) { }
|
||||
|
||||
std::string name;
|
||||
Kind type;
|
||||
};
|
||||
|
||||
The binding code for this example looks as follows:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::class_<Pet> pet(m, "Pet");
|
||||
|
||||
pet.def(py::init<const std::string &, Pet::Kind>())
|
||||
.def_readwrite("name", &Pet::name)
|
||||
.def_readwrite("type", &Pet::type);
|
||||
|
||||
py::enum_<Pet::Kind>(pet, "Kind")
|
||||
.value("Dog", Pet::Kind::Dog)
|
||||
.value("Cat", Pet::Kind::Cat)
|
||||
.export_values();
|
||||
|
||||
To ensure that the ``Kind`` type is created within the scope of ``Pet``, the
|
||||
``pet`` :class:`class_` instance must be supplied to the :class:`enum_`.
|
||||
constructor. The :func:`enum_::export_values` function exports the enum entries
|
||||
into the parent scope, which should be skipped for newer C++11-style strongly
|
||||
typed enums.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = Pet('Lucy', Pet.Cat)
|
||||
>>> p.type
|
||||
Kind.Cat
|
||||
>>> int(p.type)
|
||||
1L
|
||||
|
||||
The entries defined by the enumeration type are exposed in the ``__members__`` property:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> Pet.Kind.__members__
|
||||
{'Dog': Kind.Dog, 'Cat': Kind.Cat}
|
||||
|
||||
The ``name`` property returns the name of the enum value as a unicode string.
|
||||
|
||||
.. note::
|
||||
|
||||
It is also possible to use ``str(enum)``, however these accomplish different
|
||||
goals. The following shows how these two approaches differ.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> p = Pet( "Lucy", Pet.Cat )
|
||||
>>> pet_type = p.type
|
||||
>>> pet_type
|
||||
Pet.Cat
|
||||
>>> str(pet_type)
|
||||
'Pet.Cat'
|
||||
>>> pet_type.name
|
||||
'Cat'
|
||||
|
||||
.. note::
|
||||
|
||||
When the special tag ``py::arithmetic()`` is specified to the ``enum_``
|
||||
constructor, pybind11 creates an enumeration that also supports rudimentary
|
||||
arithmetic and bit-level operations like comparisons, and, or, xor, negation,
|
||||
etc.
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
py::enum_<Pet::Kind>(pet, "Kind", py::arithmetic())
|
||||
...
|
||||
|
||||
By default, these are omitted to conserve space.
|
289
lib/bip158/lib/pybind11/docs/compiling.rst
Normal file
289
lib/bip158/lib/pybind11/docs/compiling.rst
Normal file
|
@ -0,0 +1,289 @@
|
|||
.. _compiling:
|
||||
|
||||
Build systems
|
||||
#############
|
||||
|
||||
Building with setuptools
|
||||
========================
|
||||
|
||||
For projects on PyPI, building with setuptools is the way to go. Sylvain Corlay
|
||||
has kindly provided an example project which shows how to set up everything,
|
||||
including automatic generation of documentation using Sphinx. Please refer to
|
||||
the [python_example]_ repository.
|
||||
|
||||
.. [python_example] https://github.com/pybind/python_example
|
||||
|
||||
Building with cppimport
|
||||
========================
|
||||
|
||||
[cppimport]_ is a small Python import hook that determines whether there is a C++
|
||||
source file whose name matches the requested module. If there is, the file is
|
||||
compiled as a Python extension using pybind11 and placed in the same folder as
|
||||
the C++ source file. Python is then able to find the module and load it.
|
||||
|
||||
.. [cppimport] https://github.com/tbenthompson/cppimport
|
||||
|
||||
.. _cmake:
|
||||
|
||||
Building with CMake
|
||||
===================
|
||||
|
||||
For C++ codebases that have an existing CMake-based build system, a Python
|
||||
extension module can be created with just a few lines of code:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
cmake_minimum_required(VERSION 2.8.12)
|
||||
project(example)
|
||||
|
||||
add_subdirectory(pybind11)
|
||||
pybind11_add_module(example example.cpp)
|
||||
|
||||
This assumes that the pybind11 repository is located in a subdirectory named
|
||||
:file:`pybind11` and that the code is located in a file named :file:`example.cpp`.
|
||||
The CMake command ``add_subdirectory`` will import the pybind11 project which
|
||||
provides the ``pybind11_add_module`` function. It will take care of all the
|
||||
details needed to build a Python extension module on any platform.
|
||||
|
||||
A working sample project, including a way to invoke CMake from :file:`setup.py` for
|
||||
PyPI integration, can be found in the [cmake_example]_ repository.
|
||||
|
||||
.. [cmake_example] https://github.com/pybind/cmake_example
|
||||
|
||||
pybind11_add_module
|
||||
-------------------
|
||||
|
||||
To ease the creation of Python extension modules, pybind11 provides a CMake
|
||||
function with the following signature:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
pybind11_add_module(<name> [MODULE | SHARED] [EXCLUDE_FROM_ALL]
|
||||
[NO_EXTRAS] [SYSTEM] [THIN_LTO] source1 [source2 ...])
|
||||
|
||||
This function behaves very much like CMake's builtin ``add_library`` (in fact,
|
||||
it's a wrapper function around that command). It will add a library target
|
||||
called ``<name>`` to be built from the listed source files. In addition, it
|
||||
will take care of all the Python-specific compiler and linker flags as well
|
||||
as the OS- and Python-version-specific file extension. The produced target
|
||||
``<name>`` can be further manipulated with regular CMake commands.
|
||||
|
||||
``MODULE`` or ``SHARED`` may be given to specify the type of library. If no
|
||||
type is given, ``MODULE`` is used by default which ensures the creation of a
|
||||
Python-exclusive module. Specifying ``SHARED`` will create a more traditional
|
||||
dynamic library which can also be linked from elsewhere. ``EXCLUDE_FROM_ALL``
|
||||
removes this target from the default build (see CMake docs for details).
|
||||
|
||||
Since pybind11 is a template library, ``pybind11_add_module`` adds compiler
|
||||
flags to ensure high quality code generation without bloat arising from long
|
||||
symbol names and duplication of code in different translation units. It
|
||||
sets default visibility to *hidden*, which is required for some pybind11
|
||||
features and functionality when attempting to load multiple pybind11 modules
|
||||
compiled under different pybind11 versions. It also adds additional flags
|
||||
enabling LTO (Link Time Optimization) and strip unneeded symbols. See the
|
||||
:ref:`FAQ entry <faq:symhidden>` for a more detailed explanation. These
|
||||
latter optimizations are never applied in ``Debug`` mode. If ``NO_EXTRAS`` is
|
||||
given, they will always be disabled, even in ``Release`` mode. However, this
|
||||
will result in code bloat and is generally not recommended.
|
||||
|
||||
By default, pybind11 and Python headers will be included with ``-I``. In order
|
||||
to include pybind11 as system library, e.g. to avoid warnings in downstream
|
||||
code with warn-levels outside of pybind11's scope, set the option ``SYSTEM``.
|
||||
|
||||
As stated above, LTO is enabled by default. Some newer compilers also support
|
||||
different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause
|
||||
the function to prefer this flavor if available. The function falls back to
|
||||
regular LTO if ``-flto=thin`` is not available.
|
||||
|
||||
.. _ThinLTO: http://clang.llvm.org/docs/ThinLTO.html
|
||||
|
||||
Configuration variables
|
||||
-----------------------
|
||||
|
||||
By default, pybind11 will compile modules with the C++14 standard, if available
|
||||
on the target compiler, falling back to C++11 if C++14 support is not
|
||||
available. Note, however, that this default is subject to change: future
|
||||
pybind11 releases are expected to migrate to newer C++ standards as they become
|
||||
available. To override this, the standard flag can be given explicitly in
|
||||
``PYBIND11_CPP_STANDARD``:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
# Use just one of these:
|
||||
# GCC/clang:
|
||||
set(PYBIND11_CPP_STANDARD -std=c++11)
|
||||
set(PYBIND11_CPP_STANDARD -std=c++14)
|
||||
set(PYBIND11_CPP_STANDARD -std=c++1z) # Experimental C++17 support
|
||||
# MSVC:
|
||||
set(PYBIND11_CPP_STANDARD /std:c++14)
|
||||
set(PYBIND11_CPP_STANDARD /std:c++latest) # Enables some MSVC C++17 features
|
||||
|
||||
add_subdirectory(pybind11) # or find_package(pybind11)
|
||||
|
||||
Note that this and all other configuration variables must be set **before** the
|
||||
call to ``add_subdirectory`` or ``find_package``. The variables can also be set
|
||||
when calling CMake from the command line using the ``-D<variable>=<value>`` flag.
|
||||
|
||||
The target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION``
|
||||
or an exact Python installation can be specified with ``PYTHON_EXECUTABLE``.
|
||||
For example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -DPYBIND11_PYTHON_VERSION=3.6 ..
|
||||
# or
|
||||
cmake -DPYTHON_EXECUTABLE=path/to/python ..
|
||||
|
||||
find_package vs. add_subdirectory
|
||||
---------------------------------
|
||||
|
||||
For CMake-based projects that don't include the pybind11 repository internally,
|
||||
an external installation can be detected through ``find_package(pybind11)``.
|
||||
See the `Config file`_ docstring for details of relevant CMake variables.
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
cmake_minimum_required(VERSION 2.8.12)
|
||||
project(example)
|
||||
|
||||
find_package(pybind11 REQUIRED)
|
||||
pybind11_add_module(example example.cpp)
|
||||
|
||||
Note that ``find_package(pybind11)`` will only work correctly if pybind11
|
||||
has been correctly installed on the system, e. g. after downloading or cloning
|
||||
the pybind11 repository :
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd pybind11
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make install
|
||||
|
||||
Once detected, the aforementioned ``pybind11_add_module`` can be employed as
|
||||
before. The function usage and configuration variables are identical no matter
|
||||
if pybind11 is added as a subdirectory or found as an installed package. You
|
||||
can refer to the same [cmake_example]_ repository for a full sample project
|
||||
-- just swap out ``add_subdirectory`` for ``find_package``.
|
||||
|
||||
.. _Config file: https://github.com/pybind/pybind11/blob/master/tools/pybind11Config.cmake.in
|
||||
|
||||
Advanced: interface library target
|
||||
----------------------------------
|
||||
|
||||
When using a version of CMake greater than 3.0, pybind11 can additionally
|
||||
be used as a special *interface library* . The target ``pybind11::module``
|
||||
is available with pybind11 headers, Python headers and libraries as needed,
|
||||
and C++ compile definitions attached. This target is suitable for linking
|
||||
to an independently constructed (through ``add_library``, not
|
||||
``pybind11_add_module``) target in the consuming project.
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
cmake_minimum_required(VERSION 3.0)
|
||||
project(example)
|
||||
|
||||
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
|
||||
|
||||
add_library(example MODULE main.cpp)
|
||||
target_link_libraries(example PRIVATE pybind11::module)
|
||||
set_target_properties(example PROPERTIES PREFIX "${PYTHON_MODULE_PREFIX}"
|
||||
SUFFIX "${PYTHON_MODULE_EXTENSION}")
|
||||
|
||||
.. warning::
|
||||
|
||||
Since pybind11 is a metatemplate library, it is crucial that certain
|
||||
compiler flags are provided to ensure high quality code generation. In
|
||||
contrast to the ``pybind11_add_module()`` command, the CMake interface
|
||||
library only provides the *minimal* set of parameters to ensure that the
|
||||
code using pybind11 compiles, but it does **not** pass these extra compiler
|
||||
flags (i.e. this is up to you).
|
||||
|
||||
These include Link Time Optimization (``-flto`` on GCC/Clang/ICPC, ``/GL``
|
||||
and ``/LTCG`` on Visual Studio) and .OBJ files with many sections on Visual
|
||||
Studio (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an
|
||||
explanation on why these are needed.
|
||||
|
||||
Embedding the Python interpreter
|
||||
--------------------------------
|
||||
|
||||
In addition to extension modules, pybind11 also supports embedding Python into
|
||||
a C++ executable or library. In CMake, simply link with the ``pybind11::embed``
|
||||
target. It provides everything needed to get the interpreter running. The Python
|
||||
headers and libraries are attached to the target. Unlike ``pybind11::module``,
|
||||
there is no need to manually set any additional properties here. For more
|
||||
information about usage in C++, see :doc:`/advanced/embedding`.
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
cmake_minimum_required(VERSION 3.0)
|
||||
project(example)
|
||||
|
||||
find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11)
|
||||
|
||||
add_executable(example main.cpp)
|
||||
target_link_libraries(example PRIVATE pybind11::embed)
|
||||
|
||||
.. _building_manually:
|
||||
|
||||
Building manually
|
||||
=================
|
||||
|
||||
pybind11 is a header-only library, hence it is not necessary to link against
|
||||
any special libraries and there are no intermediate (magic) translation steps.
|
||||
|
||||
On Linux, you can compile an example such as the one given in
|
||||
:ref:`simple_example` using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
|
||||
|
||||
The flags given here assume that you're using Python 3. For Python 2, just
|
||||
change the executable appropriately (to ``python`` or ``python2``).
|
||||
|
||||
The ``python3 -m pybind11 --includes`` command fetches the include paths for
|
||||
both pybind11 and Python headers. This assumes that pybind11 has been installed
|
||||
using ``pip`` or ``conda``. If it hasn't, you can also manually specify
|
||||
``-I <path-to-pybind11>/include`` together with the Python includes path
|
||||
``python3-config --includes``.
|
||||
|
||||
Note that Python 2.7 modules don't use a special suffix, so you should simply
|
||||
use ``example.so`` instead of ``example`python3-config --extension-suffix```.
|
||||
Besides, the ``--extension-suffix`` option may or may not be available, depending
|
||||
on the distribution; in the latter case, the module extension can be manually
|
||||
set to ``.so``.
|
||||
|
||||
On Mac OS: the build command is almost the same but it also requires passing
|
||||
the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when
|
||||
building the module:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
|
||||
|
||||
In general, it is advisable to include several additional build parameters
|
||||
that can considerably reduce the size of the created binary. Refer to section
|
||||
:ref:`cmake` for a detailed example of a suitable cross-platform CMake-based
|
||||
build system that works on all platforms including Windows.
|
||||
|
||||
.. note::
|
||||
|
||||
On Linux and macOS, it's better to (intentionally) not link against
|
||||
``libpython``. The symbols will be resolved when the extension library
|
||||
is loaded into a Python binary. This is preferable because you might
|
||||
have several different installations of a given Python version (e.g. the
|
||||
system-provided Python, and one that ships with a piece of commercial
|
||||
software). In this way, the plugin will work with both versions, instead
|
||||
of possibly importing a second Python library into a process that already
|
||||
contains one (which will lead to a segfault).
|
||||
|
||||
Generating binding code automatically
|
||||
=====================================
|
||||
|
||||
The ``Binder`` project is a tool for automatic generation of pybind11 binding
|
||||
code by introspecting existing C++ codebases using LLVM/Clang. See the
|
||||
[binder]_ documentation for details.
|
||||
|
||||
.. [binder] http://cppbinder.readthedocs.io/en/latest/about.html
|
332
lib/bip158/lib/pybind11/docs/conf.py
Normal file
332
lib/bip158/lib/pybind11/docs/conf.py
Normal file
|
@ -0,0 +1,332 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# pybind11 documentation build configuration file, created by
|
||||
# sphinx-quickstart on Sun Oct 11 19:23:48 2015.
|
||||
#
|
||||
# This file is execfile()d with the current directory set to its
|
||||
# containing dir.
|
||||
#
|
||||
# Note that not all possible configuration values are present in this
|
||||
# autogenerated file.
|
||||
#
|
||||
# All configuration values have a default; values that are commented out
|
||||
# serve to show the default.
|
||||
|
||||
import sys
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#sys.path.insert(0, os.path.abspath('.'))
|
||||
|
||||
# -- General configuration ------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = ['breathe']
|
||||
|
||||
breathe_projects = {'pybind11': '.build/doxygenxml/'}
|
||||
breathe_default_project = 'pybind11'
|
||||
breathe_domain_by_extension = {'h': 'cpp'}
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['.templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
# source_suffix = ['.rst', '.md']
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The encoding of source files.
|
||||
#source_encoding = 'utf-8-sig'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# General information about the project.
|
||||
project = 'pybind11'
|
||||
copyright = '2017, Wenzel Jakob'
|
||||
author = 'Wenzel Jakob'
|
||||
|
||||
# The version info for the project you're documenting, acts as replacement for
|
||||
# |version| and |release|, also used in various other places throughout the
|
||||
# built documents.
|
||||
#
|
||||
# The short X.Y version.
|
||||
version = '2.4'
|
||||
# The full version, including alpha/beta/rc tags.
|
||||
release = '2.4.dev4'
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# There are two options for replacing |today|: either, you set today to some
|
||||
# non-false value, then it is used:
|
||||
#today = ''
|
||||
# Else, today_fmt is used as the format for a strftime call.
|
||||
#today_fmt = '%B %d, %Y'
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
exclude_patterns = ['.build', 'release.rst']
|
||||
|
||||
# The reST default role (used for this markup: `text`) to use for all
|
||||
# documents.
|
||||
default_role = 'any'
|
||||
|
||||
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||
#add_function_parentheses = True
|
||||
|
||||
# If true, the current module name will be prepended to all description
|
||||
# unit titles (such as .. function::).
|
||||
#add_module_names = True
|
||||
|
||||
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||
# output. They are ignored by default.
|
||||
#show_authors = False
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
#pygments_style = 'monokai'
|
||||
|
||||
# A list of ignored prefixes for module index sorting.
|
||||
#modindex_common_prefix = []
|
||||
|
||||
# If true, keep warnings as "system message" paragraphs in the built documents.
|
||||
#keep_warnings = False
|
||||
|
||||
# If true, `todo` and `todoList` produce output, else they produce nothing.
|
||||
todo_include_todos = False
|
||||
|
||||
|
||||
# -- Options for HTML output ----------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
|
||||
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
|
||||
|
||||
if not on_rtd: # only import and set the theme if we're building docs locally
|
||||
import sphinx_rtd_theme
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
|
||||
html_context = {
|
||||
'css_files': [
|
||||
'_static/theme_overrides.css'
|
||||
]
|
||||
}
|
||||
else:
|
||||
html_context = {
|
||||
'css_files': [
|
||||
'//media.readthedocs.org/css/sphinx_rtd_theme.css',
|
||||
'//media.readthedocs.org/css/readthedocs-doc-embed.css',
|
||||
'_static/theme_overrides.css'
|
||||
]
|
||||
}
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom themes here, relative to this directory.
|
||||
#html_theme_path = []
|
||||
|
||||
# The name for this set of Sphinx documents. If None, it defaults to
|
||||
# "<project> v<release> documentation".
|
||||
#html_title = None
|
||||
|
||||
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||
#html_short_title = None
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top
|
||||
# of the sidebar.
|
||||
#html_logo = None
|
||||
|
||||
# The name of an image file (within the static path) to use as favicon of the
|
||||
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||
# pixels large.
|
||||
#html_favicon = None
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
|
||||
# Add any extra paths that contain custom files (such as robots.txt or
|
||||
# .htaccess) here, relative to this directory. These files are copied
|
||||
# directly to the root of the documentation.
|
||||
#html_extra_path = []
|
||||
|
||||
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
|
||||
# using the given strftime format.
|
||||
#html_last_updated_fmt = '%b %d, %Y'
|
||||
|
||||
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||
# typographically correct entities.
|
||||
#html_use_smartypants = True
|
||||
|
||||
# Custom sidebar templates, maps document names to template names.
|
||||
#html_sidebars = {}
|
||||
|
||||
# Additional templates that should be rendered to pages, maps page names to
|
||||
# template names.
|
||||
#html_additional_pages = {}
|
||||
|
||||
# If false, no module index is generated.
|
||||
#html_domain_indices = True
|
||||
|
||||
# If false, no index is generated.
|
||||
#html_use_index = True
|
||||
|
||||
# If true, the index is split into individual pages for each letter.
|
||||
#html_split_index = False
|
||||
|
||||
# If true, links to the reST sources are added to the pages.
|
||||
#html_show_sourcelink = True
|
||||
|
||||
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
|
||||
#html_show_sphinx = True
|
||||
|
||||
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
|
||||
#html_show_copyright = True
|
||||
|
||||
# If true, an OpenSearch description file will be output, and all pages will
|
||||
# contain a <link> tag referring to it. The value of this option must be the
|
||||
# base URL from which the finished HTML is served.
|
||||
#html_use_opensearch = ''
|
||||
|
||||
# This is the file name suffix for HTML files (e.g. ".xhtml").
|
||||
#html_file_suffix = None
|
||||
|
||||
# Language to be used for generating the HTML full-text search index.
|
||||
# Sphinx supports the following languages:
|
||||
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
|
||||
# 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr'
|
||||
#html_search_language = 'en'
|
||||
|
||||
# A dictionary with options for the search language support, empty by default.
|
||||
# Now only 'ja' uses this config value
|
||||
#html_search_options = {'type': 'default'}
|
||||
|
||||
# The name of a javascript file (relative to the configuration directory) that
|
||||
# implements a search results scorer. If empty, the default will be used.
|
||||
#html_search_scorer = 'scorer.js'
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'pybind11doc'
|
||||
|
||||
# -- Options for LaTeX output ---------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#'papersize': 'letterpaper',
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#'pointsize': '10pt',
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
'preamble': '\DeclareUnicodeCharacter{00A0}{}',
|
||||
|
||||
# Latex figure (float) alignment
|
||||
#'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, 'pybind11.tex', 'pybind11 Documentation',
|
||||
'Wenzel Jakob', 'manual'),
|
||||
]
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top of
|
||||
# the title page.
|
||||
# latex_logo = 'pybind11-logo.png'
|
||||
|
||||
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||
# not chapters.
|
||||
#latex_use_parts = False
|
||||
|
||||
# If true, show page references after internal links.
|
||||
#latex_show_pagerefs = False
|
||||
|
||||
# If true, show URL addresses after external links.
|
||||
#latex_show_urls = False
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
#latex_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
#latex_domain_indices = True
|
||||
|
||||
|
||||
# -- Options for manual page output ---------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(master_doc, 'pybind11', 'pybind11 Documentation',
|
||||
[author], 1)
|
||||
]
|
||||
|
||||
# If true, show URL addresses after external links.
|
||||
#man_show_urls = False
|
||||
|
||||
|
||||
# -- Options for Texinfo output -------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(master_doc, 'pybind11', 'pybind11 Documentation',
|
||||
author, 'pybind11', 'One line description of project.',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
#texinfo_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
#texinfo_domain_indices = True
|
||||
|
||||
# How to display URL addresses: 'footnote', 'no', or 'inline'.
|
||||
#texinfo_show_urls = 'footnote'
|
||||
|
||||
# If true, do not generate a @detailmenu in the "Top" node's menu.
|
||||
#texinfo_no_detailmenu = False
|
||||
|
||||
primary_domain = 'cpp'
|
||||
highlight_language = 'cpp'
|
||||
|
||||
|
||||
def generate_doxygen_xml(app):
|
||||
build_dir = os.path.join(app.confdir, '.build')
|
||||
if not os.path.exists(build_dir):
|
||||
os.mkdir(build_dir)
|
||||
|
||||
try:
|
||||
subprocess.call(['doxygen', '--version'])
|
||||
retcode = subprocess.call(['doxygen'], cwd=app.confdir)
|
||||
if retcode < 0:
|
||||
sys.stderr.write("doxygen error code: {}\n".format(-retcode))
|
||||
except OSError as e:
|
||||
sys.stderr.write("doxygen execution failed: {}\n".format(e))
|
||||
|
||||
|
||||
def setup(app):
|
||||
"""Add hook for building doxygen xml when needed"""
|
||||
app.connect("builder-inited", generate_doxygen_xml)
|
297
lib/bip158/lib/pybind11/docs/faq.rst
Normal file
297
lib/bip158/lib/pybind11/docs/faq.rst
Normal file
|
@ -0,0 +1,297 @@
|
|||
Frequently asked questions
|
||||
##########################
|
||||
|
||||
"ImportError: dynamic module does not define init function"
|
||||
===========================================================
|
||||
|
||||
1. Make sure that the name specified in PYBIND11_MODULE is identical to the
|
||||
filename of the extension library (without prefixes such as .so)
|
||||
|
||||
2. If the above did not fix the issue, you are likely using an incompatible
|
||||
version of Python (for instance, the extension library was compiled against
|
||||
Python 2, while the interpreter is running on top of some version of Python
|
||||
3, or vice versa).
|
||||
|
||||
"Symbol not found: ``__Py_ZeroStruct`` / ``_PyInstanceMethod_Type``"
|
||||
========================================================================
|
||||
|
||||
See the first answer.
|
||||
|
||||
"SystemError: dynamic module not initialized properly"
|
||||
======================================================
|
||||
|
||||
See the first answer.
|
||||
|
||||
The Python interpreter immediately crashes when importing my module
|
||||
===================================================================
|
||||
|
||||
See the first answer.
|
||||
|
||||
CMake doesn't detect the right Python version
|
||||
=============================================
|
||||
|
||||
The CMake-based build system will try to automatically detect the installed
|
||||
version of Python and link against that. When this fails, or when there are
|
||||
multiple versions of Python and it finds the wrong one, delete
|
||||
``CMakeCache.txt`` and then invoke CMake as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -DPYTHON_EXECUTABLE:FILEPATH=<path-to-python-executable> .
|
||||
|
||||
.. _faq_reference_arguments:
|
||||
|
||||
Limitations involving reference arguments
|
||||
=========================================
|
||||
|
||||
In C++, it's fairly common to pass arguments using mutable references or
|
||||
mutable pointers, which allows both read and write access to the value
|
||||
supplied by the caller. This is sometimes done for efficiency reasons, or to
|
||||
realize functions that have multiple return values. Here are two very basic
|
||||
examples:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void increment(int &i) { i++; }
|
||||
void increment_ptr(int *i) { (*i)++; }
|
||||
|
||||
In Python, all arguments are passed by reference, so there is no general
|
||||
issue in binding such code from Python.
|
||||
|
||||
However, certain basic Python types (like ``str``, ``int``, ``bool``,
|
||||
``float``, etc.) are **immutable**. This means that the following attempt
|
||||
to port the function to Python doesn't have the same effect on the value
|
||||
provided by the caller -- in fact, it does nothing at all.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def increment(i):
|
||||
i += 1 # nope..
|
||||
|
||||
pybind11 is also affected by such language-level conventions, which means that
|
||||
binding ``increment`` or ``increment_ptr`` will also create Python functions
|
||||
that don't modify their arguments.
|
||||
|
||||
Although inconvenient, one workaround is to encapsulate the immutable types in
|
||||
a custom type that does allow modifications.
|
||||
|
||||
An other alternative involves binding a small wrapper lambda function that
|
||||
returns a tuple with all output arguments (see the remainder of the
|
||||
documentation for examples on binding lambda functions). An example:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
int foo(int &i) { i++; return 123; }
|
||||
|
||||
and the binding code
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
m.def("foo", [](int i) { int rv = foo(i); return std::make_tuple(rv, i); });
|
||||
|
||||
|
||||
How can I reduce the build time?
|
||||
================================
|
||||
|
||||
It's good practice to split binding code over multiple files, as in the
|
||||
following example:
|
||||
|
||||
:file:`example.cpp`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void init_ex1(py::module &);
|
||||
void init_ex2(py::module &);
|
||||
/* ... */
|
||||
|
||||
PYBIND11_MODULE(example, m) {
|
||||
init_ex1(m);
|
||||
init_ex2(m);
|
||||
/* ... */
|
||||
}
|
||||
|
||||
:file:`ex1.cpp`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void init_ex1(py::module &m) {
|
||||
m.def("add", [](int a, int b) { return a + b; });
|
||||
}
|
||||
|
||||
:file:`ex2.cpp`:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
void init_ex2(py::module &m) {
|
||||
m.def("sub", [](int a, int b) { return a - b; });
|
||||
}
|
||||
|
||||
:command:`python`:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import example
|
||||
>>> example.add(1, 2)
|
||||
3
|
||||
>>> example.sub(1, 1)
|
||||
0
|
||||
|
||||
As shown above, the various ``init_ex`` functions should be contained in
|
||||
separate files that can be compiled independently from one another, and then
|
||||
linked together into the same final shared object. Following this approach
|
||||
will:
|
||||
|
||||
1. reduce memory requirements per compilation unit.
|
||||
|
||||
2. enable parallel builds (if desired).
|
||||
|
||||
3. allow for faster incremental builds. For instance, when a single class
|
||||
definition is changed, only a subset of the binding code will generally need
|
||||
to be recompiled.
|
||||
|
||||
"recursive template instantiation exceeded maximum depth of 256"
|
||||
================================================================
|
||||
|
||||
If you receive an error about excessive recursive template evaluation, try
|
||||
specifying a larger value, e.g. ``-ftemplate-depth=1024`` on GCC/Clang. The
|
||||
culprit is generally the generation of function signatures at compile time
|
||||
using C++14 template metaprogramming.
|
||||
|
||||
.. _`faq:hidden_visibility`:
|
||||
|
||||
"‘SomeClass’ declared with greater visibility than the type of its field ‘SomeClass::member’ [-Wattributes]"
|
||||
============================================================================================================
|
||||
|
||||
This error typically indicates that you are compiling without the required
|
||||
``-fvisibility`` flag. pybind11 code internally forces hidden visibility on
|
||||
all internal code, but if non-hidden (and thus *exported*) code attempts to
|
||||
include a pybind type (for example, ``py::object`` or ``py::list``) you can run
|
||||
into this warning.
|
||||
|
||||
To avoid it, make sure you are specifying ``-fvisibility=hidden`` when
|
||||
compiling pybind code.
|
||||
|
||||
As to why ``-fvisibility=hidden`` is necessary, because pybind modules could
|
||||
have been compiled under different versions of pybind itself, it is also
|
||||
important that the symbols defined in one module do not clash with the
|
||||
potentially-incompatible symbols defined in another. While Python extension
|
||||
modules are usually loaded with localized symbols (under POSIX systems
|
||||
typically using ``dlopen`` with the ``RTLD_LOCAL`` flag), this Python default
|
||||
can be changed, but even if it isn't it is not always enough to guarantee
|
||||
complete independence of the symbols involved when not using
|
||||
``-fvisibility=hidden``.
|
||||
|
||||
Additionally, ``-fvisiblity=hidden`` can deliver considerably binary size
|
||||
savings. (See the following section for more details).
|
||||
|
||||
|
||||
.. _`faq:symhidden`:
|
||||
|
||||
How can I create smaller binaries?
|
||||
==================================
|
||||
|
||||
To do its job, pybind11 extensively relies on a programming technique known as
|
||||
*template metaprogramming*, which is a way of performing computation at compile
|
||||
time using type information. Template metaprogamming usually instantiates code
|
||||
involving significant numbers of deeply nested types that are either completely
|
||||
removed or reduced to just a few instructions during the compiler's optimization
|
||||
phase. However, due to the nested nature of these types, the resulting symbol
|
||||
names in the compiled extension library can be extremely long. For instance,
|
||||
the included test suite contains the following symbol:
|
||||
|
||||
.. only:: html
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
__ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_
|
||||
|
||||
.. only:: not html
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
__ZN8pybind1112cpp_functionC1Iv8Example2JRNSt3__16vectorINS3_12basic_stringIwNS3_11char_traitsIwEENS3_9allocatorIwEEEENS8_ISA_EEEEEJNS_4nameENS_7siblingENS_9is_methodEA28_cEEEMT0_FT_DpT1_EDpRKT2_
|
||||
|
||||
which is the mangled form of the following function type:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
pybind11::cpp_function::cpp_function<void, Example2, std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > >&, pybind11::name, pybind11::sibling, pybind11::is_method, char [28]>(void (Example2::*)(std::__1::vector<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> >, std::__1::allocator<std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > > >&), pybind11::name const&, pybind11::sibling const&, pybind11::is_method const&, char const (&) [28])
|
||||
|
||||
The memory needed to store just the mangled name of this function (196 bytes)
|
||||
is larger than the actual piece of code (111 bytes) it represents! On the other
|
||||
hand, it's silly to even give this function a name -- after all, it's just a
|
||||
tiny cog in a bigger piece of machinery that is not exposed to the outside
|
||||
world. So we'll generally only want to export symbols for those functions which
|
||||
are actually called from the outside.
|
||||
|
||||
This can be achieved by specifying the parameter ``-fvisibility=hidden`` to GCC
|
||||
and Clang, which sets the default symbol visibility to *hidden*, which has a
|
||||
tremendous impact on the final binary size of the resulting extension library.
|
||||
(On Visual Studio, symbols are already hidden by default, so nothing needs to
|
||||
be done there.)
|
||||
|
||||
In addition to decreasing binary size, ``-fvisibility=hidden`` also avoids
|
||||
potential serious issues when loading multiple modules and is required for
|
||||
proper pybind operation. See the previous FAQ entry for more details.
|
||||
|
||||
Working with ancient Visual Studio 2008 builds on Windows
|
||||
=========================================================
|
||||
|
||||
The official Windows distributions of Python are compiled using truly
|
||||
ancient versions of Visual Studio that lack good C++11 support. Some users
|
||||
implicitly assume that it would be impossible to load a plugin built with
|
||||
Visual Studio 2015 into a Python distribution that was compiled using Visual
|
||||
Studio 2008. However, no such issue exists: it's perfectly legitimate to
|
||||
interface DLLs that are built with different compilers and/or C libraries.
|
||||
Common gotchas to watch out for involve not ``free()``-ing memory region
|
||||
that that were ``malloc()``-ed in another shared library, using data
|
||||
structures with incompatible ABIs, and so on. pybind11 is very careful not
|
||||
to make these types of mistakes.
|
||||
|
||||
Inconsistent detection of Python version in CMake and pybind11
|
||||
==============================================================
|
||||
|
||||
The functions ``find_package(PythonInterp)`` and ``find_package(PythonLibs)`` provided by CMake
|
||||
for Python version detection are not used by pybind11 due to unreliability and limitations that make
|
||||
them unsuitable for pybind11's needs. Instead pybind provides its own, more reliable Python detection
|
||||
CMake code. Conflicts can arise, however, when using pybind11 in a project that *also* uses the CMake
|
||||
Python detection in a system with several Python versions installed.
|
||||
|
||||
This difference may cause inconsistencies and errors if *both* mechanisms are used in the same project. Consider the following
|
||||
Cmake code executed in a system with Python 2.7 and 3.x installed:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
find_package(PythonInterp)
|
||||
find_package(PythonLibs)
|
||||
find_package(pybind11)
|
||||
|
||||
It will detect Python 2.7 and pybind11 will pick it as well.
|
||||
|
||||
In contrast this code:
|
||||
|
||||
.. code-block:: cmake
|
||||
|
||||
find_package(pybind11)
|
||||
find_package(PythonInterp)
|
||||
find_package(PythonLibs)
|
||||
|
||||
will detect Python 3.x for pybind11 and may crash on ``find_package(PythonLibs)`` afterwards.
|
||||
|
||||
It is advised to avoid using ``find_package(PythonInterp)`` and ``find_package(PythonLibs)`` from CMake and rely
|
||||
on pybind11 in detecting Python version. If this is not possible CMake machinery should be called *before* including pybind11.
|
||||
|
||||
How to cite this project?
|
||||
=========================
|
||||
|
||||
We suggest the following BibTeX template to cite pybind11 in scientific
|
||||
discourse:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@misc{pybind11,
|
||||
author = {Wenzel Jakob and Jason Rhinelander and Dean Moldovan},
|
||||
year = {2017},
|
||||
note = {https://github.com/pybind/pybind11},
|
||||
title = {pybind11 -- Seamless operability between C++11 and Python}
|
||||
}
|
47
lib/bip158/lib/pybind11/docs/index.rst
Normal file
47
lib/bip158/lib/pybind11/docs/index.rst
Normal file
|
@ -0,0 +1,47 @@
|
|||
.. only: not latex
|
||||
|
||||
.. image:: pybind11-logo.png
|
||||
|
||||
pybind11 --- Seamless operability between C++11 and Python
|
||||
==========================================================
|
||||
|
||||
.. only: not latex
|
||||
|
||||
Contents:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
intro
|
||||
changelog
|
||||
upgrade
|
||||
|
||||
.. toctree::
|
||||
:caption: The Basics
|
||||
:maxdepth: 2
|
||||
|
||||
basics
|
||||
classes
|
||||
compiling
|
||||
|
||||
.. toctree::
|
||||
:caption: Advanced Topics
|
||||
:maxdepth: 2
|
||||
|
||||
advanced/functions
|
||||
advanced/classes
|
||||
advanced/exceptions
|
||||
advanced/smart_ptrs
|
||||
advanced/cast/index
|
||||
advanced/pycpp/index
|
||||
advanced/embedding
|
||||
advanced/misc
|
||||
|
||||
.. toctree::
|
||||
:caption: Extra Information
|
||||
:maxdepth: 1
|
||||
|
||||
faq
|
||||
benchmark
|
||||
limitations
|
||||
reference
|
93
lib/bip158/lib/pybind11/docs/intro.rst
Normal file
93
lib/bip158/lib/pybind11/docs/intro.rst
Normal file
|
@ -0,0 +1,93 @@
|
|||
.. image:: pybind11-logo.png
|
||||
|
||||
About this project
|
||||
==================
|
||||
**pybind11** is a lightweight header-only library that exposes C++ types in Python
|
||||
and vice versa, mainly to create Python bindings of existing C++ code. Its
|
||||
goals and syntax are similar to the excellent `Boost.Python`_ library by David
|
||||
Abrahams: to minimize boilerplate code in traditional extension modules by
|
||||
inferring type information using compile-time introspection.
|
||||
|
||||
.. _Boost.Python: http://www.boost.org/doc/libs/release/libs/python/doc/index.html
|
||||
|
||||
The main issue with Boost.Python—and the reason for creating such a similar
|
||||
project—is Boost. Boost is an enormously large and complex suite of utility
|
||||
libraries that works with almost every C++ compiler in existence. This
|
||||
compatibility has its cost: arcane template tricks and workarounds are
|
||||
necessary to support the oldest and buggiest of compiler specimens. Now that
|
||||
C++11-compatible compilers are widely available, this heavy machinery has
|
||||
become an excessively large and unnecessary dependency.
|
||||
Think of this library as a tiny self-contained version of Boost.Python with
|
||||
everything stripped away that isn't relevant for binding generation. Without
|
||||
comments, the core header files only require ~4K lines of code and depend on
|
||||
Python (2.7 or 3.x, or PyPy2.7 >= 5.7) and the C++ standard library. This
|
||||
compact implementation was possible thanks to some of the new C++11 language
|
||||
features (specifically: tuples, lambda functions and variadic templates). Since
|
||||
its creation, this library has grown beyond Boost.Python in many ways, leading
|
||||
to dramatically simpler binding code in many common situations.
|
||||
|
||||
Core features
|
||||
*************
|
||||
The following core C++ features can be mapped to Python
|
||||
|
||||
- Functions accepting and returning custom data structures per value, reference, or pointer
|
||||
- Instance methods and static methods
|
||||
- Overloaded functions
|
||||
- Instance attributes and static attributes
|
||||
- Arbitrary exception types
|
||||
- Enumerations
|
||||
- Callbacks
|
||||
- Iterators and ranges
|
||||
- Custom operators
|
||||
- Single and multiple inheritance
|
||||
- STL data structures
|
||||
- Smart pointers with reference counting like ``std::shared_ptr``
|
||||
- Internal references with correct reference counting
|
||||
- C++ classes with virtual (and pure virtual) methods can be extended in Python
|
||||
|
||||
Goodies
|
||||
*******
|
||||
In addition to the core functionality, pybind11 provides some extra goodies:
|
||||
|
||||
- Python 2.7, 3.x, and PyPy (PyPy2.7 >= 5.7) are supported with an
|
||||
implementation-agnostic interface.
|
||||
|
||||
- It is possible to bind C++11 lambda functions with captured variables. The
|
||||
lambda capture data is stored inside the resulting Python function object.
|
||||
|
||||
- pybind11 uses C++11 move constructors and move assignment operators whenever
|
||||
possible to efficiently transfer custom data types.
|
||||
|
||||
- It's easy to expose the internal storage of custom data types through
|
||||
Pythons' buffer protocols. This is handy e.g. for fast conversion between
|
||||
C++ matrix classes like Eigen and NumPy without expensive copy operations.
|
||||
|
||||
- pybind11 can automatically vectorize functions so that they are transparently
|
||||
applied to all entries of one or more NumPy array arguments.
|
||||
|
||||
- Python's slice-based access and assignment operations can be supported with
|
||||
just a few lines of code.
|
||||
|
||||
- Everything is contained in just a few header files; there is no need to link
|
||||
against any additional libraries.
|
||||
|
||||
- Binaries are generally smaller by a factor of at least 2 compared to
|
||||
equivalent bindings generated by Boost.Python. A recent pybind11 conversion
|
||||
of `PyRosetta`_, an enormous Boost.Python binding project, reported a binary
|
||||
size reduction of **5.4x** and compile time reduction by **5.8x**.
|
||||
|
||||
- Function signatures are precomputed at compile time (using ``constexpr``),
|
||||
leading to smaller binaries.
|
||||
|
||||
- With little extra effort, C++ types can be pickled and unpickled similar to
|
||||
regular Python objects.
|
||||
|
||||
.. _PyRosetta: http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf
|
||||
|
||||
Supported compilers
|
||||
*******************
|
||||
|
||||
1. Clang/LLVM (any non-ancient version with C++11 support)
|
||||
2. GCC 4.8 or newer
|
||||
3. Microsoft Visual Studio 2015 or newer
|
||||
4. Intel C++ compiler v17 or newer (v16 with pybind11 v2.0 and v15 with pybind11 v2.0 and a `workaround <https://github.com/pybind/pybind11/issues/276>`_ )
|
20
lib/bip158/lib/pybind11/docs/limitations.rst
Normal file
20
lib/bip158/lib/pybind11/docs/limitations.rst
Normal file
|
@ -0,0 +1,20 @@
|
|||
Limitations
|
||||
###########
|
||||
|
||||
pybind11 strives to be a general solution to binding generation, but it also has
|
||||
certain limitations:
|
||||
|
||||
- pybind11 casts away ``const``-ness in function arguments and return values.
|
||||
This is in line with the Python language, which has no concept of ``const``
|
||||
values. This means that some additional care is needed to avoid bugs that
|
||||
would be caught by the type checker in a traditional C++ program.
|
||||
|
||||
- The NumPy interface ``pybind11::array`` greatly simplifies accessing
|
||||
numerical data from C++ (and vice versa), but it's not a full-blown array
|
||||
class like ``Eigen::Array`` or ``boost.multi_array``.
|
||||
|
||||
These features could be implemented but would lead to a significant increase in
|
||||
complexity. I've decided to draw the line here to keep this project simple and
|
||||
compact. Users who absolutely require these features are encouraged to fork
|
||||
pybind11.
|
||||
|
BIN
lib/bip158/lib/pybind11/docs/pybind11-logo.png
Normal file
BIN
lib/bip158/lib/pybind11/docs/pybind11-logo.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 57 KiB |
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue