It seems to me the buffer protocol is more for exposing Python buffer to C.
I couldn't find a way to create a bytes object using existing buffer without copying in C.
Basically what I want is to implement something similar to PyBytes_FromStringAndSize() but without copying, and with a callback to free the buffer when the object is released. I don't know how big the buffer is before I receive the buffer returned from a C API. So creating bytes object in Python first and later fill it in is not an option.
I also looked into memoryview, PyMemoryView_FromMemory() doesn't copy but there is no way to pass a callback to free my buffer. And I'm not suse Python lib (e.g. Psycopg) can use memoryview object or not.
Do I have to create my own object to achieve these 2 requirements? Any other shortcut?
If I have to, how can I make sure this object works same as bytes so I can pass it to Python lib safely?
Thanks.
The only way to do this is to create a new object with PyBufferProcs* PyTypeObject.tp_as_buffer. I checked cpython source code thoroughly, as of 3.4.1, there is no out-of-box (so to speak) solution.
python c-api: create bytes using existing buffer without copying