host:
Replace the msgpack-RPC (rpclib) client with a gRPC-based
implementation. Add auto-generated C++ client and Python bindings
from protobuf definitions. Remove rpclib dependency and integrate
FindgRPC into CMake.
mpm:
Replace the msgpack-RPC server with a gRPC server generated from
protobuf definitions. Add pb_utils.py for type conversion between
Python and protobuf. Update rpc_server.py to dispatch calls via
gRPC stubs. Update mpm_shell to use gRPC. Replace pyudev with sysfs
scan in uio device discovery.
Bump MPM compatibility number to 7.0 and RPC version to 2.
This change requires new depedencies. E.g. for Ubuntu 24.04 or 25.10:
libgrpc++-dev, libprotobuf-dev, protobuf-compiler-grpc, python3-grpcio
This commit makes it explicit which assignment operators
and constructors are available on the object class.
Furthermore the copy constructor and assignment operator
make deep copies of the object. This prevents double free
operations with the rpc_error class.
Some instances of RPC code had rvalue reference arguments but did
not use std::forward wenn passing these arguments (or the other
way around). This harmonizes arguments and parameter forwarding to
always use rvalue ref and std::forward in pairs.
During tests with the reinitialization of the MPM, we discovered that the
USRP session sometimes ended with an access violation. The root cause was
a call to reset_timer_and_mgr, which was sent using notify_with_token
from the rpclib. This function internally performs a notify, which
itself makes a call. The call invokes the function as an async_call and
subsequently attempts to retrieve the result of the future from the
async_call, provided no timeout occurs.
The issue arose because the call did not use rvalue references for the
variadic array. As a result, the variadic array values were copied during
the call and then deallocated afterward. However, the rpclib also retained
a pointer to the variadic array, which led to a double free after the
async_call completed. The problem became evident when the memory
management code enabled heap protection (e.g., the ucrtd.dll on Windows,
which writes hex-D into unallocated memory). In most other cases, these
regions are initialized to 0, and freeing a 0 pointer is effectively a
no-op.
The fix introduces lvalue passing of the variadic array in the call
function, which prevents double free errors. This change ensures that the
callee no longer frees the array. The fix has also been applied to
async_call and send.
This adds uhdlib/asio.hpp, which makes it possible for UHD to use either
Boost.Asio or regular asio. Note that for using regular asio, additional
CMake changes are required, which are not part of this changeset, and
all C++ sources would need to reference namespace asio:: instead of
boost::asio::, which is also not part of this changeset.
However, the boost/asio.hpp include statement is now limited to a single
location, which makes it possible to set custom include preferences for
this file. Here, we allow gcc builds to allow some warnings when
including Boost.Asio.
In some cases, rpclib uses the value of the `__cplusplus` macro to
determine implementation details. We remove these, as UHD requires C++17
anyway now and the value of __cplusplus may be broken depending on
compilers.
Starting with Boost 1.66 and the corresponding ASIO version, there were
some changes introduced based on the C++ Networking TS. This includes
changes like replacing io_service with io_context, deprecating some
functions, etc. Starting with Boost 1.87, the old style is no longer
supported.
This commit updates all usage of ASIO in a way that makes UHD compatible
with future versions of ASIO. However, this makes UHD no longer
compatible with Boost 1.65 and below.
Summary of changes:
- Replace asio::io_service with asio::io_context
- Replace asio::io_service::strand with
asio::strand<asio::io_context::executor_type>
- This implies using asio::post() instead of asio::strand::post()
- Replace asio::buffer_cast<T>(buf) with static_cast<T>(buf.data())
- Update resolve(query) with new API
- Replace references to endpoint_iterator with resolver::results_type
- Replace ip::address::from_string() with ip::make_address()
Co-authored-by: Jörg Hofrichter <joerg.hofrichter@emerson.com>
Different compilers are not in agreement how to handle captures for
constexprs. On recent clang versions, a warning such as this is produced:
warning: lambda capture 'max_read_bytes' is not required to be captured
for this use [-Wunused-lambda-capture]
However, some versions of MSVC *require* the capture, so we have to
leave it in and live with the warnings. We therefore disable the
warnings as they have no value for us.
In C++20, copy-capture of `this` must be made explicit, which we fix
with this commit. In C++17, auto_ptr is deprecated so we replace it with
unique_ptr.
This will result into use of the in-tree version on Ubuntu 20,
and address an observed crash during object cleanup at program
shutdown that was only observed on Ubuntu 20. This restores the
behavior earlier UHD version to use the in-tree pybind11 module
for Ubuntu 20.
Also corrected some logic determining when the in-tree module
was used to avoid that both a public and our in-tree modules are
part of the INCLUDE_DIRS.
Python 3.11 did not build correctly with previous versions of pybind11.
This updates the headers to pybind 2.10.0, which does.
Signed-off-by: Steven Koo <steven.koo@ni.com>
- Copied include/pybind11 directory over from source repo
- Also re-ran remove_comments.py as before
This fixes this warning on newer Python libraries:
.../internals.h: 200:9: warning: 'PyEval_InitThreads' is deprecated
[-Wdeprecated-declarations]
PyEval_InitThreads();
^
/usr/include/python3.9/ceval.h:130:1: note: 'PyEval_InitThreads' has
been explicitly marked deprecated here
Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
^
/usr/include/python3.9/pyport.h:508:54: note: expanded from macro
'Py_DEPRECATED'
#define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
It also obviates the need for patches to Pybind11 to silence clang
warnings.
The rpclib server session closes its socket, which can fail. In our
case, we only use the server for some tests, and we close the server on
process exit, so we let the error slide. Since we close servers in
dtors, not doing so would occasionally cause SIGABRT.
This changes two things in all applicable files:
- Remove imports from __future__
- Change default shebangs from /usr/bin/env python to /usr/bin/env
python3
FlatBuffers is used to (de-)serialize calibration data. For most use
cases, only a few headers are required, so we provide them with UHD
instead of requiring all UHD users to install their own version of
FlatBuffers.
This adds the headers to host/lib/deps, and updates the Debian copyright
file appropriately.
This removes the following Boost constructs:
- boost::shared_ptr, boost::weak_ptr
- boost::enable_shared_from_this
- boost::static_pointer_cast, boost::dynamic_pointer_cast
The appropriate includes were also removed. All C++11 versions of these
require #include <memory>.
Note that the stdlib and Boost versions have the exact same syntax, they
only differ in the namespace (boost vs. std). The modifications were all
done using sed, with the exception of boost::scoped_ptr, which was
replaced by std::unique_ptr.
References to boost::smart_ptr were also removed.
boost::intrusive_ptr is not removed in this commit, since it does not
have a 1:1 mapping to a C++11 construct.
The former was marked as deprecated numerous Boost versions ago and
finally was actually removed & replaced in 1.66.0 with a new one with a
template API. The version in rpclib need to be updated, and Boost docs
say to use the latter. Moving to this usage takes care of this issue.
Reviewed-by: Martin Braun <martin.braun@ettus.com>
