host:
Replace the msgpack-RPC (rpclib) client with a gRPC-based
implementation. Add auto-generated C++ client and Python bindings
from protobuf definitions. Remove rpclib dependency and integrate
FindgRPC into CMake.
mpm:
Replace the msgpack-RPC server with a gRPC server generated from
protobuf definitions. Add pb_utils.py for type conversion between
Python and protobuf. Update rpc_server.py to dispatch calls via
gRPC stubs. Update mpm_shell to use gRPC. Replace pyudev with sysfs
scan in uio device discovery.
Bump MPM compatibility number to 7.0 and RPC version to 2.
This change requires new depedencies. E.g. for Ubuntu 24.04 or 25.10:
libgrpc++-dev, libprotobuf-dev, protobuf-compiler-grpc, python3-grpcio
With the most recent changes, Boost.Thread and all Boost date/time
libraries are no longer required. This updates the CMake and Debian
files to reflect those changes.
Fixes issue introduced in UHD 4.9.0.1, that prevents local builds on Windows
against prebuild Boost binary (as described in the user manual).
Change is preserving use of Boost library symbols (i.e. Boost::<module>),
but requires use of a different cmake configuration and Boost 1.73.0 or later.
For details see updates to host/docs/build.dox.in for details.
This has multiple reasons:
- We use CMake's FindBoost
(https://cmake.org/cmake/help/v3.12/module/FindBoost.html) which can
resolve dependencies itself. We only use boost::system in tandem with
Boost::asio, which we already list as a dependency.
- There are combinations of CMake versions and Boost versions, where
specifying this will link against the stub library of Boost::system
(this is a header-only library since Boost 1.69). Removing it from the
list avoids any kind of linkage.
Note that UHDBoost.cmake has some compatibility hacks for Boost::system,
which we now also remove.
- Instead of blindly linking against all ${Boost_LIBRARIES}, we
selectively specify the various Boost::* targets.
- Replace include_directories() or link_directories() with target_*()
CMake functions to avoid cluttering global scope.
- Specify PRIVATE vs. PUBLIC on all link targets.
The GNUInstallDirs module supports absolute paths for
CMAKE_INSTALL_LIBDIR and CMAKE_INSTALL_INCLUDEDIR. Some package managers
like Nix depend on this behaviour.
See https://github.com/NixOS/nixpkgs/issues/144170 for the nixpkgs
tracking issue.
UHD_VERSION needs to be also checked against an empty string
since cmake 4.1 sets this variable during project setup, also
if it was undefined before. Furthermore we need to backup and
restore custom UHD_VERSION when defined for e.g. by using option
-DUHD_VERSION=<version>, because project() setup will reset the
version to an empty string.
With -DENABLE_RFNOC_DEV=ON (the default is OFF), the HDL sources for
building RFNoC blocks and RFNoC images through rfnoc_image_builder are
now also installed into $PREFIX/share/uhd/rfnoc/fpga/usrp3. This re-uses
the existing share/uhd/rfnoc/fpga directory.
This commit also includes a Python script that copies the fpga/usrp3
directory into a staging directory inside the build dir. The
installation happens from the staging directory, not the fpga/usrp3
directory itself. This allows us to filter the usrp3 source dir without
having to use highly verbose CMake for that.
The uhd-host Debian install file is also modified so that the RFNoC
sources are not packaged into the uhd-host package.
This is a vestage from the UHD-3 era, where RFNoC was optional. To
reduce confusion, we remove the CMake variable UHD_RFNOC_FOUND, although
we leave it in UHDConfig.cmake for not breaking 3rd party applications.
The UHD CMake code re-implemented some of the features of GNUInstallDirs,
which is not necessary (this feature has been available in CMake for
longer than version 3.12). Here, we replace all hand-defined CMake
variables with their equivalents from GNUInstallDirs.
Short table of replacements:
INCLUDE_DIR -> CMAKE_INSTALL_INCLUDEDIR
RUNTIME_DIR -> CMAKE_INSTALL_BINDIR
LIBRARY_DIR -> CMAKE_INSTALL_LIBDIR
UHD_LIB_DIR -> CMAKE_INSTALL_LIBDIR
LIB_SUFFIX -> n/a (no longer necessary)
- Newer versions of cmake do not set install_name, so we need to do it
ourselves.
- Made Boost a system dependency to suppress compiler warnings from it.
Packages may use the build type to decide whether to look for
release or debug binaries. It is therefore cruicial to set the
build type before searching any package.
Simplifying the Windows installer experience by reducing the
number of individual downloads from 4 (for USB devices) to 2.
Including the libusb.dll into the installer will remove a common
onboarding error.
This change introduces a now, optional cmake feature called
-DENABLE_EXTEND_WIN_PACKAGING=ON, that defaults to off.
Enabling this feature will cause a number of convenience components
to be included into the windows deployment package (aka. installer).
Enabling this feature has no effect on other platforms.
The changes to the NSIS itself are trivial and limited to another
license dialog (for libusb redistribution). Most changes were in
the cmake code used by cpack (for which NSIS is the utilized Windows
cpack generator option).
These additional components now controlled via the feature
component are currently:
- incl. of MSVC Runtime binaries into ${RUNTIME_DIR}
- incl. of libusb.dll into ${RUNTIME_DIR}
- incl. of erllc_uhd_winusb_driver into ${PKG_DATA_DIR}/usbdriver
This installs UHDPython.cmake to lib/uhd/, so that OOT RFNoC modules can
use it (and don't have to use a version that ships with the OOT). This
makes maintanance of that file a lot easier.
Now that this file can (and should) be used outside of UHD, we
"namespace" the macros and functions by prefacing them with 'UHD_'
(e.g., there's now UHD_PYTHON_CHECK_MODULE).
This file requires creating UHDMinDepVersions.cmake, which also gets
installed.
This will result into use of the in-tree version on Ubuntu 20,
and address an observed crash during object cleanup at program
shutdown that was only observed on Ubuntu 20. This restores the
behavior earlier UHD version to use the in-tree pybind11 module
for Ubuntu 20.
Also corrected some logic determining when the in-tree module
was used to avoid that both a public and our in-tree modules are
part of the INCLUDE_DIRS.
While 'Python Module (Utils only)' is a subset only of the Python API
'uhd' module for conveniently installing some Python utils only, to the
user it may look like they are missing something if the output of cmake
states that 'Python Module (Utils only)' is not in the enabled
components list by default. This commit changes the default from OFF to
ON.
This options allows to install Python utils, including a subset of the
Python API 'uhd' module, without having to compile UHD. The main reason
to do this is to allow installing the RFNoC image builder without the
rest of UHD. To achieve this, run cmake as such:
cmake /path/to/uhd -DENABLE_LIBUHD=OFF -DENABLE_PYMOD_UTILS=ON
You might also want to disable building the manual in this case.
The refactoring required to enable this also fixes an issues when
installing UHD from source into a venv. This updates the CMake to use
pip3 to install the module, fixing some warnings and install issues.
- Use find_package(pybind11) to find a system-wide Pybind11. This will
take preference over the vendored version of Pybind11, unless
requested otherwise.
- Fix a typo.
Adds cmake option for replacing dashes in the UHD Component version with
underscores. This is necessary as Fedora package management is now
enforcing the format <Package>-<Version>-<Release> for all rpm builds,
so dashes in the UHD_VERSION from dev branches is not supported.
Signed-off-by: mattprost <matt.prost@ni.com>
This commit replaces the use of distutils.version.LooseVersion() with
CMake's version comparison operator, which implements relational version
string checking in the same manner (i.e., comparing numeric components
of a version string numerically).
current implementation uses version strings for comparisons. This led
version 3.10 to be smaller than 3.6 which is obviously wrong. Use
LooseVersion to have correct version comparison.
See the CMake 3.8 documentation on these two variables:
https://cmake.org/cmake/help/v3.8/variable/PROJECT-NAME_SOURCE_DIR.htmlhttps://cmake.org/cmake/help/v3.8/variable/CMAKE_SOURCE_DIR.html
Under normal circumstances, these two are identical. For sub-projects
(i.e., when building UHD as part of something else that is also a CMake
project), only the former is useful. There is no discernible downside of
using UHD_SOURCE_DIR over CMAKE_SOURCE_DIR.
This was changed using sed:
$ sed -i "s/CMAKE_SOURCE_DIR/UHD_SOURCE_DIR/g" \
`ag -l CMAKE_SOURCE_DIR **/{CMakeLists.txt,*.cmake}`
$ sed -i "s/CMAKE_BINARY_DIR/UHD_BINARY_DIR/g" \
`ag -l CMAKE_BINARY_DIR **/{CMakeLists.txt,*.cmake}`
At the same time, we also replace the CMake variable UHD_HOST_ROOT (used
in MPM) with UHD_SOURCE_DIR. There's no reason to have two variables
with the same meaning and different names, but more importantly, this
means that UHD_SOURCE_DIR is defined even in those cases where MPM calls
into CMake files from UHD without any additional patches.
Shoutout to GitHub user marcobergamin for bringing this up.
cmake has newer variables to identify specific Visual Studio version.
This change updates to use those. This also changes how images
files are globbed to be relative paths, which is useful if build files
are moved from machine to machine. This case is possible in Azure
Pipeline's CI.
Signed-off-by: Steven Koo <steven.koo@ni.com>
The minimum version of the following dependencies have been updated for
UHD 4.1:
* GCC: 5.4.0 --> 6.3.0
* Clang: 3.8.0 --> 6.0.0
* Apple Clang: 600 --> 800
* Python: 3.5 --> 3.6
* Boost: 1.58 --> 1.65
This reverts commit 93acdfe380.
The commit introduced an issue when building without specifying
ENABLE_TESTS explicitly to CMake.
Normally, if the user doesn't set ENABLE_TESTS, it gets set to ON by
default, assuming the prerequisites are met. However, there's a
chicken-and-egg problem here. UHDBoost.cmake takes the list of required
Boost components from UHD_BOOST_REQUIRED_COMPONENTS and adds them to the
Boost_LIBRARIES list. This happens before ENABLE_TESTS gets a default
value, so the Boost unit test framework library is never added to this
list, even if ENABLE_TESTS ends up getting set to ON later on by default
in the script.
But, moving LIBUHD_REGISTER_COMPONENT (the macro that sets ENABLE_TESTS
to a default value) to go before UHDBoost.cmake is included won't work,
because that macro needs variables set by UHDBoost.cmake.
It may be possible to break the circular dependency by refactoring
UHDBoost.cmake somewhat, but for the time being, master shouldn't be
broken.
Up until now, we completely ignore the XDG specification.
This commit does the following to change that:
- It uses XDG_DATA_HOME and XDG_CONFIG_HOME for cal and config data,
respectively.
- If config data is in ~/.uhd/uhd.conf, that is still accepted, but if
it conflicts with $XDG_CONFIG_HOME/uhd.conf, it is ignored and a
warning is displayed
- The default location for cal data is thus ${HOME}/.local/share/uhd/cal
on Unix, and %LOCALAPPDATA%\uhd\cal on Windows. This is a change in
location!
- The UHD_CONFIG_DIR environment variable was confusingly named and is
now removed. It provided an alternative location than the home
directory. The same purpose is now much better served by XDG_DATA_HOME
and XDG_CONFIG_HOME.
The specification can be found here:
specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
While merging the FPGA code into the UHD codebase, the manual building
process was not modified, resulting in either a link to the website
instead of building the FPGA manual as part of the rest, or by using a
residual submodule.
boost::regex was a requirement until the minimum version of gcc was
increased. Since it is at version 5.3 now, using Boost.Regex is no
longer necessary.
This change is a pure search-and-replace; Boost and std versions of
regex are compatible and use the same syntax.
The builder has two major jobs:
* generate an image core file which reflects the FPGA image
configuration given by the user
* invoke Xilinx toolchain to actually build the FPGA image
For this purpose it needs to know where to find the FPGA source tree.
This tree can be give by the -F option.
The code that represents the user configurable part of the image is
written to a file called <device>_rfnoc_sandbox.v. To generate the file
these configuration files are needed:
* io_signatures.yml: A file describing the known IO signatures. This file
is global for all devices and contains the superset
of all signatures (not all signatures are used by all
devices). It resides in usrp3/top/ of the tree given
by -F.
* bsp.yml: A file describing interfaces of a specific device such as AXIS
transport interfaces or IO ports as well as device specific
settings. It resides in usrp3/top/<device> of the tree given by -F.
* <image>.yml: a file provided by the user with freely chosen name.
It describes which elements the image should contain
(RFNoC blocks, streaming endpoints, IO ports) and how
to connect them. The file also contains image setting
such as the CHDR width to be used.
The script uses mako templates to generate the sandbox file. Before the
template engine is invoked sanity checks are executed to ensure the
configuration is synthactic correct. The script also build up structures
to ease Verilog code generation in the template engine. The engine should
not invoke more Python than echoing variables or iterating of lists or
dictionaries. This eases debugging as errors in the template engine are
hard to track and difficult to read for the user.
All Python code is placed in a package called rfnoc. The templates used
by the builder are also part of this package. image_builder.py contains
a method called build_image which is the main entry point for the builder.
It can also be utilized by other Python programs. To align with the
existing uhd_image_builder there is also a wrapper in bin called
rfnoc_image_builder which expects similar commands as the uhd_image_builder.
For debugging purpuse the script can be invoked from host/utils using
$ PYTHONPATH=. python bin/rfnoc_image_builder <options>
When installed using cmake/make/make install the builder installs to
${CMAKE_INSTALL_PREFIX}bin and can be invoked without specifying a
PYTHONPATH.
One can also install the package using pip from host/utils
$ pip install .
Image config generation can also be done from GNU Radio Companion
files. The required GRC files are merged into gr-ettus.
Example usage:
$ rfnoc_image_builder -F ~/src/fpgadev -d x310 \
-r path/to/x310_rfnoc_image_core.grc \
-b path/to/gr-ettus/grc
Co-Authored-By: Alex Williams <alex.williams@ni.com>
Co-Authored-By: Sugandha Gupta <sugandha.gupta@ettus.com>
Co-Authored-By: Martin Braun <martin.braun@ettus.com>
- Boost >= 1.58
- CMake >= 3.5.1
- gcc >= 5.4.0
- Clang >= 3.8, AppleClang >= 600
- Python >= 3.5 (Py2k no longer supported)
- Numpy >= 1.11
- C++14 for lib, include may now use C++11 constructs.
- Because there is no more code requiring C++03 syntax, we remove the
include-specific clang-format file
Also install UHDBoost.cmake along with the CMake UHD find scripts, so
that it can be used by external modules. Tweak the UHDConfig.cmake
script so that, once installed, the new UHDBoost.cmake script is found.
Note that the project using UHD must first find UHD before trying to find
Boost. Also note that if the C++ standard is not set before UHDBoost is
loaded, it will be set to that currently used by UHD: C++14.
- Makes use of more modern find_package(Python2/3) if available
- Moves almost all Python-related code to UHDPython.cmake
- ENABLE_PYTHON3 is no longer necessary
