Robotic projects with ROS can involve multiple desktop/stationary computers and Single Board Computers that need to communicate. A very common communication protocol is Wi-Fi. When using this connection, you need to consider in which machine to start the ROS core node, which nodes publish topics, and which nodes subscribe to topics.
In my robotic project, I have a mobile robot with a Realsense D435 camera. The robot publishes raw image data and pointcloud data for navigational purposes over Wi-Fi. When measuring the topic frequency on the sender, the mobile robot, and the receiver, a stationary Linux workstation, I see a severe data transmission drop: From 30 images to 10, and from 20 point cloud samples to only 3.
To find the root cause, I made a systematic test that combines three areas: ROS1 vs ROS2, different versions of the camera SDKs, and different startup parameter for the ROS node. But the problem persisted. In the last stages of the investigation, I measured the raw network upload speed of my Raspberry Pi 4 - its capped at 6 MBs. Assuming the raw Image data are JPEG files, then a topic which publishes 30hz would require about 30 MB/s upload speed. Is this the bottleneck?
This article investigates network performance optimizations when using a Raspberry Pi 4 with ROS over Wi-Fi. It details the considerations, findings, and experiments to get to an adequate performance.
Hardware Considerations
First of all, lets consider the hardware itself. The different versions of the Raspberry Pi also provide different hardware capabilities. Comparing the Wi-FI Capabilities, here are the specs:
- Raspberry Pi 4: 2.4 & 5GhZ, 802.11 b/g/n/ac
- Raspberry Pi 3B+: 2.4 & 5GhZ, 802.11 b/g/n/ac
The 802.11ac 5GHz protocol delivers 433 to 6933 Mbit/s, and the 802.11n protocol between 72 to 600 Mbit/s. In principle, this should be the same performance. But according to a performance benchmark of the magpi magazine , the Raspberry Pi 4 can provide up 58/114 Mbps, while the Raspberry Pi 3 is at 59/97.67 Mbps. This measurement is taken under optimal conditions - line of sight to the router.
Source: https://magpi.raspberrypi.org/articles/raspberry-pi-4-specs-benchmarks
Given these numbers, I became very skeptical about the 6Mb/s measured in my home network. Looking in other articles, I could see that other users also considered the Raspberry Pi 4 as slow, for example in this this raspberrypi.org forum. But we need more data to verify this.
Internet Upload Speed on a 2.4GHz and 5GHz network
The first stage of my investigation measures the network speed of a wirelessly connected Raspberry Pi to the internet. The variables in this investigation are:
- Measurement tool: Speedtest
- Model: Raspberry Pi 3B+, Raspberry Pi 4
- Operating System: Raspberry Pi Os 2021-05-07, Ubuntu Server 20.04
- Network: 2.4 Ghz, 5 Ghz
Raspberry Pi 3B+, 2.4GHz Network, Raspberry Pi OS
Testing download speed................................................................................
Download: 12.76 Mbit/s
Testing upload speed......................................................................................................
Upload: 25.93 Mbit/s
Raspberry Pi 3B+, 5GHz Network, Raspberry Pi OS
Testing download speed................................................................................
Download: 72.61 Mbit/s
Testing upload speed......................................................................................................
Upload: 40.84 Mbit/s
Raspberry Pi 4, 2.4Ghz, Ubuntu 20.4
Testing download speed................................................................................
Download: 21.22 Mbit/s
Testing upload speed......................................................................................................
Upload: 22.05 Mbit/s
Raspberry Pi 4, 5Ghz, Ubuntu 20.4
Testing download speed................................................................................
Download: 34.35 Mbit/s
Testing upload speed......................................................................................................
Upload: 28.88 Mbit/s
Comparison
Overall, the Raspberry Pi 3B+ achieves about 26 Mbit/s in the 2.4 GHz, and about 40 Mbit/S in the 5Ghz network. Comparing the operating systems, Ubuntu show better performance with the 2.4Ghz network, but also a severe drop when using the 5Ghz network.
Speed Measurement Streaming Data
The second stage of my investigation measures the raw data transmission speed between a wirelessly connected Raspberry Pi and a stationary, ethernet connected Linux workstation (running Ubuntu 20.04). The variables in this investigation are:
- Measurement tool: iperf
- Model: Raspberry Pi 3B+, Raspberry Pi 4 (running Raspberry Pi OS 2021-05-07)
- Network: 2.4 Ghz, 5 Ghz, Ethernet
- Data Direction: From Raspberry Pi Workstation, between different Raspberry Pi’s
The measurement tool iperf will be started in server mode on one computer, and then in client mode on another computer: The client starts streaming data, and performance will be measured.
Ethernet / Raspberry Pi 4, => Linux Workstation
[ 4] local 192.168.2.102 port 5001 connected with 192.168.2.204 port 52960
[ 4] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec
5Ghz Wi-Fi / Raspberry Pi 4, => Linux Workstation
[ 3] local 192.168.2.116 port 39080 connected with 192.168.2.102 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.1 sec 15.0 MBytes 12.4 Mbits/sec
2.4Ghz Wi-Fi / Raspberry Pi 4 => Raspberry Pi 3B+
[ 4] local 192.168.2.116 port 5001 connected with 192.168.2.204 port 56496
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-11.9 sec 26.4 MBytes 18.6 Mbits/sec
2.4Ghz Wi-Fi / Raspberry Pi 3B+ => Raspberry Pi 4
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.2 sec 12.5 MBytes 10.3 Mbits/sec
Comparison
The results are somewhat shocking: Wirelessly streaming data from a Raspberry Pi to my Linux Workstation or another Raspberry Pi is limited to max 18 Mbits/sec! The minimum upload speed to the Internet was 22 Mbits/sec. What could be the reasons? I only came up with the idea that my router is very busy to handle the traffic for all other devices in my home, and therefore limits the bandwidth. To test this, I bought another, dedicated access point and repeated the experiments.
Speed Measurement Streaming Data with Dedicated Access Point
The setup for these tests looks very similar to the one beforehand.
- Measurement tool: iperf
- Model: Raspberry Pi 3B+, Raspberry Pi 4 (running Raspberry Pi Os 2021-05-07)
- Network: 2.4 Ghz, 5 Ghz
- Data Direction: From Raspberry Pi Workstation, between different Raspberry PI
Network Speed Optimization
To be as precise as possible, I want to report the speed measurements between an ethernet connected Linux Workstation and the Raspberry Pi. Let’s measure the direction of the streaming to determine where the ROS core node should be running, and let’s check the difference between the 2.4 Ghz and the 5.0 Ghz.
2.4 GHz Wi-Fi / Workstation => Raspberry Pi 4
Connecting to host 192.168.3.101, port 5201
[ 5] local 192.168.2.202 port 50960 connected to 192.168.3.101 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 11.2 MBytes 93.8 Mbits/sec 0 615 KBytes
[ 5] 1.00-2.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.01 MBytes
[ 5] 2.00-3.00 sec 7.50 MBytes 62.9 Mbits/sec 0 1.39 MBytes
[ 5] 3.00-4.00 sec 7.50 MBytes 62.9 Mbits/sec 0 1.77 MBytes
[ 5] 4.00-5.00 sec 7.50 MBytes 62.9 Mbits/sec 0 2.14 MBytes
[ 5] 5.00-6.00 sec 7.50 MBytes 62.9 Mbits/sec 0 2.50 MBytes
[ 5] 6.00-7.00 sec 7.50 MBytes 62.9 Mbits/sec 0 2.54 MBytes
[ 5] 7.00-8.00 sec 6.25 MBytes 52.4 Mbits/sec 0 2.54 MBytes
[ 5] 8.00-9.00 sec 7.50 MBytes 62.9 Mbits/sec 0 2.54 MBytes
[ 5] 9.00-10.00 sec 7.50 MBytes 62.9 Mbits/sec 0 2.54 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 78.7 MBytes 66.0 Mbits/sec 0 sender
[ 5] 0.00-10.35 sec 78.3 MBytes 63.4 Mbits/sec receiver
2.4 GHz Wi-Fi / Raspberry Pi 4 => Workstation
Connecting to host 192.168.2.202, port 5201
[ 5] local 192.168.3.101 port 35756 connected to 192.168.2.202 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 6.46 MBytes 54.2 Mbits/sec 0 294 KBytes
[ 5] 1.00-2.00 sec 7.39 MBytes 62.0 Mbits/sec 0 465 KBytes
[ 5] 2.00-3.00 sec 6.52 MBytes 54.7 Mbits/sec 0 628 KBytes
[ 5] 3.00-4.00 sec 6.84 MBytes 57.3 Mbits/sec 0 628 KBytes
[ 5] 4.00-5.00 sec 6.46 MBytes 54.2 Mbits/sec 0 694 KBytes
[ 5] 5.00-6.00 sec 6.71 MBytes 56.3 Mbits/sec 0 694 KBytes
[ 5] 6.00-7.00 sec 7.46 MBytes 62.6 Mbits/sec 0 694 KBytes
[ 5] 7.00-8.00 sec 6.77 MBytes 56.8 Mbits/sec 0 694 KBytes
[ 5] 8.00-9.00 sec 6.71 MBytes 56.3 Mbits/sec 0 694 KBytes
[ 5] 9.00-10.00 sec 6.71 MBytes 56.3 Mbits/sec 0 694 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 68.0 MBytes 57.1 Mbits/sec 0 sender
[ 5] 0.00-10.02 sec 66.7 MBytes 55.8 Mbits/sec receiver
5GHz Wi-FI / Workstation => Raspberry Pi 4
Connecting to host 192.168.3.101, port 5201
[ 5] local 192.168.2.202 port 51148 connected to 192.168.3.101 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 11.6 MBytes 97.4 Mbits/sec 0 460 KBytes
[ 5] 1.00-2.00 sec 9.54 MBytes 80.0 Mbits/sec 0 909 KBytes
[ 5] 2.00-3.00 sec 10.0 MBytes 83.9 Mbits/sec 0 1.34 MBytes
[ 5] 3.00-4.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.77 MBytes
[ 5] 4.00-5.00 sec 10.0 MBytes 83.9 Mbits/sec 0 2.23 MBytes
[ 5] 5.00-6.00 sec 10.0 MBytes 83.9 Mbits/sec 0 2.55 MBytes
[ 5] 6.00-7.00 sec 8.75 MBytes 73.4 Mbits/sec 0 2.55 MBytes
[ 5] 7.00-8.00 sec 10.0 MBytes 83.9 Mbits/sec 0 2.55 MBytes
[ 5] 8.00-9.00 sec 10.0 MBytes 83.9 Mbits/sec 0 2.55 MBytes
[ 5] 9.00-10.00 sec 10.0 MBytes 83.9 Mbits/sec 0 2.55 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 98.7 MBytes 82.8 Mbits/sec 0 sender
[ 5] 0.00-10.03 sec 97.9 MBytes 81.9 Mbits/sec receiver
5GHz Wi-FI / Raspberry Pi 4 => Workstation
Connecting to host 192.168.2.202, port 5201
[ 5] local 192.168.3.101 port 35768 connected to 192.168.2.202 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 8.53 MBytes 71.6 Mbits/sec 0 163 KBytes
[ 5] 1.00-2.00 sec 8.39 MBytes 70.4 Mbits/sec 0 171 KBytes
[ 5] 2.00-3.00 sec 8.58 MBytes 71.9 Mbits/sec 0 180 KBytes
[ 5] 3.00-4.00 sec 8.33 MBytes 69.9 Mbits/sec 0 180 KBytes
[ 5] 4.00-5.00 sec 8.45 MBytes 70.9 Mbits/sec 0 180 KBytes
[ 5] 5.00-6.00 sec 8.58 MBytes 71.9 Mbits/sec 0 180 KBytes
[ 5] 6.00-7.00 sec 8.64 MBytes 72.5 Mbits/sec 0 180 KBytes
[ 5] 7.00-8.00 sec 8.58 MBytes 71.9 Mbits/sec 0 180 KBytes
[ 5] 8.00-9.00 sec 8.39 MBytes 70.4 Mbits/sec 0 180 KBytes
[ 5] 9.00-10.00 sec 8.70 MBytes 73.0 Mbits/sec 0 259 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 85.2 MBytes 71.4 Mbits/sec 0 sender
[ 5] 0.00-12.24 sec 84.6 MBytes 58.0 Mbits/sec receiver
Comparison
We see a clear pattern: Using the 5GHZ network improves data transmission rate from 18Mbit/sec to 58.0 Mbit/sec. When sending data from the Linux Workstation to the Raspberry Pi, another 25 Mbits/sec are added.
Conclusion
In order to detect the root cause of slow image data transmission between a Raspberry Pi and a Linux workstation, I did three sets of measurements with different parameters: a) Using a Raspberry Pi Model 3B+ and Raspberry Pi Model 4, b) running the Raspberry Pi with either Raspberry Pi OS 2021-05-07 or Ubuntu Server 20.04, and c) Wi-Fi connection to a 2.4 Ghz and 5 Ghz network. The first measurement was done with the online tool Speedtest, and it showed that my Raspberry Pi 3B+ achieves the maximum upload speed of 40MB/s. The second measurements used the tool iperf to compare streaming data between two Raspberry Pi and/or a Linux workstation. I was shocked to see that Wi-Fi connections were limited to 18 MB/s. The final measurement just switched from a general router to a dedicated access point. Using the 5GHz Wi-Fi, I could achieve 58.0 MB/s. These results show that my router is just very busy with managing the network traffic of all other devices in my home, and that adding a dedicated access points removes this burden.