Your YOLO model is running on the Jetson and it finds the target in the camera frame at 15-20 FPS. Then you run into the next question: how do we get that detection to the flight controller? Who tells the aircraft to fly to those coordinates, hold there, and release the payload?

Our first instinct was "let's just put everything on the flight controller." The microcontroller on a Pixhawk-class board is already busy reading the IMU and computing motor outputs hundreds of times a second. Running a neural network there isn't possible, and even if you managed it, you would wreck the control loop.

The fix is to split the job in two: real-time control stays on the flight controller (FC), and the heavy computation moves to a second computer bolted next to it. That second computer is called a companion computer; ArduPilot defines it as a unit that connects to the FC over the MAVLink protocol and makes decisions during flight. This post covers the cable between those two boxes, the protocol that runs over it, and code that actually works.

Why the work is split this way

The flight controller's job is hard real-time: the stabilization loop is never missed, and there isn't even an operating system (ArduPilot runs on ChibiOS). The companion computer's job is "soft" real-time — processing a frame late delays the mission but doesn't drop the aircraft. That split is also a safety boundary: if the companion crashes, the FC keeps flying, failsafe kicks in, and the aircraft executes RTL (return to launch). The reverse is not true.

Typical companion jobs: object detection, gimbal control, mapping, streaming video to the ground station, and the high-level mission logic. On our SUAS 2026 aircraft the Jetson Orin NX 16GB fills that role: it pulls the RTSP stream from the SIYI A8 mini camera, runs detection with YOLO11m, and translates the result into a position command for the FC over MAVLink.

The physical link: UART, USB, Ethernet

There are three options, and for most teams the right answer is the first one.

In ArduPilot the serial ports map as SERIAL0 = USB, SERIAL1 = TELEM1, SERIAL2 = TELEM2, SERIAL3 = GPS. A SERIALx_PROTOCOL of 1 speaks MAVLink1 and 2 speaks MAVLink2 — use 2 for a companion. The SERIALx_BAUD parameter is the baud rate divided by a thousand: 57=57600, 115=115200, 921=921600.

The wiring is crossed: FC TX → companion RX, FC RX → companion TX, GND → GND. Pixhawk UARTs run at 3.3 V logic levels; don't wire a 5 V TTL board straight to them, and don't power the FC from the companion's USB.

On the Linux side, make sure the port isn't being used as a serial console. On a Raspberry Pi you turn off "login shell over serial" in raspi-config and leave the hardware UART enabled; the port then comes up as /dev/serial0. On Jetson it's usually /dev/ttyTHS1, and the nvgetty service has to be disabled.

MAVLink in three concepts

Messages. MAVLink is packet based. Every packet carries a System ID, a Component ID, a Message ID and the data fields; the maximum is 263 bytes in MAVLink1 and 280 bytes in MAVLink2. The vehicle defaults to system ID 1 (MAV_SYSID), and ground stations conventionally use 255.

Heartbeat. Once the link is up, each side sends a HEARTBEAT (message ID 0) at 1 Hz. It is both the "I'm alive" signal and the carrier for the vehicle's flight mode (custom_mode) and arm state. The first thing you do in code is wait for a heartbeat, because that's where you learn the target system number.

Stream rates. The FC doesn't push every message on its own; you have to say which ones you want and at what rate. The old approach is asking for groups of messages with REQUEST_DATA_STREAM. On ArduPilot 4.0 and later the preferred approach is requesting messages individually with MAV_CMD_SET_MESSAGE_INTERVAL (command ID 511) — the interval is in microseconds, so for 10 Hz you write 100000.

Commonly used messages and their units:

Connecting and reading telemetry with pymavlink

Install it with pip install pymavlink. The block below runs on its own; uncomment the UDP line and it connects to a SITL simulation instead.

import time
from pymavlink import mavutil

# Real hardware: Jetson UART to TELEM2
master = mavutil.mavlink_connection('/dev/ttyTHS1', baud=921600)
# Alternative for SITL / mavlink-router output:
# master = mavutil.mavlink_connection('udpin:0.0.0.0:14550')

# Wait for the first HEARTBEAT. target_system/target_component get filled in here.
master.wait_heartbeat()
print(f"Connected -> system: {master.target_system}, "
      f"component: {master.target_component}")

def set_message_rate(msg_id, hz):
    """Set the send rate of a single message (ArduPilot 4.0+)."""
    master.mav.command_long_send(
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_CMD_SET_MESSAGE_INTERVAL,  # 511
        0,                    # confirmation
        msg_id,               # param1: message ID
        int(1_000_000 / hz),  # param2: interval, MICROSECONDS
        0, 0, 0, 0, 0)

set_message_rate(mavutil.mavlink.MAVLINK_MSG_ID_GLOBAL_POSITION_INT, 10)
set_message_rate(mavutil.mavlink.MAVLINK_MSG_ID_ATTITUDE, 20)

t_end = time.time() + 10
while time.time() < t_end:
    msg = master.recv_match(type='GLOBAL_POSITION_INT',
                            blocking=True, timeout=2)
    if msg is None:
        print("No telemetry: check the baud rate, port and PROTOCOL settings")
        continue
    print(f"lat={msg.lat / 1e7:.7f} lon={msg.lon / 1e7:.7f} "
          f"rel_alt={msg.relative_alt / 1000.0:.1f} m "
          f"vz={msg.vz / 100.0:+.2f} m/s")

import time
from pymavlink import mavutil

# Real hardware: Jetson UART to TELEM2
master = mavutil.mavlink_connection('/dev/ttyTHS1', baud=921600)
# Alternative for SITL / mavlink-router output:
# master = mavutil.mavlink_connection('udpin:0.0.0.0:14550')

# Wait for the first HEARTBEAT. target_system/target_component get filled in here.
master.wait_heartbeat()
print(f"Connected -> system: {master.target_system}, "
      f"component: {master.target_component}")

def set_message_rate(msg_id, hz):
    """Set the send rate of a single message (ArduPilot 4.0+)."""
    master.mav.command_long_send(
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_CMD_SET_MESSAGE_INTERVAL,  # 511
        0,                    # confirmation
        msg_id,               # param1: message ID
        int(1_000_000 / hz),  # param2: interval, MICROSECONDS
        0, 0, 0, 0, 0)

set_message_rate(mavutil.mavlink.MAVLINK_MSG_ID_GLOBAL_POSITION_INT, 10)
set_message_rate(mavutil.mavlink.MAVLINK_MSG_ID_ATTITUDE, 20)

t_end = time.time() + 10
while time.time() < t_end:
    msg = master.recv_match(type='GLOBAL_POSITION_INT',
                            blocking=True, timeout=2)
    if msg is None:
        print("No telemetry: check the baud rate, port and PROTOCOL settings")
        continue
    print(f"lat={msg.lat / 1e7:.7f} lon={msg.lon / 1e7:.7f} "
          f"rel_alt={msg.relative_alt / 1000.0:.1f} m "
          f"vz={msg.vz / 100.0:+.2f} m/s")

recv_match blocks; if you don't want it stalling your main mission loop, read telemetry on a separate thread and write it into a shared dictionary. That's exactly what we do on the Jetson: the detection loop and the MAVLink loop run on separate threads.

Sending commands: changing mode and flying to a position in GUIDED

GUIDED is the mode in which the vehicle follows position and velocity targets handed to it from outside. The ArduPilot documentation states explicitly that this mode can be used with a companion computer or Lua scripts. The prerequisites are a GPS lock and — if you are commanding targets on the vertical axis — the vehicle already being airborne.

POS_ONLY = 0b110111111000  # use lat/lon/alt only; ignore velocity, acceleration, yaw

def set_mode(name, timeout=5):
    """Change mode and confirm it from the HEARTBEAT."""
    mode_id = master.mode_mapping()[name]   # e.g. 'GUIDED'
    master.mav.set_mode_send(
        master.target_system,
        mavutil.mavlink.MAV_MODE_FLAG_CUSTOM_MODE_ENABLED,
        mode_id)
    t_end = time.time() + timeout
    while time.time() < t_end:
        hb = master.recv_match(type='HEARTBEAT', blocking=True, timeout=1)
        if hb and hb.custom_mode == mode_id:
            return True
    return False

def arm(timeout=10):
    master.mav.command_long_send(
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_CMD_COMPONENT_ARM_DISARM,  # 400
        0, 1, 0, 0, 0, 0, 0, 0)                        # param1=1 -> arm
    ack = master.recv_match(type='COMMAND_ACK', blocking=True, timeout=timeout)
    return ack is not None and ack.result == mavutil.mavlink.MAV_RESULT_ACCEPTED

def goto(lat_deg, lon_deg, rel_alt_m):
    """Fly to a GPS target at an altitude relative to the launch point."""
    master.mav.set_position_target_global_int_send(
        0,                                   # time_boot_ms (unused)
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT_INT,
        POS_ONLY,
        int(lat_deg * 1e7), int(lon_deg * 1e7), float(rel_alt_m),
        0, 0, 0,        # vx, vy, vz   (ignored)
        0, 0, 0,        # afx, afy, afz (ArduPilot ignores these anyway)
        0, 0)           # yaw, yaw_rate

if set_mode('GUIDED') and arm():
    goto(39.9250000, 32.8360000, 25.0)   # swap in coordinates for your own field
else:
    print("Mode change or arm rejected — check the arming prerequisites")

POS_ONLY = 0b110111111000  # use lat/lon/alt only; ignore velocity, acceleration, yaw

def set_mode(name, timeout=5):
    """Change mode and confirm it from the HEARTBEAT."""
    mode_id = master.mode_mapping()[name]   # e.g. 'GUIDED'
    master.mav.set_mode_send(
        master.target_system,
        mavutil.mavlink.MAV_MODE_FLAG_CUSTOM_MODE_ENABLED,
        mode_id)
    t_end = time.time() + timeout
    while time.time() < t_end:
        hb = master.recv_match(type='HEARTBEAT', blocking=True, timeout=1)
        if hb and hb.custom_mode == mode_id:
            return True
    return False

def arm(timeout=10):
    master.mav.command_long_send(
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_CMD_COMPONENT_ARM_DISARM,  # 400
        0, 1, 0, 0, 0, 0, 0, 0)                        # param1=1 -> arm
    ack = master.recv_match(type='COMMAND_ACK', blocking=True, timeout=timeout)
    return ack is not None and ack.result == mavutil.mavlink.MAV_RESULT_ACCEPTED

def goto(lat_deg, lon_deg, rel_alt_m):
    """Fly to a GPS target at an altitude relative to the launch point."""
    master.mav.set_position_target_global_int_send(
        0,                                   # time_boot_ms (unused)
        master.target_system, master.target_component,
        mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT_INT,
        POS_ONLY,
        int(lat_deg * 1e7), int(lon_deg * 1e7), float(rel_alt_m),
        0, 0, 0,        # vx, vy, vz   (ignored)
        0, 0, 0,        # afx, afy, afz (ArduPilot ignores these anyway)
        0, 0)           # yaw, yaw_rate

if set_mode('GUIDED') and arm():
    goto(39.9250000, 32.8360000, 25.0)   # swap in coordinates for your own field
else:
    print("Mode change or arm rejected — check the arming prerequisites")

type_mask decides which fields are honored: a bit set to 0 means the field is used, a 1 means it is ignored. 0b110111111000 means "position only." Getting this mask wrong is the most common silent failure — the vehicle changes altitude but never budges horizontally.

A position target stays valid until the vehicle gets there. If you are sending velocity or acceleration targets, you have to send a fresh one within GUID_TIMEOUT (3 seconds by default), otherwise the vehicle slows down and stops. Horizontal speed is capped by WP_SPD (10 m/s by default).

The MAVROS and ROS 2 alternative

When pymavlink on its own isn't enough, a ROS layer comes into play.

ROS 1 reached end of life in 2025, so for new projects it makes sense to look at the ROS 2 side (as of 2026). We use Gazebo + SITL + ROS 2 in simulation, but the critical path of the flight software still talks pymavlink directly: fewer layers, fewer surprises.

Common pitfalls

• Leaving the serial console enabled. If Linux is running a login shell on /dev/serial0, boot messages get mixed into the MAVLink data and not a single packet parses.

• Opening one port from two applications. Only one process can hold a serial port at a time. If both your own code and MAVProxy or a ground control station need to connect, install mavlink-router and fan the port out to several UDP endpoints.

• Sending commands without calling wait_heartbeat(). Until then master.target_system is 0; your commands go nowhere and you don't even get an error.

• Mixing up units. Latitude/longitude in degE7 (an integer scaled by 10⁷), altitude in mm, velocity in cm/s, angles in radians, SET_MESSAGE_INTERVAL in microseconds. Get any of these confused and the resulting error is silent.

• Cranking stream rates all the way up. On a 57600 bps telemetry link, asking for every message at 50 Hz saturates the bandwidth and causes packet loss. Request only the messages you need, at the rate you actually need.

• Giving the companion the same system ID as the ground station. The companion's heartbeat can silence the GCS failsafe by making the FC think the link is still up; even if the ground station connection has genuinely dropped, failsafe never fires.

Safety notes

The FC can reject an arm command; that is not a bug, it is a protection layer. The ARMING_CHECK checks cover areas like GPS quality, EKF consistency, compass health and battery state. Your code should inspect the COMMAND_ACK result and refuse to start the mission when the command is rejected.

Think about GCS failsafe separately once a companion is in the picture: with FS_GCS_ENABLE on, the FC watches the heartbeat from the primary ground station (the system identified by MAV_GCS_SYSID) and triggers failsafe after FS_GCS_TIMEOUT (5 seconds by default) of silence. Give your companion a different system ID so that protection keeps working.

Finally, the pilot's mode switch must always be able to override the companion's commands. Behind our test of detecting two targets from ~20 m and releasing autonomously sit weeks of SITL repetitions; no piece of code ever ran for the first time out in the field.

Practical summary

1. Use TELEM2: SERIAL2_PROTOCOL=2, SERIAL2_BAUD=921, wiring crossed TX↔RX, common GND. Don't trust USB in flight.

2. Call wait_heartbeat() on every connection, then request only the messages you need with MAV_CMD_SET_MESSAGE_INTERVAL.

3. Verify your commands: confirm mode changes with HEARTBEAT.custom_mode, and arm plus other commands with COMMAND_ACK.result. "I sent it" and "it was accepted" are not the same thing.

4. Double-check type_mask and the units — degE7, mm, cm/s, radians, microseconds.

5. Keep failsafe working in spite of the companion: use a separate system ID, never block pilot intervention, and rehearse the scenario in SITL before you head to the field.

Sources: ArduPilot dev/copter documentation (Companion Computers, Telemetry Port Setup, MAVLink Basics, Requesting Data, Guided Mode, Network Setup) and the mavlink.io common message definitions.