Project EagleEye: Joe Nasr research on Meta Quest 3, thermal imaging, LoRa/Meshtastic, gaze interaction and low-cost wearable mixed-reality systems.

1. Introduction

1.1 Background

Modern tactical awareness relies on two distinct data streams: geospatial coordination (Blue Force Tracking) and multispectral imaging (Thermal/Night Vision). Historically, integrating these capabilities into a single head-borne system required proprietary hardware costing upwards of $20,000 per unit, limiting access to elite military units.

1.2 Objective

This research aims to democratize this capability by leveraging the "Passthrough" API of modern Mixed Reality (MR) headsets. The objective is to overlay thermal data and team positions onto the user's real-world view, creating a functional prototype for search and rescue (SAR), tactical research, and off-grid coordination in cellular-denied environments.

2. System Architecture

The EagleEye system is comprised of four primary subsystems:

1. Visual Compute: Meta Quest 3 (Android-based VR/MR headset).
2. Communications: Heltec V3 (LoRa Mesh Radio).
3. Sensor: Topdon TC001 (LWIR Thermal Camera).
4. Software Core: ATAK-CIV (Android Team Awareness Kit) and a custom Unity-based OS layer.

3. Hardware Implementation

3.1 Component Selection

• Meta Quest 3: Selected for its high-fidelity color Passthrough and depth sensors, which allow digital overlays to "lock" to the physical world, minimizing motion sickness.
• Heltec WiFi LoRa 32 (V3): A low-power 915MHz/868MHz radio running Meshtastic firmware. It creates an off-grid mesh network for transmitting GPS coordinates without cellular infrastructure.
• Topdon TC001: A USB-C thermal camera originally designed for smartphones, providing 256x192 resolution thermal imaging.

3.2 Power and Data Topology

A critical engineering challenge is the Meta Quest 3’s single USB-C port. The system requires simultaneous power delivery (charging) and data ingestion (thermal camera). Passive splitters are insufficient.

Solution: An active USB-C Hub with Power Delivery (PD) Pass-Through is utilized.

3.3 Physical Assembly

To maintain ergonomic viability, the system uses a counter-weight distribution on a standard tactical "bump" helmet.

• Front: Headset and Thermal Camera (mounted via 3M Dual Lock tape).
• Rear: Battery and USB Hub (mounted via heavy-duty Velcro to the nape).
• Spine: Cables routed along the top helmet strap with strain-relief loops to prevent port damage.

4. Software Environment

The software architecture operates on two levels: standard Android app sideloading for geospatial data, and a custom Unity environment for hardware abstraction.

4.1 Android Manifest Override

Objective: Force USB Host permissions to allow the headset to read the external camera.

<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.EagleEye.Research">
    <!-- Force USB Host Mode -->
    <uses-feature android:name="android.hardware.usb.host" />
    <uses-permission android:name="android.permission.USB_PERMISSION" />
    <uses-permission android:name="android.permission.CAMERA" />
    
    <application android:label="EagleEye OS" android:theme="@android:style/Theme.Black.NoTitleBar.Fullscreen">
        <activity android:name="com.unity3d.player.UnityPlayerActivity" android:launchMode="singleTask">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
            <intent-filter>
                <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
            </intent-filter>
        </activity>
    </application>
</manifest>

4.2 Unity C# Script: Thermal Overlay

Objective: Read USB Camera feed into Texture.

using UnityEngine;
using UnityEngine.UI;

public class ThermalOverlay : MonoBehaviour {
    public RawImage overlayDisplay;
    private WebCamTexture thermalCam;

    void Start() {
        // Iterate through devices to find the Topdon/USB Camera
        foreach (var device in WebCamTexture.devices) {
            if (device.name.Contains("USB") || device.name.Contains("Topdon")) {
                thermalCam = new WebCamTexture(device.name, 256, 192, 25);
                overlayDisplay.texture = thermalCam;
                thermalCam.Play();
            }
        }
    }
}

5. Field Testing and Validation

5.1 Validation Protocol

Validation was conducted in three stages:

1. Bench Test: Verifying USB enumeration and charging status.
2. Range Test: Assessing LoRa mesh connectivity at 100m, 500m, and 1km intervals.
3. Calibration Walk: Ensuring the digital thermal overlay aligns with physical heat signatures.

5.2 Results

Latency: The USB hub introduces ~50ms of latency to the thermal feed. This is acceptable for observation but requires operator adaptation for rapid movement.

Runtime: The 20,000mAh power supply yields approximately 3-4 hours of continuous operation.

Signal Hygiene: To maintain Operational Security (OpSec), the Meshtastic radios were configured with AES256 encryption using the CLI command --set-chan psk random.

6. Bill of Materials (BOM)

The following table details the acquisition cost for a single unit (2024 Market Prices).

Comparative Analysis: The nearest military equivalent (IVAS 1.0) has a reported unit cost exceeding 80,000 AED, representing a cost reduction factor of >24x.

7. Conclusion

Project EagleEye successfully demonstrates that high-end situational awareness can be replicated using commodity hardware. While the user interface requires manual window management compared to a native military OS, the cost disparity and open-source flexibility make this a highly effective platform for research and non-standard tactical applications.

7.1 Future Work

Future iterations (v2.0) will focus on direct ESP32 integration of thermal sensors to reduce latency and the implementation of Shared Augmented Reality (SAR) to visualize team positions as 3D anchors in the real world.

8. References

1. Department of Defense (2022). TAK Server: Architecture and Deployment Guide. TAK Product Center.
2. Meshtastic Project (2024). Meshtastic Firmware Documentation: LoRa Mesh Networking. Retrieved from meshtastic.org.
3. Meta Developers (2023). Passthrough API and OpenXR Guidelines for Mixed Reality. developer.oculus.com.
4. Topdon Technology. TC001 Thermal Imaging Sensor SDK for Android.