Sailboat Control featuring a 3D Augmented Reality Interface

I In order to safely operate sailboats, captains often rely on proper interpretation of several marine aspects to make decisions. In this project, we have developed an Augmented Reality System (ARS) to provide captains of sailboats with a centralized sensor data server and a visualization method. We have partially deployed this system on our research vessel, SV Moonshadow, seen below. In particular small sailing vessels are sometimes sailed by the captain alone. At the same time there are a large number of data inputs such as wind, tide, weather, position, and presence of obstacles such as trees or kelp that have to be considered to choose the proper course of action. We introduce a visualization tool that provides an interface for representing a wide spectrum of relevant marine data. The interface relies on a real-time data server that provides information about the status of the vessel (wind, GPS, gyro, accelerometer, depth sounder etc.)

Real Time Sensor Data Processing
The vessel collects and processes real time sensor data from:
  • GPS
  • depth sounder
  • magnetometer
  • gyroscope
  • accelerometer
  • anemometer
  • barometer
  • thermometer
  • knotmeter
Augmented Reality Wind Visualization System
The visualization system receives real-time data and renders it in different forms as part of the following layers:
  • background (3D, both in the visible spectrum and infra-red)
  • debris layer (3D highlighted detected debris)
  • wind animation using a particle system
  • current animation
  • course estimation
  • technical overlay
  • numeric overlay
  • barometric tendency monitor
  • depth tendency monitor
Real Time Debris Detection
For visual recognition, there is a system of cameras installed. The data is processed in real time and potential hazzards are highlighted (logs, crab traps, buoys etc.) Upon detection, evasive manouvers are first recommended and ultimately executed automatically.
Augmented reality visualization system
Technical overlay
SV Moonshadow in Hartley Bay