Sensors¶

AC functions on a unique simple moncoular camera.

We decided to take up the challange to build the entire sensing system to boost the accessibility and the portability of the sensing system. Through the raster feed from the camera, AC is capable of providing all the computed information necessary for the fabrication. Plus, it is compact, lightweight and easy to install on any tool. Finally, we believe that this can ensure a better future-proofing of any AR system.

For the AC prototype we integrated a monocular camera specifically developed for drone applications: RunCam 2 4K¹. Here are the features of the camera we were interested in given the constraints during the fabrication. If you pick another camera we recommend you to pick one with similar specifics:

Resolution (HD)

The hgiher the resolution the more refined and accurate your computed information will be. Nonetheless, the higher the resolution the more computational power you will need to process the feed. We recommend a resolution of 1280x720 pixels.

Frame rate (30 fps)

As long as it does not drop below ~23 fps human eye cannot catch the difference. Nevertheless, this value important to have a smooth tracking and a reactive system.

Large field of view (170°)

A large fish-eye lens is recommended to have a large field of view. This is important to have a better understanding of the environment and to have a better tracking of the tool at the same time. The larger the view the more information e.g., pixels, to work with.

Low weight (49g)

The lighter the camera the less it will impact the tool balance and the less it will be a burden for the user of course.

Shock and vibration resistant

The tool is a vibrating machine and the camera will be exposed to a lot of vibrations. We found monocular sensors designed for drones to be the most adapted to this scenario.

Capture distance

This is a key feature that monocular camera offers: they are able to capture information about both close and far subjects at once. Range finding or laser-based systems although providing distance information, often struggle to provide information about close subjects and their capture distance is very high.

Calibration¶

No matter the monocular camera you choose, you will need to calibrate it. A calibration is a process to determine the intrinsic and extrinsic parameters of the camera and it undistort the image feed. This is a fundamental step to ensure the accuracy of the system. The better the calibration, the better the precision you will get from the AR system.

Before you start print this chessboard and follow the instructions below.

git clone https://github.com/ibois-epfl/rgb-camera-calibration.git

pip install opencv-python
cd rgb-camera-calibration
python ./capture_img.py -I 0

python ./calibrate_camera.py -H 19 -V 13 -S 20

AC reads and saves the calibration in a .yml file. Here is an example of the calibration file.

%YAML:1.0
---
image_width: 1280
image_height: 720
camera_matrix: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 562.120361, 0., 640.797729, 0.,
       562.737671, 346.069397, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data: [ -0.25688748402037903, 0.089453678964044672,
       -0.00030771674818804194, 0.00017539880556521496,
       -0.016307368788539566 ]

The calibration file can be found and set manually from the config.ini file.

[AIAC]
ACInfoModel = assets/ACModel/test.acim
AlignFlip = 0
AlignOffset = 0
AlignRotation = 0
CamFlipHorizontal = 0
CamFlipVertical = 0
CamID = 0
CamParamsFile = assets/tslam/calibration_orange_A_1280_720_000B.yml
SaveDirMaps = 5
ScaleFactor = 50
ScannedModel = ./scanned_map/map-2024-08-19-15-43-17.ply

[TSlam]
MapFile = scanned_map/map-2024-08-19-15-43-17.map
ReconstructConfigDefaultFile = assets/tslam/reconstruct_default.ini
SaveDirMaps = scanned_map/
VocFile = assets/tslam/orb.fbow

[TTool]
CachedToolhead = auger_drill_bit_20_235
ConfigFile = deps/TTool/assets/config.yml
DatasetDir = deps/TTool/assets/toolheads
TToolRootPath = deps/TTool

[TouchMonitorSpecs]
LinkMode = HDMI
Name = WaveShare WS170120
Resolution = 1024x600

[Utils]
UtilsPath = ./temp

Components list¶