پديد آورندگان :
عظيمي فرد، آرش دانشگاه صنعتي خواجه نصيرالدين طوسي - دانشكده مهندسي ژئودزي و ژئوماتيك - گروه فتوگرامتري و سنجش از دور، تهران، ايران , حسيني نوه احمدآباديان، علي دانشگاه صنعتي خواجه نصيرالدين طوسي - دانشكده مهندسي ژئودزي و ژئوماتيك - گروه فتوگرامتري و سنجش از دور، تهران، ايران
كليدواژه :
اودومتري بينايي , اسلم بينايي , فتوگرامتري برد كوتاه , انتخاب فريم هاي كليدي , قيود هندسي , حد آستانه انطباقي
چكيده فارسي :
به‌دليل پيچيدگي‌هاي پردازش فريم براي تعيين موقعيت و تهيه نقشه در الگوريتم‌هاي ماشين بينايي و فتوگرامتري، روش‌هاي انتخاب فريم‌هاي كليدي به‌منظور افزايش كارايي الگوريتم‌ها معرفي شدند كه در عين حفظ دقت و استحكام الگوريتم، حجم پردازش‌ها را كاهش مي‌دهند. يكي از معروف‌ترين الگوريتم‌هاي تعيين موقعيت و تهيه نقشه هم‌زمان مبتني بر تصوير (ويژوال اسلم)، الگوريتم ORB-SLAM3 [1] است. انتخاب فريم كليدي در اين الگوريتم و ساير الگوريتم‌هاي اين حوزه وابسته به حد آستانه‌هاي ابتكاري است. در اين مقاله يك روش هندسي و بر پايه اصول طراحي شبكه تصويربرداري در فتوگرامتري به‌منظور انتخاب فريم‌هاي كليدي در بهبود الگوريتم ORB-SLAM3 پيشنهاد شده است. در اين روش، حد آستانه‌هاي ابتكاري با اصول فتوگرامتري جايگزين شده است كه علاوه بر استحكام الگوريتم، كيفيت ابر نقطه حاصل از فريم‌هاي كليدي را تضمين مي‌كند. در روش پيشنهادي، ابتدا يك حد آستانه انطباقي در مورد مجاز بودن تعداد نقاطي كه ناحيه مخروطي خط ديد آن‌ها در يك مخروط چهار ناحيه‌اي تشكيل‌شده بر روي هر نقطه، تغيير كرده است، تصميم مي‌گيرد. سپس با تشكيل يك شبكه 3 در 3 در هر فريم و شمارش نقاط مؤثر در هر سلول اين شبكه، معيار تعادل مركز ثقل (ECOG) [2] در مورد مناسب بودن توزيع نقاط داخل اين فريم تصميم مي‌گيرد. از طرف ديگر سنسور اينرسي[3] (IMU) در صورت مشاهده تغييرات شديد شتاب حركت، مستقل از دوربين اقدام به اخذ فريم كليدي مي‌كند. به‌منظور ارزيابي روش پيشنهادشده، آزمايش‌هاي وسيعي روي داده [4] EuRoC در حالت تك‌دوربينه و دو دوربين انجام شده است. ارزيابي‌هاي كيفي و كمي با مقايسه مسير رديابي شدۀ هر الگوريتم با مسير مرجع، مقايسه ابر نقطه تشكيل‌شده از فريم‌هاي كليدي و مقايسه مقدار خطاي مطلق مسير حركت[5] (ATE) انجام شده است. همچنين زمان اجراي هر الگوريتم براي تمامي دنباله تصاوير داده EuRoC ارزيابي شده است. نتايج نشان مي‌دهد، الگوريتم پيشنهادي در حالت دو دوربين 18.1% و در حالت تك‌دوربينه 20.4% دقت تعيين موقعيت ORB-SLAM3 را بهبود داده و علاوه بر اين ابر نقطه متراكم‌تري توليد كرده است.
چكيده لاتين :
Introduction
Due to the complexity of frame processing used for positioning and mapping in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms, key-frame selection methods have been introduced to improve the performance and decrease the number of frames required for processing while maintaining accuracy and robustness of the algorithms. Selected key-frames in these methods make a very good representation of all available frames. The current key-frame selection methods rely on heuristic thresholds in their selection procedure. Researchers have used several datasets to find optimum values for these thresholds through trial and error. In fact, proposed methods may not work as expected with a new dataset due to changes occurring in the sensor, environment and the platform.
Materials & Method
The present study has proposed an improved geometric and photogrammetric key-frame selection method built upon ORB-SLAM3, as the state of the art visual SLAM algorithm. The proposed Photogrammetric Key-frame Selection (PKS) algorithm has replaced inflexible heuristic thresholds with photogrammetric principles and thus guaranteed the robustness of the algorithm and the quality of the point cloud obtained from the key-frames. First, an adaptive threshold decides the allowable number of points whose line of sight zone has changed on a four-zone cone built upon each point. Increased number of points whose line of sight zone has changed means increased changes and displacements of the frame and thus, increased need for a new key-frame. Then, a 3*3 grid was formed in each frame and the number of points with a more than 30-degree change in line of sight angle (effective points) in each cell were counted. Later, the Equilibrium of Center Of Gravity (ECOG) criterion decides whether the distribution of points is appropriate using the center of gravity of the points inside the frame. Appropriate distribution of effective points within the frame shows a high geometric strength and thus will improve the strength of key-frames network. IMU sensor is not dependent on the position of the frames and the camera sensor. Thus, it independently obtains the key-frame in case significant changes occur in acceleration. The threshold value of acceleration has been experimentally considered equal to 1 meter per square second, which entirely depends on the type of robot. For ground robots with slower moving speeds, this threshold must be reset.
Results & Discussion
The present study has employed data collected by the European Robotics Challenge (EuRoC) flying robot containing the information collected by the synchronized camera and IMU information, as well as the ground truth data such as the robot trajectory and point cloud formed by the laser scanner. To evaluate the proposed method, extensive experiments have been implemented on the EuRoC dataset in mono-inertial and stereo-inertial modes. Then, trajectory of each algorithm was compared with the reference trajectory and point clouds formed by the key-frames were also compared. Apart from these qualitative evaluations, absolute trajectory error (ATE) obtained from running the PKS and ORB-SLAM3 algorithm 10 times were also compared quantitatively and finally, the error histogram was used to evaluate the point clouds. The processing time of each algorithm was also evaluated for each EuRoC dataset sequence. Results indicated that the proposed algorithm has improved ORB-SLAM3 accuracy in stereo-inertial by 18.1% and in the mono-inertial mode by 20.4% producing a more complete and accurate point cloud and thus, extracting more details from the environment. Furthermore, despite higher density of the point cloud, the error histogram has not changed significantly and fewer errors were observed in the ORB-SLAM3 algorithm.
Conclusion
Findings indicated that the PKS method has succeeded in extracting key-frames using photogrammetric and geometric principles. Apart from improving the positioning accuracy of the robot, the method has produced a much more complete and dense point cloud as compared to the ORB-SLAM3 algorithm. Also, dependency of the PKS method on the environment conditions and the type of system used (stereo camera or mono camera) was greatly reduced. Future studies can expand our key-frame selection method to include fisheye cameras or visual-only systems. More geometric conditions (near and far point condition and the vertex angle in the triangle formed by the points in the current frame, the camera and the corresponding points in the last key-frame) can also be added to the key-frame selection method.
