update readme, add ir threshold

Author: stytim

README.md

@@ -1,21 +1,21 @@
 # IR Retro-reflective Marker Tracking with Intel RealSense
 
-[![GitHub release](https://img.shields.io/github/release/stytim/RealSense-ToolTracker.svg?style=flat-square)](https://github.com/stytim/RealSense-ToolTracker/releases)
+[![GitHub release](https://img.shields.io/github/release/stytim/AzureKinect-ToolTracker.svg?style=flat-square)](https://github.com/stytim/AzureKinect-ToolTracker/releases)
 
-This project leverages the versatility of Intel RealSense cameras, enabling high-quality tracking of passive IR sphere markers without the need for large and expensive equipment like NDI trackers.
+This project leverages the versatility of Microsoft Azure Kinect cameras, enabling high-quality tracking of passive IR sphere markers without the need for large and expensive equipment like NDI trackers.
 
 <p align="center">
 	<img width="80%" src="image/overview.gif">
 </p>
 
 
 ## Project Highlights
-* **Compact and Cost-Effective**: Break free from the constraints of traditional tracking systems. Our project utilizes the small, affordable Intel RealSense camera for efficient tracking.
+* **Compact and Cost-Effective**: Break free from the constraints of traditional tracking systems. Our project utilizes the small, affordable Azure Kinect camera for efficient tracking.
 * **Versatile and User-Friendly**: Efficiently handle simultaneous tracking of various markers. Enjoy robust features such as multiple marker tracking, occlusion resistance, and support for various marker types and sizes.
 * **Smooth and Stable Tracking**: Integrate Kalman and low-pass filters for stable tracking.
 * **Enhanced Communication and Compatibility**: Utilize UDP messaging for transmitting tracking results and support for NDI .rom files
 * **Simple Calibration**: A simple marker array calibration is provided if the marker configuration is not known beforehand
-* **Cross-Platform Availability**: Works across Linux, Windows, and MacOS environments.
+* **Cross-Platform Availability**: Works across Linux and Windows environments.
 
 ## Installation
 **Quick Start with Precompiled Binary (Windows):** Download the latest precompiled binary from our releases section for immediate use.
@@ -24,14 +24,14 @@ This project leverages the versatility of Intel RealSense cameras, enabling high
 
 ## Prerequisites
 
-* Intel RealSense SDK (MacOS users, please refer to the FAQ for installation guidance)
+* Azure Kinect SDK
 * OpenCV 4
 * CMake 
 
 ## Building
 Ensure all the prerequisites are installed before proceeding with the build process.
 
-### For Linux and MacOS:
+### For Linux:
 
 ```bash
 mkdir build && cd build
@@ -49,32 +49,10 @@ Note: I have only tested this build process with Visual Studio 2022.
 ```bash
 ./ir-tracking-app
 ```
-### For MacOS
-Due to certain permissions and security features in MacOS, you might need to run the application with elevated privileges.
+### Offline Processing from Recording
 ```bash
-sudo ./ir-tracking-app
+./ir-tracking-app -i path_to_recording.mkv
 ```
-## RealSense Camera Modification: Adding a Light Diffuser
-
-The laser projector of the RealSense camera emits a sharp, focused IR dot pattern. While this is generally beneficial for depth sensing, it is not ideal for doing thresholding on IR stream to find retroreflective surfaces.
-
-Therefore, a crucial modification is required: the addition of a physical light diffuser filter in front of the camera's laser projector. This softens the laser dot pattern in the IR stream. 
-    <table>
-        <tr>
-            <td align="center" width="45%">
-                <img src="image/light_diffuser.jpg" alt="RealSense with light diffuser"><br>
-                RealSense with Light Diffuser
-            </td>
-            <td align="center" width="45%">
-                <img src="image/no_diffuser.png" alt="Without Light Diffuser"><br>
-                Without Light Diffuser, the visible dot pattern can interfere with detecting retroreflective surfaces in the IR stream.
-            </td>
-        </tr>
-    </table>
-</p>
-
-
-
 
 ## Usage Guide
 
@@ -101,7 +79,14 @@ Upon launching the RealSense Tool Tracker, the application will attempt to load
 * Transmission Rate: Confirm the "Frequency" of tracking updates to suit your needs.
 * UDP Enable: Ensure the "UDP" checkbox is selected to activate network transmission.
 
-### 3. Start Tracking
+### 3. Save Tracking Results Locally to CSV
+
+* Rate: Input the "Frequency" of writing the CSV file.
+* Duration: Input how long in seconds to save the file
+* Save Location: Click "Save To" to select the name and location for the CSV file.
+* Record Enable: Ensure the "Record" checkbox is selected to activate recording to file.
+
+### 4. Start Tracking
 * Initiate Tracking: With all parameters set, initiate tracking by clicking "Start Tracking."
 * Real-Time Adjustments: You can tweak the Tracking Parameters in real-time to adapt to varying conditions.
   - Infrared Sensitivity: Adjust the "IR Threshold" to optimize infrared tracking sensitivity.
@@ -115,17 +100,9 @@ A sample Python and a C# Unity scripts is provided for receiving and processing
 Contributions are welcome! When submitting a pull request, please include a description of your improvements and reference any relevant issue numbers.
 
 ## Frequently Asked Questions (FAQs)
-### Can I use Azure Kinect Cameras for this project?
+### Can I use Intel RealSense Cameras for this project?
 
-Currently, Azure Kinect Cameras are not supported due to an issue with marker overexposure, which affects the retrieval of valid depth values. For more insights, please refer to the discussion on [Azure-Kinect-Sensor-SDK Issue #1349](https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1349)
-
-### I'm encountering issues while installing the Intel RealSense SDK on MacOS. What should I do?
-
-Installation issues on MacOS can be tricky due to Intel dropping support for newer MacOS and Mac with Apple silicon. I recommend this comprehensive guide on [setting up Intel RealSense on MacOS](https://lightbuzz.com/realsense-macos/). 
-
-### What are the future plans for this project?
-
-My next big goal is to implement support for a multi-camera setup, enabling marker tracking over larger areas. However, as this project is a part-time endeavor, progress might be gradual. Stay tuned for updates!
+Yes, please refer to the [RealSense Tool Tracker](https://github.com/stytim/RealSense-ToolTracker)
 
 
 ## License and Citation
@@ -157,9 +134,8 @@ A significant portion of the tracking code is adapted from Andreas Keller's work
 ```
 
 
-
 <picture>
-  <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=stytim/RealSense-ToolTracker&type=Date&theme=dark" />
-  <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=stytim/RealSense-ToolTracker&type=Date" />
-  <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=stytim/RealSense-ToolTracker&type=Date" />
+  <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=stytim/Azure-ToolTracker&type=Date&theme=dark" />
+  <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=stytim/AzureKinect-ToolTracker&type=Date" />
+  <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=stytim/AzureKinect-ToolTracker&type=Date" />
 </picture>

include/IRToolTrack.h

@@ -102,7 +102,7 @@ class IRToolTracker
 
 	IRToolTracking* m_pRealSenseToolTracking;
 
-	uchar m_Threshold = 100;
+	ushort m_Threshold = 3000;
 	int m_MinSize = 10;
 	int m_MaxSize = 180;
 

include/IRToolTracking.h

@@ -120,7 +120,7 @@ class IRToolTracking {
 
     // int frame_width = 848;
     // int frame_height = 480;
-    int irThreshold = 100;
+    int irThreshold = 3000;
     int minSize = 10;
     int maxSize = 280;
 

include/ViewerWindow.h

@@ -79,10 +79,8 @@ class ViewerWindow {
     std::string recordedFile = "";
 
     IRToolTracking tracker;
-    int irThreshold = 100;
-    int laserPower = 300;
-    int minlasPower = 0;
-    int maxlasPower = 360;
+    int irThreshold = 3000;
+
     int minPixelSize = 10;
     int maxPixelSize = 300;
     bool isToolAdded = false;

src/IRToolTrack.cpp

@@ -620,21 +620,16 @@ void IRToolTracker::regionGrowing(const cv::Mat &depthImage, const std::vector<c
 }
 
 bool IRToolTracker::ProcessFrame(AHATFrame* rawFrame, ProcessedAHATFrame &result) {
-	uchar lowerLimit = m_Threshold;
-	uchar upperLimit = lowerLimit + 1;// 256 * 20;
+	//uchar lowerLimit = m_Threshold;
 	int minSize = m_MinSize, maxSize = m_MaxSize;
 	cv::Mat labels, stats, centroids;
 	std::vector<float> irToolCenters;
 
-	// rawFrame->cvAbImage.forEach<uchar>(
-	// 	[&](uchar& ir, const int* position) -> void {
-	// 		ir = (std::clamp(ir, lowerLimit, upperLimit) - lowerLimit) / (upperLimit - lowerLimit)*255;
-	// 	}
-	// );
+	std::cout << m_Threshold << std::endl;
 
 	// Threshold IR image
 	cv::Mat thresholdedIR;
-	cv::threshold(rawFrame->cvAbImage, thresholdedIR, 3000, 255, cv::THRESH_BINARY);
+	cv::threshold(rawFrame->cvAbImage, thresholdedIR, m_Threshold, 255, cv::THRESH_BINARY);
 
 	thresholdedIR.convertTo(rawFrame->cvAbImage, CV_8UC1);
 

src/IRToolTracking.cpp

@@ -58,10 +58,6 @@ void IRToolTracking::initialize(int index, int width, int height) {
         return;
     }
 
-
-    // SetThreshold(irThreshold);
-    // SetMinMaxSize(minSize, maxSize);
-
     if (K4A_RESULT_SUCCEEDED != k4a_device_open(index, &device))
     {
         std::cout << "Failed to open device" << std::endl;
@@ -80,6 +76,9 @@ void IRToolTracking::initialize(int index, int width, int height) {
         return;
     }
 
+	irThreshold = 3000;
+    SetThreshold(irThreshold);
+
     Terminated = false;
 }
 

src/ViewerWindow.cpp

@@ -686,7 +686,6 @@ void ViewerWindow::Render() {
                     else
                     {
                         tracker.initialize(selectedDeviceIndex, 848, 480);
-                        tracker.getLaserPower(laserPower, minlasPower, maxlasPower);
                     }
                     tracker.StartToolTracking();
                     irThreshold = tracker.GetThreshold();
@@ -708,20 +707,11 @@ void ViewerWindow::Render() {
         // Add a slider for setting the IR threshold
         ImGui::SetNextWindowPos(ImVec2(400, 20), ImGuiCond_FirstUseEver);
         ImGui::Begin("IR Threshold Control", nullptr, overlayFlags);
-        // if not on Mac
-        #if !defined(__APPLE__)
-        {
-            if(ImGui::SliderInt("Laser Power", &laserPower, minlasPower, maxlasPower))
-            {
-                tracker.setLaserPower(laserPower);
-            }
-        }
-        #endif
-        if(ImGui::SliderInt("IR Threshold", &irThreshold, 0, 255))
+        if(ImGui::SliderInt("IR Threshold", &irThreshold, 2000, 4000))
         {
             tracker.SetThreshold(irThreshold);
         }
-        ImGui::SetNextItemWidth(100);
+       /* ImGui::SetNextItemWidth(100);
         if (ImGui::InputInt("Min Px", &minPixelSize))
         {
             tracker.SetMinMaxSize(minPixelSize, maxPixelSize);
@@ -731,7 +721,7 @@ void ViewerWindow::Render() {
         if (ImGui::InputInt("Max Px", &maxPixelSize))
         {
             tracker.SetMinMaxSize(minPixelSize, maxPixelSize);
-        }
+        }*/
         ImGui::End();
 
         // Checkbox for enabling UDP