Need help with Hacking the MS Kinect.
Patrick, Fri Feb 17 2012, 02:13AMim researching the MS Kinect device, to give my flying bot some vision. But i dont know how or if the kinect can be run via microcontroller or if it has to berun from a X86 type lame machine...? id like to get it running off a embedded microprocessor if possible.
EDIT: so far the web sources dont seem all that great, if some one knows of a good credible source please post it here.
im searching diydrones too.
So far, it looks like it has to report to a conventional computer, and im not wanting to drag one of those around the sky.
Re: Need help with Hacking the MS Kinect.
Carbon_Rod, Fri Feb 17 2012, 03:21AM
OpenNI PPA built lib (should have the Git url for local builds):
demo:
ROS module:
Built info for 1.15GHz SheevaPlug ARM board:
Note the stability issues some people have reported:
Low level driver lib with SLAM support:
People normally use a powerful computer to run this type of mapping system.
However, some simply relay the streams of information over a light-weight wireless-USB-cable-adapter or router (about 100m range). Few routers will natively support these types of USB media devices, but writing a client proxy stream server should be easy.
Note a used Dual-core mini-Netbook with a broken LCD is very inexpensive...
Best of luck,
Carbon_Rod, Fri Feb 17 2012, 03:21AM
OpenNI PPA built lib (should have the Git url for local builds):
demo:
ROS module:
Built info for 1.15GHz SheevaPlug ARM board:
Note the stability issues some people have reported:
Low level driver lib with SLAM support:
People normally use a powerful computer to run this type of mapping system.
However, some simply relay the streams of information over a light-weight wireless-USB-cable-adapter or router (about 100m range). Few routers will natively support these types of USB media devices, but writing a client proxy stream server should be easy.
Note a used Dual-core mini-Netbook with a broken LCD is very inexpensive...
Best of luck,
Re: Need help with Hacking the MS Kinect.
Patrick, Fri Feb 17 2012, 03:41AM
Youre links are great !!!
small objects seen on youtube.
SLAM seen on youtube.
Patrick, Fri Feb 17 2012, 03:41AM
Youre links are great !!!
Carbon_Rod wrote ...
People normally use a powerful computer to run this type of mapping system.
However, some simply relay the streams of information over a light-weight wireless-USB-cable-adapter or router (about 100m range). Few routers will natively support these types of USB media devices, but writing a client proxy stream server should be easy.
Note a used Dual-core mini-Netbook with a broken LCD is very inexpensive...
yeah but my professors think its "lame" to strap a laptop to a machine needing some computational ability... but wifi is what everyone else uses to link the drone to the desktop. id rather use my STM32F4 if possible.People normally use a powerful computer to run this type of mapping system.
However, some simply relay the streams of information over a light-weight wireless-USB-cable-adapter or router (about 100m range). Few routers will natively support these types of USB media devices, but writing a client proxy stream server should be easy.
Note a used Dual-core mini-Netbook with a broken LCD is very inexpensive...
small objects seen on youtube. SLAM seen on youtube.Re: Need help with Hacking the MS Kinect.
Carbon_Rod, Fri Feb 17 2012, 04:30AM
You are likely going to have to get something a little more powerful if a 1.15GHz chip is choking on the bandwidth. Your board is perfect for control problems, but will be unlikely to handle the data.
I have seen quite a few people get caught by the build-it-from-scratch curse.
Some students end up trying to fix driver issues, erratic hardware glitches, power failures, and non-repairable system failures.
These guys made it work, but fell for the "easy" sonar scam:
Papers:
LOL... seems like they did it the easy way too...
Here are the ROS packages in the videos (note link 1 uses ground estimation so in theory an IMU is secondary):
cheers,
Carbon_Rod, Fri Feb 17 2012, 04:30AM
You are likely going to have to get something a little more powerful if a 1.15GHz chip is choking on the bandwidth. Your board is perfect for control problems, but will be unlikely to handle the data.
I have seen quite a few people get caught by the build-it-from-scratch curse.
Some students end up trying to fix driver issues, erratic hardware glitches, power failures, and non-repairable system failures.
These guys made it work, but fell for the "easy" sonar scam:
Papers:
LOL... seems like they did it the easy way too...
Here are the ROS packages in the videos (note link 1 uses ground estimation so in theory an IMU is secondary):
cheers,
Re: Need help with Hacking the MS Kinect.
Patrick, Fri Feb 17 2012, 06:11AM
im not wanting to build it entirely from scratch, im willing to use existing libraries for C and C++.
(PS -> the unexpected ultrasonic " departure, resulting in fatal deceleration " did cost me $190 US for new gyros.)
Patrick, Fri Feb 17 2012, 06:11AM
Carbon_Rod wrote ...
You are likely going to have to get something a little more powerful if a 1.15GHz chip is choking on the bandwidth. Your board is perfect for control problems, but will be unlikely to handle the data.
I have seen quite a few people get caught by the build-it-from-scratch curse.
Some students end up trying to fix driver issues, erratic hardware glitches, power failures, and non-repairable system failures.
These guys made it work, but fell for the "easy" sonar scam:
Papers:
LOL... seems like they did it the easy way too...
Here are the ROS packages in the videos (note link 1 uses ground estimation so in theory an IMU is secondary):
cheers,
LOL !!! you keep calling Sonar a "SCAM" lol!!! ok its error prone, but not uselesss dam it, and i already spent my pizza money!You are likely going to have to get something a little more powerful if a 1.15GHz chip is choking on the bandwidth. Your board is perfect for control problems, but will be unlikely to handle the data.
I have seen quite a few people get caught by the build-it-from-scratch curse.
Some students end up trying to fix driver issues, erratic hardware glitches, power failures, and non-repairable system failures.
These guys made it work, but fell for the "easy" sonar scam:
Papers:
LOL... seems like they did it the easy way too...
Here are the ROS packages in the videos (note link 1 uses ground estimation so in theory an IMU is secondary):
cheers,
im not wanting to build it entirely from scratch, im willing to use existing libraries for C and C++.
(PS -> the unexpected ultrasonic " departure, resulting in fatal deceleration " did cost me $190 US for new gyros.)
Re: Need help with Hacking the MS Kinect.
klugesmith, Sat Feb 18 2012, 12:22AM
Just a couple of months ago, I saw a lecture by JDSU about the vision system they developed for MS Kinect. Perhaps the first commercial application for "gesture recognition" technology.
IIRC, to track a user's hands against a cluttered background, they illuminate the scene with the Kinect's own IR laser. Distance detection is based not on light transit time, but on patterned illumination and parallax measurement. Not sure how much of the image processing depends on dedicated hardware.
At distances greater than a few meters, it might be no more useful than a plain camera.
klugesmith, Sat Feb 18 2012, 12:22AM
Patrick wrote ...
im researching the MS Kinect device, to give my flying bot some vision.
Please tell us what you learn. Is there a developer's kit available?im researching the MS Kinect device, to give my flying bot some vision.
Just a couple of months ago, I saw a lecture by JDSU about the vision system they developed for MS Kinect. Perhaps the first commercial application for "gesture recognition" technology.
IIRC, to track a user's hands against a cluttered background, they illuminate the scene with the Kinect's own IR laser. Distance detection is based not on light transit time, but on patterned illumination and parallax measurement. Not sure how much of the image processing depends on dedicated hardware.
At distances greater than a few meters, it might be no more useful than a plain camera.
Re: Need help with Hacking the MS Kinect.
Patrick, Sat Feb 18 2012, 05:28AM
i have not found anything as far as an official dev thingie. but ill report everything here, and yes the laser has a dot making filter over it. so yes it projects dots and then uses line-by-line scan comparisons. i wonder how much of the processing is done on the kinect.
im still researching it, optical flow processing is really complicated and im already thinking of graduate level ideas i want to explore over the next 3-5 years.
Patrick, Sat Feb 18 2012, 05:28AM
klugesmith wrote ...
Just a couple of months ago, I saw a lecture by JDSU about the vision system they developed for MS Kinect. Perhaps the first commercial application for "gesture recognition" technology.
IIRC, to track a user's hands against a cluttered background, they illuminate the scene with the Kinect's own IR laser. Distance detection is based not on light transit time, but on patterned illumination and parallax measurement. Not sure how much of the image processing depends on dedicated hardware.
At distances greater than a few meters, it might be no more useful than a plain camera.
Patrick wrote ...
im researching the MS Kinect device, to give my flying bot some vision.
Please tell us what you learn. Is there a developer's kit available?im researching the MS Kinect device, to give my flying bot some vision.
Just a couple of months ago, I saw a lecture by JDSU about the vision system they developed for MS Kinect. Perhaps the first commercial application for "gesture recognition" technology.
IIRC, to track a user's hands against a cluttered background, they illuminate the scene with the Kinect's own IR laser. Distance detection is based not on light transit time, but on patterned illumination and parallax measurement. Not sure how much of the image processing depends on dedicated hardware.
At distances greater than a few meters, it might be no more useful than a plain camera.
i have not found anything as far as an official dev thingie. but ill report everything here, and yes the laser has a dot making filter over it. so yes it projects dots and then uses line-by-line scan comparisons. i wonder how much of the processing is done on the kinect.
im still researching it, optical flow processing is really complicated and im already thinking of graduate level ideas i want to explore over the next 3-5 years.
Re: Need help with Hacking the MS Kinect.
Carbon_Rod, Sat Feb 18 2012, 11:42AM
The free MS SDK EULA basically states anything you create belongs to Microsoft:
Most people end up using open drivers, APIs, and high-level motion capture libraries:
Cheers,
Carbon_Rod, Sat Feb 18 2012, 11:42AM
The free MS SDK EULA basically states anything you create belongs to Microsoft:
Most people end up using open drivers, APIs, and high-level motion capture libraries:
Cheers,
Re: Need help with Hacking the MS Kinect.
Pinky's Brain, Sun Feb 19 2012, 03:00PM
Pinky's Brain, Sun Feb 19 2012, 03:00PM
Patrick wrote ...
im still researching it, optical flow processing
Don't get too seduced by the mathematical elegance of PDE based optical flow ... the underlying assumptions to allow that elegance are utter BS.im still researching it, optical flow processing
Re: Need help with Hacking the MS Kinect.
Tetris, Sun Feb 19 2012, 05:05PM
Someone hacked a MS Kinect, attached it to a controller which was attached to a Tesla Coil. The results were so cool, a person could move their body around, and control the sound of the coil.
Tetris, Sun Feb 19 2012, 05:05PM
Someone hacked a MS Kinect, attached it to a controller which was attached to a Tesla Coil. The results were so cool, a person could move their body around, and control the sound of the coil.
Re: Need help with Hacking the MS Kinect.
Forty, Wed Feb 29 2012, 03:42AM
That SLAM video looks like it's creating a minecraft level. anyways, what's the range on those sonar sensors? "altitude" detection isn't that impressive when it's only 1.5ft off the ground. Have they tried using only sonar to map out the environment (yes, like a bat) rather than hacking the kinect?
sorry if those are dumb questions, I haven't been following the uav developments for a few months because they always get me stuck on youtube for hours.
Forty, Wed Feb 29 2012, 03:42AM
That SLAM video looks like it's creating a minecraft level. anyways, what's the range on those sonar sensors? "altitude" detection isn't that impressive when it's only 1.5ft off the ground. Have they tried using only sonar to map out the environment (yes, like a bat) rather than hacking the kinect?
sorry if those are dumb questions, I haven't been following the uav developments for a few months because they always get me stuck on youtube for hours.
Re: Need help with Hacking the MS Kinect.
Carbon_Rod, Wed Feb 29 2012, 07:47AM
There is a Kinect script to import animated 3D captures into minecraft:
Carbon_Rod, Wed Feb 29 2012, 07:47AM
Forty wrote ...
That SLAM video looks like it's creating a minecraft level.
That SLAM video looks like it's creating a minecraft level.
There is a Kinect script to import animated 3D captures into minecraft:
Re: Need help with Hacking the MS Kinect.
Ben Solon, Thu Mar 01 2012, 01:32PM
there is an arduino usb host shild if that helps... http://www.sparkfun.com/products/9947 if you cant directly use the arduino, this thing is controlled via spi. you could of course reverse engeneer it too.
hth, ben
[Edit: Fixed link]
Ben Solon, Thu Mar 01 2012, 01:32PM
there is an arduino usb host shild if that helps... http://www.sparkfun.com/products/9947 if you cant directly use the arduino, this thing is controlled via spi. you could of course reverse engeneer it too.
hth, ben
[Edit: Fixed link]
Re: Need help with Hacking the MS Kinect.
Patrick, Sat Mar 03 2012, 07:52PM
my L1's and F4's all support on chip USB, (stm32) but the drivers are developement time killers.
Patrick, Sat Mar 03 2012, 07:52PM
ben123324 wrote ...
there is an arduino usb host shild if that helps... http://www.sparkfun.com/products/9947 if you cant directly use the arduino, this thing is controlled via spi. you could of course reverse engeneer it too.
hth, ben
[Edit: Fixed link]
This would be great if i could find an existing embeded library optical flow source code for the Kinect, id like to run it on a M4 Cortex, instead of the lame solution of straping a laptop on with duct tape.there is an arduino usb host shild if that helps... http://www.sparkfun.com/products/9947 if you cant directly use the arduino, this thing is controlled via spi. you could of course reverse engeneer it too.
hth, ben
[Edit: Fixed link]
my L1's and F4's all support on chip USB, (stm32) but the drivers are developement time killers.
Re: Need help with Hacking the MS Kinect.
Carbon_Rod, Sat Mar 03 2012, 11:16PM
"This would be great if i could find an existing embedded library optical flow source code for the Kinect, id like to run it on a M4 Cortex, instead of the lame solution of strapping a laptop on with duct tape."
Actually, a $60 dual core 2.x GHz CPU, CUDA (1.1+) enabled GPU, and 4GB ram is quite an acceptable alternative. Add an old $20 Buspirate dongle interface and the i2c/spi sensors are running the same day.... =P
Note, dumping the non-essential laptop parts on Ebay meant the students actually generated a net profit to purchase additional parts like bootable SD cards. These cards are slow when running Debian/Ubuntu 10.04, and start to burn-out after a few months of use.... However, they have less mass than solid state drives, can be legally cloned/replicated with G4L in under a few hours, and tend to survive rough landings.
I tend to recommend keeping the external monitor port, WiFi card (the antennas are usually in the LCD plastic bezel), and keyboard for special BIOS hot-keys if needed at some-point. Note that some laptop motherboards can have almost every peripheral cable removed, but will not continue to function if the docking port plug is unplugged. Often it is less weight to only keep the cooling fan, and custom make copper plate heat exchangers for the GPU, CPU, and MB bridge chips.
Although some people have attempted tapping into the high capacity laptop battery for drive (or on-board charger).... I have not seen anyone complete it successfully.
Note that the power management systems of Mobile chips can often misrepresent their computational performance. People should review the benchmark CPU performance prior to overpaying for a modern and often less computationally powerful system:
Trajectory planning quad rotors:
Best of luck,
Rod
Carbon_Rod, Sat Mar 03 2012, 11:16PM
"This would be great if i could find an existing embedded library optical flow source code for the Kinect, id like to run it on a M4 Cortex, instead of the lame solution of strapping a laptop on with duct tape."
Actually, a $60 dual core 2.x GHz CPU, CUDA (1.1+) enabled GPU, and 4GB ram is quite an acceptable alternative. Add an old $20 Buspirate dongle interface and the i2c/spi sensors are running the same day.... =P
Note, dumping the non-essential laptop parts on Ebay meant the students actually generated a net profit to purchase additional parts like bootable SD cards. These cards are slow when running Debian/Ubuntu 10.04, and start to burn-out after a few months of use.... However, they have less mass than solid state drives, can be legally cloned/replicated with G4L in under a few hours, and tend to survive rough landings.
I tend to recommend keeping the external monitor port, WiFi card (the antennas are usually in the LCD plastic bezel), and keyboard for special BIOS hot-keys if needed at some-point. Note that some laptop motherboards can have almost every peripheral cable removed, but will not continue to function if the docking port plug is unplugged. Often it is less weight to only keep the cooling fan, and custom make copper plate heat exchangers for the GPU, CPU, and MB bridge chips.
Although some people have attempted tapping into the high capacity laptop battery for drive (or on-board charger).... I have not seen anyone complete it successfully.
Note that the power management systems of Mobile chips can often misrepresent their computational performance. People should review the benchmark CPU performance prior to overpaying for a modern and often less computationally powerful system:
Trajectory planning quad rotors:
Best of luck,
Rod
Print this page