Hey! That looks interesting I’ll check it out and add it to the list of topics to revisit.
Summarized in another comment but I am a bit worried about hand tracking for a couple reasons. Also mentioned in reply to another comment, but will summarize here for convenience as well!
1. While I know there is promising tech in this area (and I am very interested in it!) the tolerance for error while playing/performing music is slim to none. Will a solution end up being 75% accurate? 90%? Both are two low IMO. Pure MIDI keeps things in a relatively more controllable realm
2. Camera rigging. There are just so many variables here for things to go wrong/right. Quality of the camera, position, lighting. It’s challenging just finding a good method of positioning a camera even if you have the right equipment to locate it.
All that said, I’ll still reinvestigate this zone of ideas in the future and your insight is much appreciated! First, need to prove that I can make the simple case work as well as I think I can :)
You're right to be wary, I re-read your use case and hand tracking alone won't cut it. However I still think that hand tracking _combined_ with audio input can be precise enough (above 99%).
Regardless of the feasibility, it won't be as reliable as direct midi input, that's for sure! I think your goal of getting things working for a simpler case first is right, and wish you best luck :)
Summarized in another comment but I am a bit worried about hand tracking for a couple reasons. Also mentioned in reply to another comment, but will summarize here for convenience as well!
1. While I know there is promising tech in this area (and I am very interested in it!) the tolerance for error while playing/performing music is slim to none. Will a solution end up being 75% accurate? 90%? Both are two low IMO. Pure MIDI keeps things in a relatively more controllable realm
2. Camera rigging. There are just so many variables here for things to go wrong/right. Quality of the camera, position, lighting. It’s challenging just finding a good method of positioning a camera even if you have the right equipment to locate it.
All that said, I’ll still reinvestigate this zone of ideas in the future and your insight is much appreciated! First, need to prove that I can make the simple case work as well as I think I can :)