Welcome to Conjure, a controller-free system for controlling your computer! Requires an iPhone with FaceID.
- Mouse movement according to hand position, controlled using the depth and RGB sensors in your front-facing phone camera
- Specific gestures to control common inputs ("index finger" for left click, "peace" for right click, "pinch" to click and drag)
- Upload the ios-client to your iPhone by building through XCode
- Set up a Tailscale net on both your phone and computer
- Run the python server file on the computer you wish to control
Hand movement in the camera’s 2D projection of 3D space coincides with mouse movement across a computer’s monitor. Depth values relative to the camera's plane limit available actions. More details are available in the docs.
The following is a table detailing the correspondence between gestures and Conjure's actions:
| Area | Gesture | Action |
|---|---|---|
| Anywhere | Palm / Stop | Stop all movement |
| Movement Plane | One (pointing with index finger) | Cursor tracks index finger movement precisely |
| Two-up (pointing with index and middle finger) | Cursor follows index finger movement with decaying velocity | |
| Click-through Plane | One | Mouse left-click |
| Peace | Mouse right-click | |
| OK (pinch) | Mouse left-click and hold (for clicking+dragging) |
High level architecture (certain modes may work wirelessly):
