Compatibility
Minecraft: Java Edition
1.20.1–1.20.2
1.19.2–1.19.4
1.18.2
1.16.5
Platforms
Fabric
Supported environments
90% of ad revenue goes to creators
Support creators and Modrinth ad-free with Modrinth+Creators
Details
Licensed MIT
Published 9 months ago
Updated 8 months ago
Virtual Reality Gesture Recognition API for Minecraft VR
Play Vivecraft or QuestCraft? Love mods? Ever wish mods were more VR compatible? Wait no further. This is a solution for devs and players alike!
This API enables Minecraft mod developers to easily implement support for their mod to be more VR compatible. Create any custom gestures that map to any key binds and trigger events. No longer do you need to map controller keybindings to keyboard keys. With this API, your body is the controller.
Demos:
- https://imgur.com/a/UYaVFCj
- https://imgur.com/a/sYXcike
- https://i.imgur.com/kBD1TIU.mp4
- https://imgur.com/a/0fpiQ9a
- https://i.imgur.com/6xUNF6k.mp4
Installation:
- Install required dependency mods. Note that Questbind is only required for QuestCraft usage.
- Download the VR Jester API mod & place it in the
mods
folder. - Run Minecraft. Necessary config files will generate on initialization.
Usage:
- While in Minecraft VR, bind one of your controller buttons to the "Gesture Listener Trigger". I recommend a grip button. For Vivecraft you must do this through Steam VR's controller bindings menu. For QuestCraft you do this through QuestBind.
- These are 2 ways to create gestures:
- Using the
/gesture
command - Modifying the
config/gesture_store.json
directly where gestures are stored. Sample gesture store:
"GESTURE 1": { "RIGHT_CONTROLLER|LEFT_CONTROLLER": [ { "vrDevice": "RIGHT_CONTROLLER|LEFT_CONTROLLER", "movement": "forward", "elapsedTime": 0, "speed": 0.0, "direction": { "x": 0.0, "y": 0.0, "z": 0.0 }, "devicesInProximity": {} } ] }, ...
- Using the
- Once your gestures are created, you can map them to key binds in
config/VRJesterAPI.cfg
by creating key value objects under the field "GESTURE_KEY_MAPPINGS". You can find the key bind names in.minecraft/options.txt
. The "KEY_ACTION" field determines whether the recognized gesture triggers a single click key press or a hold down key press. A hold is triggered when you keep your gesture in place after it's recognized.
"GESTURE_KEY_MAPPINGS": {
"GESTURE 1": {
"KEY_MAPPING": "examplemod.key.ability_1",
"KEY_ACTION": "click"
},
"GESTURE 2": {
"KEY_MAPPING": "key.hotbar.2,key.sneak",
"KEY_ACTION": "hold"
},
"GESTURE 3": {
"KEY_MAPPING": "key.keyboard.j",
"KEY_ACTION": "click"
}
}
Download:
Info:
- Consider each object within the square brackets [ ] a mere piece of the gesture. You can add multiple of these GestureComponent objects. This API recognizes gestures as complex as you want! Just know the more conplex, the more difficult it is to perform a gesture correctly to match your stored gesture.
- Different values for "movement" are
forward, back, left, right, up, down
. The movement direction is relative to your facing direction at the moment you click the Jester Trigger which initiates the gesture listener. Simple example is punching in the same direction you're facing would be recognized as "forward". - Different values for "vrDevice" are
RIGHT_CONTROLLER, LEFT_CONTROLLER, HEAD_MOUNTED_DISPLAY
and if you want a gesture recognized on multiple VR devices you can incorporate a 'logical or' using a pipe|
like soRIGHT_CONTROLLER|LEFT_CONTROLLER|HEAD_MOUNTED_DISPLAY
- The "elapsedTime" is in milliseconds. So putting a value of
2000
would mean that part of the gesture lasts 2 seconds. This would be good for a charge up sort of move. - The "speed" field is a float value representing the velocity of that GestureComponent. A decent threshold to recognize the speed of a punch would be
1500.0-2000.0
and feel free to play around with the values as much as you want. - The "direction" field takes
x,y,z
float values that create a 3D normalized vector. Example usage of this would be having{0.0, 1.0, 0.0}
to recognize when the player's brick hand is facing upwards. There's a certain threshold so it doesn't have to be exact. - Finally there's "devicesInProximity" which is formatted like so:
What this example means is that the vrDevice of this GestureComponent has to be within proximity of the left controller for 20 ticks."devicesInProximity": { "LEFT_CONTROLLER": 20 }
- Subscribe to the event bus to handle events triggered by a recognized gesture (Forge) or register to the event callback interface (Fabric).
Forge:
@SubscribeEvent
public void onGestureEvent(GestureEvent event) {
// gesture handler code here -> event.getGestureName()
}
Fabric:
public static void init() {
GestureEventCallback.EVENT.register((gestureEvent) -> {
// gesture handler code here -> gestureEvent.getGestureName()
});
}
Planned:
In the future I'll add continous gesture recognition, more configuration options, and optimizations.