Extend ContextActionService with ClickDetector-like solution

[GollyGreg]

As developers, if we end up making a game that utilizes a proximity-based, single-action command system we have to remake the entire interaction system in lua.

I am proposing that we introduce a solution that handles this work for us.

EDIT: Previously I indicated that this needs to be in engine. While I’m purposefully trying to avoid the complications of api design, we should also not restrict this solution to the engine either. I’ll leave the following paragraph in the spoiler for posterity

Redacted a bit

Why is would this be important to bring into the engine?

Currently we have an extendable user input interface through ContextActionService, however CAS on its own does not work “out of the box” for proximity based use cases.
Having this system standardized from the engine will assist developers in writing gameplay systems that work out of the box for all of Roblox’s supported platforms and formfactors.
This has benefits for both the accessibility of Roblox games and alleviates the burden of reimplementing this system on developers so they can focus on their gameplay systems instead.

To developers: How you can support this feature request:

Please reply with any other games you have played or created that have implemented a intractable system like this in lua. Additionally please respond with any other user stories that could be included in the list below.

Some user stories

As a developer, I want to be able to quickly and easily mark intractable points of interest in my worlds that are attached to Instances with a Position or World CFrame property.

As a developer, I want to be able to bind all intractable points of interest in my worlds to a single set of input types.

As a developer, I want to be able to rebind this set of input types to a new set at any point while running the program.

As a developer, I want to have additional ContextActionService bound actions optionally take priority over a nearby intractable point of interest.

As a developer, when I have multiple points of interest in a small area, I want to be able to prioritize which point of interest will capture by interaction input.

As a developer, I want to be able to use a default Robloxian-styled input action prompt cue that appears when players are near an intractable point of interest so I do not always need to design my own.

As a developer, when players are nearby a point of interest, I want to be able to easily design my own input action prompt cue.

As a developer, I want to be able to minimize and/or hide my input action prompt cue when players are not near a point of interest.

As a developer I want to be able to set the radius threshold that a player’s avatar needs to be near in order to interact with the point of interest.

As a developer, when players enter the radius of a point of interest, I want to be able to run additional custom code.

As a developer, when players leave the radius of a point of interest, I want to be able to run additional custom code.

As a developer, I want to be able to have all points of interests use the Player’s avatar by default to check if a Player is within the radius threshold of the point of interest.

As a developer, I want to be able to override the object all points of interests use to check if a Player is within the radius threshold of a point of interest.

As a developer, I want to be able to easily disable or enable individual points of interests through code.

As a developer, I want to be able to easily disable or enable the ability to interact with enabled points of interests through code.

As a developer, I’d like to be able to listen for interactions from either client code or server code, similar to ClickDetectors.

As a developer, I’d want to be able to opt-in to allow points of interest to be Activated for mobile and PC users, similarly to how ClickDetectors work. I want this functionality to be in a single object so I can bind to a single Instance. MeepCity and AdoptMe currently have intractable points of interest that respond to keybinds OR touch OR mouse input.

As a developer, I want my code to have the same experience for mobile platforms, console platforms, and PC platforms when setting up an intractable point of interest.

Some games that currently implement this (links included)

Jailbreak

cc @asimo3089 and @badcc

Egg Hunt 2018

cc @PeteyK473 and @buildthomas

Finder’s Keeper’s

cc @Imaginaerum

Firefly Quarry

cc @Zomebody

Mount of the Gods

cc @Wheatlies

Whatever Floats Your Boat

cc @Quenty

Adopt Me

cc @NewFissy

[Quenty]

I’ve iterated on this system several times. This is a non-trivial system to implement in Roblox–and in general. And tackling all the uses cases will be very game specific.

Note the following usecases

• Syncronized cooldown state on both the client and server for debounced systems
  • Think of a cannon reloading
• Permission system to allow showing (although this tends to be something you want to be careful with)
• Stylistic differences in the GUI between classes of menu items
• Translations
• Scoring system to pick the “right” 3D context menu to show based upon important
• Different hints that change based upon different input modes (Even between WASD and arrow keys)
• Networking requests back to the server
• Additional contextual options that can be shown/hide at the same time.

I don’t think Roblox CAS should, and can, fulfill all of these needs. However, whatever CAS provides should be extensible in that I can pass contentual information into this.

My needs right now is having CAS synchronize which state can be active. I think I’m going to have to eventually synchronize my dynamic discovery of intractable objects with my more expensive static-trigger based system.

Several things I’ve discovered

• Attachments and models are good for this
• Having an actual object in charge of this can work many times (Trigger), but imparts a significant performance cost in other cases
• Finding parts can either be a loop or a Region3. Working to unify this now.
• Sometimes you want text, sometimes you want an icon
• In either case, you need an “Active” and “Inactive” case that hints input vs. no input

I think Roblox’s case here is to make something that makes things easier to prototype. In this case, I highly recommend that Roblox makes the initial version of a feature like this in a forkable local script that can be used.

Edit: Separate feature, but before you fix this, Roblox really should fix the regular “touch buttons” in CAS. They just need to position correctly (around the jump button, moved when the jump button moves), look a tiny bit more modern to be useable for prototyping. The fact these don’t even work imo, precludes Roblox from working on this more complicated feature rquest.

[GollyGreg]

It should be something so at least the new developers can still have decent UX, but I’d really like to use this as well so extending it would be nice

My only concern about having this serviced by a lua script is that we still dont have a great way to distribute lua code… I find features that are built into the engine have been created with a bit more engineering rigor behind them.

**Edit: ** Separate feature, but before you fix this, Roblox really should fix the regular “touch buttons” in CAS. They just need to position correctly (around the jump button, moved when the jump button moves), look a tiny bit more modern to be useable for prototyping. The fact these don’t even work imo, precludes Roblox from working on this more complicated feature rquest.

Agreed that there’s a lot of work that needs to be done here and there’s a few ways to go about it. I’d like to break this part down into its own thread once we’ve figured out where the problem space is

[Quenty]

This is a SUPER valid feature, but it’s also really scary because there’s so many ways to mess this up.

I’m just very afraid of this feature because:

• Use cases are deeply varied per a game
• Picking the right object to be active is SUPER dependent upon things. Here’s some examples in my accelerator
  • Raycasting to make sure stuff isn’t in another object
  • Distance to character
  • Angle of character
  • Mouse location (sometimes)
  • Has the player moved in the last X minutes
  • If the player is “holding” an object (THIS IS SUPER SPECIFIC TO MY GAME!)
• I’ve refactored this system 3-5 times, and I’ll do it again in my game
• Style should not be built into engine
• I have very specific animations that need to be changed

Have fun building this into the engine.

function PickupHintUtils.getScore(adorneeData, requiredAngleRad, maxDistance)
	if adorneeData.isHeld then
		-- Held stuff always gets good score distance
		return 10000 - adorneeData.distanceFromHumanoid/maxDistance
	end

	if not adorneeData.canPickUp
		or not adorneeData.onScreen
		or not adorneeData.hitAdornee -- IMPORTANT: We gotta keep this in because we're comparing the distance of the hit next
		or adorneeData.hitDistance > maxDistance
		or adorneeData.angleRad > requiredAngleRad then
		return false
	end

	-- Score based upon distance and angle!
	local closeDistPercent = Math.map(adorneeData.hitDistance, 0, maxDistance, 1, 0)
	local closeAnglePercent = Math.map(adorneeData.angleRad, 0, requiredAngleRad, 1, 0)

	return 2 * (closeDistPercent*closeDistPercent) + closeAnglePercent
end

[GollyGreg]

updated the OP. I dont really care if its in engine or not. I care about the problem statements

[Quenty]

Yes! Problem is incredibly valid. I’ve thought a ton about this problem, and I’d really like to be able to prototype faster.

This is a primary way to interact with a world. So getting the UX right is really important, and really hard. And the UX varies per a game.

The problem is this is a really hard problem. For context, my new accelerator project probably has 2-3k lines of code making sure all of this feels good, and I’m going to rewrite part of it again at some point to finish unifying systems.

Nailing this is really important, but it’s also going to be hard. This is what makes up a non-trivial amount of games–it’s a core interaction model.

[GollyGreg]

Another angle we can take to look at this is that we could consider that there may be 1 or 2 smaller solutions that could come together to make this larger problem easier.

For example, right now its tricky to do a box cast that only filters primitives with a given tag. If we do a Region3 cast, we can give it a flat whitelist or ignore list, but we cannot give the cast any more additional information to have it filter things with a given tag or property.
Tackling that issue would make this problem a little bit easier as well as a few others.