[Updated] Gamepad UI Highlight Improvements

[AykeriZero]

Hi developers,

We aim to make the gamepad UI highlight work in every possible situation and layout, so your UI works automatically with the gamepad with little effort. We’re excited to share the first set of changes coming to the gamepad UI highlight!

We’ve overhauled the selection algorithm to be able to consider more buttons in all four directions and placed more weight on buttons in the direct path of the direction requested, so you might see some differences in how the UI highlight moves and what objects it selects in your experiences.

Beyond the improvements we are releasing, there is a key point at the end that we’d like your feedback on, so please read through all of the info!

Summary of Improvements

Highlight Movement Logic

• Prioritizes buttons in the requested direction so the highlight moves where players expect

Button Reachability

• Considers all buttons in the requested direction so more buttons will be considered when moving the highlight
• The result is that all of your buttons/selectable UI objects can be accessed by moving the UI highlight with the gamepad

Selection Groups

• If the selected button is part of two selection groups, the highlight is restricted to the intersection of the groups
• This will allow for the creation of nested groups: Selection Group - AddSelectionParent

Highlight Selection

• Improved default highlight selection when first pressing the view button to go into UI selection mode

Bug Fixes

• Calculation of the position of buttons inside scrolling frames
• UI highlight could sometimes move to buttons that were off screen

Details

Movement and Reachability

In this example, the UI highlight is able to property move to all the buttons visible on the screen and reach all the buttons in the scrolling frame.

Selection Groups

More information on how to set Selection Groups can be found here: Selection Group - AddSelectionParent

In this picture, buttons B, C and D are all part of selection group 1. Buttons C and D are also part of selection group 2. The highlight will be able to move from any outside buttons to any button in selection group 1 or 2 but won’t be able to move out. Similarly, any button in selection group 1 will be able to move to buttons in selection group 2 but will not be able to move out.

Example use case - you have a pop up menu and you want to restrict the highlight to the popup and then a smaller section within the popup window. You can leave a selection group by either setting GuiService.SelectedObject yourself or NextSelectionUp/Down/Left/Right on any buttons in the selection group (or you can close the popup).

Previously, you could only create one selection group for each group of UI elements and would not be able to restrict the UI highlight for GuiObjects which are already in a selection group. This change will now allow you to nest multiple selection groups.

Coming Next Year

We have a few additional fixes coming out in the next version.

Scrolling Frames

• If the current selection is in a scrolling frame, the gamepad highlight will prefer other buttons in the same scrolling frame
• The highlight will only move back outside of the scrolling frame if you are at the edge of one of the directions and there are no more selections

Bug Fixes

• Invisible buttons were being highlighted in scrolling frames

Tip: How to Improve UI Highlight Selection

Since our change affects movement logic, we recommend verifying that all the buttons in all your experiences are reachable. If you find that more buttons and labels are being selected than you expect, please mark them as not selectable. We have seen some developers mark labels selectable, that don’t act as buttons and have no functionality.

Give Us Your Feedback

Occluded Buttons

As part of our goal to make gamepad UI selection “just work” in any circumstance or configuration, we have run into a few tricky use cases and are interested in your feedback.

• An Occluded Button is defined as a button that is inside the same layer collector (within the same ScreenGui, SurfaceGui or BillboardGui) and is completely covered by another GuiObject which is not transparent

• In this situation, we feel the best logic for our “it just works” goal is to ignore the buttons that are occluded, as this can commonly happen when a popup UI sits on top of another UI that is already open

• However, this does mean if you purposely put a selectable button on the screen, and then put an image on top of it that is not included with the button UI instance (e.g. not an ImageButton), the gamepad UI highlight would not move to that button (views it as a button that can’t currently be seen by the user)

We realize that some of your experiences might be purposefully layering graphics on top of buttons or other selectable UI objects to get a desired look/design so we want to give you the opportunity to give us feedback on this point.

General Feedback

As always, we’d love to hear your thoughts on all of the changes above. Please respond to this post with any issues due to the changes or suggestions you might have on how we can continue to improve the gamepad UI highlighting system.

Update - 3/16/2022

@AykeriZero

We have a set of additional improvements to the Gamepad UI Highlight system that have been released.

Summary

This update includes improvements within the following areas:

• Scrolling Frames - improved logic for selecting GuiObjects within the same scrolling frame and moving out of the scrolling frame
• Billboard GUI’s - allowing selection of objects within a Billboard GUI
• Selection Logic - GuiObjects that are completely surrounded by other GuiObjects can now be reached and selected

Details

Scrolling Frames

• If the current selection is in a scrolling frame, the gamepad highlight will prefer other buttons in the same scrolling frame
• The highlight will only move back outside of the scrolling frame if you are at the edge of one of the directions and there are no more selections

BillboardGuis

• GuiObjects under BillboardGuis can now be navigated with highlight selection
• You can set the initial selection with GuiService.SelectedObject and the selection system will be able to automatically move through the UI objects in the BillboardGui
• UI highlight selection won’t select the BillboardGui UI initially and it won’t jump from a BillboardGui to other BillboardGuis, SurfaceGuis or ScreenGuis
• This change also fixes some bugs with moving the highlight selection on SurfaceGuis.

Algorithm Adjustments

• Adjustment to the selection algorithm for more consistent tie-breaking
• Slight adjustment to the selection algorithm so that overlapping GuiObjects are selectable!
  

In the image above, there is a large outer button selection. Pressing down will select the inside A button. With the A button selected, down will choose the E button outside of both.

Disappearing Selections

Previously, if there was a single GuiObject on the screen, and that GuiObject disappeared (e.g. a confirmation pop up window that closes), the user would stay in UI selection mode, but there was no UI to select, so it can appear to the user as if the game is broken and the avatar can’t move. We tested an improvement to this behavior and saw player session times go up when this was active:

• If the currently selected GuiObject disappears or becomes not selectable, we look for a new GuiObject selection and we set that as the new SelectedObject (this is the same behavior as if you toggled into UI selection mode)
• If we don’t find any selectable GuiObjects, we turn off UI selection mode and return control of the avatar to the player

Feedback

As always, we’d like to know what you think about these changes and whether you run into any issues in the comment section below.

[Baileyeatspizza]

any chance the highlight ui will be able to be round when highlighting buttons rounded by UiCorner in the future?

[CaelanHyperionRBLX (Caelan)]

This is going to be really amazing to Xbox users!

[ghidras (popbob)]

This is very nice to see. I’m glad that you guys are focusing towards better support for devices such as Xbox. The improved logic for UI Highlighting is a much needed change, and I can’t wait until this change is pushed out. Thank you for giving more light and support towards Xbox devices.

[acaba1806]

cool very nice it was about time they made some changes

[Polyhex]

Nested SelectionGroups is fantastic as someone who’s used them a lot in recent work.

My main point of feedback is that, visually, scrolling through items in gamepad mode doesn’t look/feel so great. Typically, games that have such a feature have some padding when scrolling against the edge of the frame, so that you can see the next item in the scrolling list, but Roblox scrolls to the EXACT edge of the next item.

Having some sort of “scroll padding” option would be super beneficial for general UX in menus.

sloppy drawn example:

[Reditect]

The selection box button when pressed stops the player from moving, which is indented from the looks of it, but I would like to be able to customize that.

I am guessing the disabling of the players movement is done in Roblox’s code via the controls module and doing Controls:Disable() and Controls:Enable() when the selection button is pressed. This is how I am currently disabling the players controls in my game so competing for disabling a players controls breaks things sometimes.

Why do I want this to be changeable?

A bit complex to say: I am currently making a pause menu and want the game to be on Xbox. The only button that can really be used for a pause button is the selection button because Roblox already uses the menu button. The issue with using the selection button to open a pause menu is that the player could switch to a keyboard for input to exit the pause menu, and then go back to controller with the selection box being already being selected, thus disabling the controls when they should be able to move. I could provide a video example if needed.

Put short: Can we be able to control whether the selection button disables the players controls?

[rogeriodec_games]

However, this hasn’t fixed this bug yet:

ScrollingFrame disappears when clicking an item when mobile

Reproduction Steps Run this project: Baseplate.rbxl (37.2 KB) Click on the leftmost menu button Repeat the steps above using a mobile and the ScrollingFrame will not be shown Expected Behavior ScrollingFrame should appear on both platforms Actual Behavior As you can see in the video above, ScrollingFrame works normally when running inside Windows. But when I run on Android and click the left menu button, the ScrollingFrame disappears. The problem is the same, even publishing t…

[Rototally]

Appreciate the feedback. That’s a reasonable polish item for us and generally a good practice so the player knows there are additional items you can scroll to. The spacing might be different for each implementation so we’d have to consider that. The various different and crazy layouts that developers can make might be a factor but we can test this.

[Elttob (dphfox)]

@AykeriZero

An Occluded Button is defined as a button that is inside the same layer collector (within the same ScreenGui, SurfaceGui or BillboardGui) and is completely covered by another GuiObject which is not transparent

I think it’d perhaps be more flexible to add an ‘OccludesInput’ property on GuiObjects to specify whether we want something to block inputs behind it - this would not only let us mark up our decorative elements properly but also could extend to new things like allowing mouse input or touch to pass through multiple layers

[Polyhex]

@Rototally

Appreciate the feedback. That’s a reasonable polish item for us and generally a good practice so the player knows there are additional items you can scroll to. The spacing might be different for each implementation so we’d have to consider that. The various different and crazy layouts that developers can make might be a factor but we can test this.

True, it’s a more difficult problem to solve than it’d seem-- if a new property is out of the question, I could maybe see it based off a percentage of the ScrollingFrame’s Y size or maybe even the size of the currently selected/surrounding buttons.

[FirezDevv (Fire)]

Wow amazing for the selection group that relieves so much now useless code for many games!

[DevJanick]

Image Buttons seems to be broken:
Picture from an Xbox player on our game (as all xbox players currently can’t play the game)

I have tested myself also within studio with a controller. And it really doesn’t react on image buttons.

What exactly happens?

• Doesn’t highlight
• Can’t click on it by using A or RT
• The mouse icon of it also doesn’t highlight

To anyone experiencing this issue a temporary fix would be putting a TextButton inside the imagebutton.

[NoUniqueAddress]

Hey! Virtual cursor uses slightly different rules. Buttons with active == false will not be selected with the virtual cursor. If you set active to true, they should work again!

[Rototally]

Can you let us know if that fixed the issue you were having?

[DevJanick]

Putting Active to true does indeed fix the problem, thank you!
Sorry for the late response, been busy.

[AykeriZero]

We have a set of additional improvements to the Gamepad UI Highlight system that have been released.

Summary

This update includes improvements within the following areas:

• Scrolling Frames - improved logic for selecting GuiObjects within the same scrolling frame and moving out of the scrolling frame

• Billboard GUI’s - allowing selection of objects within a Billboard GUI

• Selection Logic - GuiObjects that are completely surrounded by other GuiObjects can now be reached and selected

Details

Scrolling Frames

• If the current selection is in a scrolling frame, the gamepad highlight will prefer other buttons in the same scrolling frame

• The highlight will only move back outside of the scrolling frame if you are at the edge of one of the directions and there are no more selections

BillboardGuis

• GuiObjects under BillboardGuis can now be navigated with highlight selection

• You can set the initial selection with GuiService.SelectedObject and the selection system will be able to automatically move through the UI objects in the BillboardGui

• UI highlight selection won’t select the BillboardGui UI initially and it won’t jump from a BillboardGui to other BillboardGuis, SurfaceGuis or ScreenGuis

• This change also fixes some bugs with moving the highlight selection on SurfaceGuis.

Algorithm Adjustments

• Adjustment to the selection algorithm for more consistent tie-breaking

• Slight adjustment to the selection algorithm so that overlapping GuiObjects are selectable

  

In the image above, there is a large outer button selection. Pressing down will select the inside A button. With the A button selected, down will choose the E button outside of both.

Disappearing Selections

Previously, if there was a single GuiObject on the screen, and that GuiObject disappeared (e.g. a confirmation pop up window that closes), the user would stay in UI selection mode, but there was no UI to select, so it can appear to the user as if the game is broken and the avatar can’t move. We tested an improvement to this behavior and saw player session times go up when this was active:

• If the currently selected GuiObject disappears or becomes not selectable, we look for a new GuiObject selection and we set that as the new SelectedObject (this is the same behavior as if you toggled into UI selection mode)

• If we don’t find any selectable GuiObjects, we turn off UI selection mode and return control of the avatar to the player

Feedback

As always, we’d like to know what you think about these changes and whether you run into any issues in the comment section below.

[Reditect]

Hey. I know this is more than a month old now, but I am still having issues with the gamepad UI selection. When pressing the selection box on a gamepad, mainly an Xbox controller, the players controls are disabled. This is an issue for my game as I disable and enable the player controls for multiple reasons like talking to an NPC.

I feel that having the ability to choose whether or not the players controls are disabled with the selection box would fix many issues I am currently having in my game.

Here is how I am disabling the players controls:

local LocalPlayer = game:GetService("Players").LocalPlayer
local Controls = require(LocalPlayer.PlayerScripts.PlayerModule):GetControls()

Controls:Disable()

I used the method shown due to the announcement by Roblox: New Player Scripts are coming (10.11.18), and how you can prepare

[Reditect]

Also, controls are seemingly enabled when switching input types. For example, when I switch to an Xbox controller, I gain my controls back. Despite the fact that I disabled them. This means that players can switch input types to break things like NPC interaction.