Improve iPadOS pointer interactions - Part 1: buttons

Recently, I integrated pointer interactions to the Karafun app and I thought it would be worth to share some techniques I found to improve those interactions.

Particularly, I watched the videos Design for the iPadOS pointer and Build for the iPadOS pointer and I understood something about the way the pointer is supposed to move to a control and from a control to another when those controls are contiguous. It’s explained in the video Design for the iPadOS pointer at 20’40’’ and states that the pointer shape should move from one controller to another seamlessly, without changing its shape or frame. It’s emphasized again in the other video at 19’20’’ and it’s explained that it can be done by changing the view’s hit test area.

But how can you do that? Well, if you watched those videos and look for a solution, here is one that I found reliable. In this article we will see how to get the result shown in those videos with buttons and contiguous buttons.

The first step will be to make the pointer clips to a button before it reaches its content. This will then be easier to tackle the contiguous buttons matter.

The project for this article can be downloaded here

Simplest approach

The pointer API offers a very simple way to add a pointer interaction on a button, as explained in the videos. We’ll start by this one and see if we are satisfied with it.

To follow along this article, you can create a new iOS app project on Xcode. In the provided ViewController class, start by adding a simple UIButton.

let button = UIButton(type: .system)

Then in the viewDidLoad() method add the following below the super call.

// 1
button.setTitle("Tap me", for: .normal)
button.titleLabel?.font = .preferredFont(forTextStyle: .title3)
view.addSubview(button)

// 2
button.translatesAutoresizingMaskIntoConstraints = false
NSLayoutConstraint.activate([
    button.centerXAnchor.constraint(equalTo: view.centerXAnchor),
    button.centerYAnchor.constraint(equalTo: view.centerYAnchor)
])

// 3
button.isPointerInteractionEnabled = true
// 4
button.pointerStyleProvider = { button, proposedEffect, proposedShape in
    var rect = button.bounds.insetBy(dx: -12, dy: -10)
    rect = button.convert(rect, to: proposedEffect.preview.target.container)
    return UIPointerStyle(effect: proposedEffect, shape: .roundedRect(rect))
}

Here are a few notes:

1. Simple setup here to provide a title and a large enough font for the text
2. The button is centered in the view
3. UIButton has a boolean property that can be set to allow a pointer interaction. It’s more convenient than setting a UIPointerInteraction with a UIPointerInteractionDelegate.
4. With the boolean set above, the system will call the function pointerStyleProvider when a pointer start entering the button’s region. This is where we provide the code to set the pointer style (like the shape or the effect). It’s quite similar to the one provided in the documentation. Here, we simply set a UIHoverEffect.highlight with a rectangle larger than the button frame by 12 points horizontally and 10 points vertically.

I could not find recommendations for padding for the pointer shape. I guess it depends on controls. I found the values below correct for a text button.

So let’s see this in action! Launch an iPad simulator (or a device with a trackpad/mouse if you have one). If you are running the simulator, you can capture the mouse to emulate a pointer with the appropriate button in the tool bar of the window or with ⌃⌘K.

That’s quite nice already! And with only 6 lines of code. Unfortunately, we do not get the hit test area for free, and the pointer will clip to the button only when it enters its visible content.
Let’s try to change that.

Augmenting the hit test area

In the videos, they advice to augment the hit test area for the pointer to adopt the style for the controls sooner. It’s called magnetism.

As UIKit uses object oriented paradigm, we have to implement a subclass of UIBUtton to override its point(insde:with) function. Whether this class is final or not is up to you. Meanwhile, I found that using UIEdgeInsets to augment a hit test area is quite simple to use, so that’s the path we will take.

Here is the button implementation with comments.

extension UIEdgeInsets {

    static func all(_ value: CGFloat) -> UIEdgeInsets {
        UIEdgeInsets(top: value, left: value, bottom: value, right: value)
    }
}

final class CustomEdgeInsetsHitTestButton: UIButton {

    // 1
    static let defaultHitTestEdgeInset = UIEdgeInsets.all(-20)
    // 2
    var hitTestEdgeInset: UIEdgeInsets?

    // 3
   override func point(inside point: CGPoint, with event: UIEvent?) -> Bool {
       bounds.inset(by: hitTestEdgeInset ?? Self.defaultHitTestEdgeInset).contains(point)
   }
}

1. A default UIEdgeInsets value is defined to be used when no other one is provided. It uses a convenience function to initialize a UIEdgeInsets defined above.
2. Alternatively, another UIEdgeInsets value can be set to be used rather than the default one.
3. This function is called by UIKit when a touch event event can concern the view. As mentioned in the documentation, the provided point is in the "receiver’s local coordinate system (bound)". Which means that the coordinates conversion has already been done. So we can use the button’s bounds directly. Meanwhile, we use the proper UIEdgeInsets value to inset the rectangle to then call its contains(_:) method.

Using a negative value can be surprising. In fact, an inset diminishes a rectangle by a certain amount. Thus, to rather augment a rectangle, we have to provide negative values. It’s possible to make the button asks for positive values instead, but it could be confusing for developers who knows the inset behavior already. Also this is the choice made by the engineers of UIKit and I think it’s better to stick with it.

With that in place, let’s change the button property to a CustomEdgeInsetHitTestButton (we’ll use the default inset of 20 points).

let button = CustomEdgeInsetsHitTestButton(type: .system)

It seems that nothing changed 🤔 Although we can tap on the button outside of its visible content, the pointer will not clip to it sooner.

This is where I struggled to find a solution, and it requires to get rid of the pointerStyleProvider function to use the UIPointerInteractionDelegate.

UIPointerInteractionDelegate

Fortunately, we just have a few adjustments to make to get the desired results. Start by removing the code involving the button with the pointer's interaction and replace it with this single line

addInteraction(UIPointerInteraction(delegate: self))

The compiler might complain so let's make the CustomEdgeInsetsHitTestButton implements the UIPointerInteractionDelegate and provide the first function to customize the pointer style the way we want.

extension CustomEdgeInsetsHitTestButton: UIPointerInteractionDelegate {

    func pointerInteraction(
        _ interaction: UIPointerInteraction,
        styleFor region: UIPointerRegion
    ) -> UIPointerStyle? {
        guard let view = interaction.view else { return nil }

        let preview = UITargetedPreview(view: view)
        let rect = view.frame.insetBy(dx: -12, dy: -10)

        return UIPointerStyle(effect: .highlight(preview), shape: .roundedRect(rect))
    }
}

The code is quite similar to the one that was in pointerStyleProvider although here we first have to get the view the interaction is attached to and use its frame directly rather than doing a conversion.

I tried to keep a conversion to the target preview container but I found the view's frame to be more reliable. If you know the proper way to do it please let me know as I could not find anything on the topic.

I would say that logically, if you run the code right know, you should see that we get back to the same state as with the pointerStyleProvider. But it seems that another function is required for the button to get a pointer style. The good news is: we can also use this function to get some magnetism 🧲

Here it is

func pointerInteraction(
    _ interaction: UIPointerInteraction,
    regionFor request: UIPointerRegionRequest,
    defaultRegion: UIPointerRegion
    ) -> UIPointerRegion? {
    UIPointerRegion(
        rect: defaultRegion.rect.inset(
            by: hitTestEdgeInset ?? CustomEdgeInsetsHitTestButton.defaultHitTestEdgeInset
        )
    )
}

This function is called when the pointer enters the hit test area. What we do here is that we simply use the visible region of the button (the default region) and we augment it similarly to the hit test area, using hitTestEdgeInset if set to a non nil value or defaultHitTestEdgeInset.

Let's run the app once again

Now, when the pointer is approaching the button, it clips to it before reaching the button’s content. Neat!

You can play with the hitTestEdgeInset property of the buttons if you want to see the difference. We are then taking a look at contiguous controls.

Contiguous controls

In the videos, it is also advised that the pointer should not change its shape while it transitions from one contiguous control to another when the shape is the same for both controls. To better understand the point, you can go to the Calendar app and position the pointer on the top left buttons in the navigation bar. That’s what we are going to achieve.

To test that, we will add another button. But first, to avoid repetition, it is better to move the pointer logic inside the CustomEdgeInsetsHitTestButton class. So please add a new function to the class.

func addDefaultPointerInteraction() {
    addInteraction(UIPointerInteraction(delegate: self))
}

Then, let’s add another button. For this article, we will just copy and paste the configuration code. So please replace the overall ViewController implementation with this one.

let button1 = CustomEdgeInsetsHitTestButton(type: .system)
let button2 = CustomEdgeInsetsHitTestButton(type: .system)

override func viewDidLoad() {
    super.viewDidLoad()

    button1.setTitle("Tap me", for: .normal)
    button1.titleLabel?.font = .preferredFont(forTextStyle: .title3)
    button2.setTitle("Tap me", for: .normal)
    button2.titleLabel?.font = .preferredFont(forTextStyle: .title3)

    button1.addDefaultPointerInteraction()
    button2.addDefaultPointerInteraction()
}

Now, to have controls aligned horizontally, the best way is to add a UIStackView and put the buttons inside it. So let’s add that.
First declare the stack view below button2.

let stackView = UIStackView()

Then add the configuration at the end of viewDidLoad().

stackView.axis = .horizontal
let buttonSpacing = -(button1.hitTestEdgeInset?.left ??
    CustomEdgeInsetHitTestButton.defaultHitTestEdgeInset.left)
stackView.spacing = buttonSpacing * 2
stackView.addArrangedSubview(button1)
stackView.addArrangedSubview(button2)
view.addSubview(stackView)

stackView.translatesAutoresizingMaskIntoConstraints = false
NSLayoutConstraint.activate([
    stackView.centerXAnchor.constraint(equalTo: view.centerXAnchor),
    stackView.centerYAnchor.constraint(equalTo: view.centerYAnchor)
])

This is basic configuration to add the buttons to the stack view and to center it in the parent view.

What’s more interesting is that we give the stack view a spacing that equals twice the button’s hit test area extension (in production code we would have to think about a safer way to do it like having a method on the class but it’s fine for the example). Doing so, the space between the two controls will still be capable of receiving touch events, which prevents a touch to be ignored if it does fall between the buttons. Thus the pointer is able to move from one button to another because it is content that we can interact with.

Meanwhile when using a finger, it prevents from missing a button when the tap happens in the empty space, which is more convenient for the user (as explained in the videos and others related to user interactions).

But, did you notice something? We lost the magnetism behavior on the buttons for the edge outside of the stack view! What happened?

Well now when UIKit wants to know if the button is interested by the event using its point(inside:with) function, it first asks to its parent view - here the stack view - if it is interested by the event. Since the stack view does not have an override of the function, it answers that it is not interested and thus UIKit will not ask to the button.

Augment the hit test area of the container

There are two possibilities to fix that. First, we could add some margins to the stack view so that when the pointer arrives at the button, it is already inside the stack view and UIKit asks the button directly. Or we could extend the stack view hit test area similarly to what we did for the buttons. I think that the second approach is more flexible as it does not force us to add visible padding. So let’s do that.

Add another class to the project and name it CustomEdgeInsetsHitTestStackView.

final class CustomEdgeInsetsHitTestStackView: UIStackView {

   static let defaultHitTestEdgeInset = UIEdgeInsets.all(-20)
   var hitTestEdgeInset: UIEdgeInsets?

   override func point(inside point: CGPoint, with event: UIEvent?) -> Bool {
       bounds.inset(by: hitTestEdgeInset ?? Self.defaultHitTestEdgeInset).contains(point)
    }
}

Using the same value to extend the hit test area for the stack view and for the buttons might not a good idea for all use cases. It’s ok here since we have space. But sometimes it could be more interesting to have a large area for the stack view and then a smaller area for the controls inside the stack view if the space is limited.

Then, change the stack view declaration to the following.

let stackView = CustomEdgeInsetsHitTestStackView()

You can then run the project again.

Now we get the magnetism back from the outside edges, phew!

That’s all for this article. In further ones, we will explore a solution with SwiftUI to extend the hit test area and implement the same behavior for contiguous controls. Stay tuned!

📡

I hope you found this tip useful! If so, feel free to share it on Twitter. If you want to reach out to me or to know when new posts are available, you can find me on Twitter. Also you can send me an email.