WWDC 2023: App Intents, App Shortcuts and Spotlight

Find hereafter a detailed summary of the hereunder named videos that belong to a taxonomy of some WWDC footages.

The original ones are available on the Apple official website (session 10102, session 10103 and session 10193).

"Discover how to use App Shortcuts to surface frequently used features from your app in Spotlight or through Siri. Find out how to configure search results for your app and learn best practices for creating great App Shortcuts. We'll also show you how to build great visual and voice experiences and extend to other Apple devices like Apple Watch and HomePod."

"Explore the latest updates and learn how you can take advantage of dynamic options and user interactivity to build better experiences for your App Shortcuts. We'll share how you can integrate with Apple Pay, structure your code more efficiently, and take your Shortcuts app integration to the next level."

"Learn about the latest updates to the visual language of App Shortcuts and find out how to design your shortcut to appear as a top hit in Spotlight. We'll share how shortcuts can appear on iOS or iPadOS, and show you how to customize the visual appearance of a shortcut, personalize its order, select its correct behavior, and increase discoverability."


Hereafter, the underlined elements lead directly to the playback of the WWDC video at the appropriate moment.

Design #

Three main principles come to mind when dealing with the design for Spotlight.

🎬

Personalizing the app shortcuts through the decisions made by the users is also a good mean to improve the predictability.


In iOS 17, starting any search with the Search utility makes it easy to render apps and their shortcuts as well nearby in the top hit as in the Shortcuts app.

Few areas should be analyzed in depth regarding the design with Spotlight.

🎬

Once the concepts of action and entity are integrated, it's crucial to avert truncations in the names...


... and to maintain a coherence between the shapes of the shortcuts.

🎬

While the background color of the Top Hit in Spotlight may be customized and spread to the Shortcuts app, it's also possible to supply a secondary tint color to the symbols for highlighting a two-tone appearance.

App Launch, Live Activity and Snippet are the possible options to be followed according to the appropriate screen where it's suggested to the user.

🎬

To make the App Shortcuts discoverable in Spotlight, it's important to write phrase synonyms to tailor and direct the search.

Providing app name synonyms may be a good mean for some specific situations as well.



App Shortcuts #

Basics #

Made to ease the app's functionality discovery, the App Shortcuts may be reached from various tools and leads to a high range of outcomes through its usage.

Intent & shortcut #

A basic app shortcut is easy to be implemented by following the creations of the app intent and the app shortcut.

Entity & query #

To dive deeper into the possible opportunities, it's important to understand the concepts of entities and queries.

Once implemented, the entity and the query elements' code writing leads to an upgraded version of the previous example.

Implementation #

Expanding the previous example to a broader range including Siri and Spotlight gives rise to an app shortcut implementation.

The result relies on the mandatory notion of updating the App Shortcut parameters.

Discovery #

Finally, the main two ways for finding out the App Shortcuts through the device is based on Spotlight and Siri Tips.



iOS 17 enhancements #

New APIs have been introduced to support and develop the iOS 17 features that relate many areas.

🎬

Check out the next section for a better understanding of this App Shortcuts Preview new feature of Xcode.



Demo #

Testing the app in another language like Mandarin becomes easier when using this Xcode new feature.

String Catalog #

iOS 17 brings many enhancements to localize the App Shortcuts trigger phrases.

The String Catalog format introduced in iOS 17 is easy to be implemented in your app whatever App Shortcuts has already been integrated or not.

🎬

Creating a new file from your app in Xcode gives rise to the opportunity of building some localized files based on the String Catalog format.

🎬

If the App Shortcuts is already integrated, it's quite easy to migrate the AppShortcuts.strings file to String Catalog.



Tips & tricks #

It's essential to keep in mind the App Shortcuts' limitations that could lessen the impact of Siri.


Results across devices #


App Intents #

App intents are made to expose an app’s capabilities to the system through actions that need the AppIntent protocol approval for their creations and the App Shortcuts with Siri to be highlighted.

Widgets #

The widget configuration provides more customization for the user programatically inserted inside the Widget extension code.

Migrate the existing widgets from SiriKit to App Intents becomes easier thanks to Xcode 15.

The widget interactivity is highly improved by the buttons and toggles whose actions can be fired through this interface.


Advanced techniques for configuration also come in with iOS 17.

🎬

Aimed at accessing the parameters from the Intents, the dynamic options interface comes in with a new IntentParameterDependency property wrapper that can be used within DynamicOptionsProvider or Query.

This dependency might also rely on multiple parameters.

🎬

Define an array size during declaration is now possible while refining which one of these parameters should be displayed according to some conditions.

🎬

The action to be performed when a tap occurs on the widget relies on the widgetConfigurationIntent method call on the user activity to reach the associated configuration Intent and navigate through the app as desired.

Note that the new RelevantContext type that specifies conditions for relevance regarding the Widget suggestions is the way to follow to optimize the widgets' Smart Stacks.



Developer experience #

The framework support has been enhanced to become more flexible through the new AppIntentsPackage protocol to make app intent declarations directly available with the framework.


Instead of defining the App Shortcuts in the main app bundle, it's now possible to create an AppShortcutsProvider and define App Shortcuts in the App Intents extensions thus avoiding to launch the app in the background when its App Shortcut is running.

The hereabove enhancements are based on the process of data static extraction that has been improved in Xcode 15 to keep on executing an Intent in the app even if it was initially started in background.


Besides the support for Apple Pay, two iOS 17 significant improvements have been inserted into App Intents.

🎬

The ForegroundContinuableIntent protocol is the way to follow if it's necessary to keep on executing an Intent in the foreground while it was started in the background.

The needsToContinueInForegroundError method should be used when it's required to stop the Intent execution and to perform an action to carry on in the foreground.

🎬

As an instance method of the ForegroundContinuableIntent protocol, the requestToContinueInForeground should be used when it's asked to the user to continue execution in the foreground without stopping the Intent execution.



Shortcuts app integration #


As the foundation to make an app's functionality revealed through Siri and App Shortcuts, the App Intents usage relies on the following good practice:

  1. It's important to get into the habit of using the parameter summary to expose clearer information to the user.

When App Intents should be used in specific circumstances and not opened to the whole app, using the isDiscoverable type property is appropriate if it deals with a widget or a button in a SwiftUI app.
Introduced in iOS 17 as a boolean value, the goal of this element is to determine whether Shortcuts and Spotlight can discover an app intent.

  1. The ProgressReportingIntent protocol has been introduced in order to report progress to the system during its execution.

  1. The EnumerableEntityQuery protocol now supersedes the EntityPropertyQuery protocol introduced in iOS 16 for a simpler and a more scalable search usage even if special attention should be paid to some significant cases.

  1. A new resultValueName property has been added to the IntentDescription structure so as to be displayed in the Shortcuts editor as a variable output for instance.

  1. iOS 17 provides the opportunity to add an Intent description for the Find actions whatever the query protocol.