Implementing Android Input Method Subtypes

4th March 2015

By Craig Warren, SwiftKey engineer

SwiftKey Tech Blog – posts by developers, for developers. Find more here: swiftkey.com/tech-blog

Something we’ve been grappling with at SwiftKey recently are Android’s Input Method Subtypes. As an active member of the Android ecosystem we wanted to make sure our implementation of the Input Method API was done correctly, but we found the official documentation of Subtypes to be a bit sparse; as a result we started digging into how Subtypes worked. We’ve included what we found here to help others trying to implement an Input Method for Android.

The information here is based on looking at:

What is a Subtype
A subtype is a way to present multiple modes of operation for an Input Method Service. Commonly these different modes represent different languages, but they can also be totally different means of input such as voice input or handwriting recognition.

Before the introduction of Subtypes in Android Honeycomb (API version 11) an Input Method mapped directly to an implementation of InputMethodService.

Screenshot 2015-02-23 13.57.02

Mapping of Input Method to Input Method Service prior to Input Method Subtypes

The introduction of Subtypes in Honeycomb allowed a single Input Method Service to behave as if it were multiple Input Methods. In the example shown below the google keyboard has two input methods, English (UK) and Russian, but these Input Methods are both being implemented by the same Input Method Service.

Screenshot 2015-02-23 13.57.29

Mapping of Input Method to Input Method Service with Input Method Subtypes

What does the system do with Subtypes
Aside from using Subtypes to present a single Input Method Service as multiple Input Methods, Android uses Subtypes in another, undocumented, way. The android spellchecker, when set to use the default dictionary and not a specific language, will use the current Input Method Subtype’s locale, not the device locale as you might expect. If the Subtype’s locale is empty then Android will use the current device locale.

This means that Subtypes MUST report their locale correctly, or it will cause confusing problems for people using the spell checker.

Do Input Methods need to use Subtypes
Input Methods do not need to use Subtypes. But if the Input Method will be included with a device as a system Input Method then it is included in Android’s Compatibility Test Suite (CTS). The CTS include a set of tests which a new device must pass, and some of these tests cover any Input Methods included as system apps.

There aren’t currently many tests which cover subtypes of system Input Methods, but one testcase exists which has a test – testInputMethodSubtypesOfSystemImes() – which requires that all System Input Methods have Subtypes, and that at least one of the subtypes has a real Locale.

It seems likely that this test case might become more detailed in later versions of Android.

How to add subtypes to an Input Method Service
When you add an Input Method to an Android application you have to declare the service in the AndroidManifest.xml file (see the example from the Android Latin IME), you can also provide a reference to another xml file with meta-data about the IME.

<service android:name=”LatinIME”
android:label=”@string/english_ime_name”
android:permission=”android.permission.BIND_INPUT_METHOD”>
<intent-filter>
<action android:name=”android.view.InputMethod” />
</intent-filter>
<meta-data android:name=”android.view.im” android:resource=”@xml/method” />
</service>
The extract from the AndroidManifest.xml file above includes a <meta-data> tag nested within the <service> tag which references the file method.xml. The method.xml file can be found in the applications res directory.

A sample version of Input Method meta-data can be found in the Latin IME here.

It includes details about the Input Method itself as attributes of the <input-method> tag and then a list of <subtype> tags provide information about the subtypes of the input method. A list of valid attributes for the <subtype> tag can be found here.

How do Subtype IDs work
The ID of a subtype is used to identify it by the Android system. The Subtype’s ID can be set programmatically from Jelly Bean onwards, but if the application is installed on an older version of Android or the Subtype ID is not specified then the Subtype’s HashCode is used. There is no method to fetch the Subtype’s ID, but you can use the HashCode() method and, if a subtype was provided on Android Jelly Bean or later, the Subtype ID will be returned.

Subtype Names
A Subtype’s name must be the ID of a String resource. This allows the Subtype name to be localized by the system for different locales. You can include the placemarker %s in the String resource and it will be replaced at runtime with a localized version of the language name, based on the locale of the subtype.

For example:

  • if you set the locale to en_GB, then %s will be replaced with a translation of “English (UK)” based on the current device locale.
  • if you set the locale to be ko_KR, then %s will be replaced with a translation of “Korean” based on the current device locale.

How do Subtypes affect the Input Method
Subtypes do not change the behavior of an Input Method Service, it is up to the Input Method Service to change its behavior based on the currently enabled subtype reported to it by the Android framework. There are two ways the Input Method Service can find out the currently enabled subtype:

  1. Ask the Input Method Manager what the current subtype is
  2. Receive a callback from to the onCurrentInputMethodSubtypeChanged(InputMethodSubtype newSubtype) method

Asking the Input Method Manager
The Input Method Manager is a system service so you can fetch it from an Android Context using (InputMethodManager)context.getSystemService(Context.INPUT_METHOD_SERVICE). Once you have a reference to the Input Method Manager you can ask it for the current Input Method Subtype using final InputMethodSubtype currentSubtype = richImm.getCurrentInputMethodSubtype();.

Full documentation of the Input Method Manager can be found here.

On Subtype Changed Listener
The onCurrentInputMethodSubtypeChanged(InputMethodSubtype newSubtype) method can be overridden in an Input Method Service implementation to handle a change to the current Input Method Subtype. This handles changes to the subtype during input, but it is important that you do not rely solely on this method.

An Input Method Service (which uses subtypes) should always check for the current subtype during onStartInput() or onStartInputView().

What to do with the current Input Method Subtype
What the Input Method Service does when it knows the current Input Method Subtype is up to the Input Method Service. Common things are:

  • Display a layout specific to the subtypes language
  • Change the language of predictions or corrections offered
  • Use a different form of Input such as voice or handwriting recognition

The Input Method Subtype class has String attributes for the locale and mode of the keyboard. These attributes can be used by the Input Method Service implementation to decide how to behave.

Input Methods Subtypes can also have extra information included in the xml using the android:imeSubtypeExtraValue attribute. This should be formatted using key/value pairs e.g. android:imeSubtypeExtraValue="KeyboardLayoutSet=qwerty,AsciiCapable,EmojiCapable". You can fetch the extra values using InputMethodSubtype.getExtraValue() or a specific value using InputMethodSubtype.getExtraValueOf(String key).

Which Subtype is used?
When a new Input Method Service is installed on Android none of its Subtypes are enabled. Android implicitly selects a subtype based on the locales of the subtype and the current locale of the device. If a subtype with a matching locale is found then it is implicitly enabled and set as the current Subtype. If no matching locales are found then Android will implicitly enable and set the first subtype in the list. This default behavior updates the current enabled subtype to follow any changes to the device locale. In effect you can think of this default behavior as “Use system language” but remember that problems may occur if the device is set to a locale that your group of subtypes cannot support.

You can prevent the system from using the system language to select the default subtype by marking one of your subtypes using the android:overridesImplicitlyEnabledSubtype="true” attribute. This will make the subtype the default subtype for your Input Method Service, unless another subtype is explicitly enabled.

To be clear the order which subtypes are evaluated to decide which one is the implicitly enabled is as follows:

  1. The subtype marked with android:overridesImplicitlyEnabledSubtype="true”
  2. The subtype with a locale which matches the device locale
  3. The first subtype in the list

Can I add more Subtypes at runtime?
The Input method Manager provides a method which allows you to add an array of additional subtypes to the list provided with the Input Method’s meta-data. InputMethodManager.setAdditionalInputMethodSubtypes(...) takes the Input Method’s ID (see “Getting an Input Method’s ID”) and an array of InputMethodSubtype to add. The additional subtype array provided will replace all current additional subtypes. You can remove all the additional subtypes by providing an empty array.

To create subtypes to be added you can use the InputMethodSubtype Constructors; these were deprecated in Android KitKat (API 19) in favor of using the new InputMethodSubtypeBuilder class, but you will need to use the Constructors if you support older versions of Android.

Limitations when adding subtypes
When creating Subtypes dynamically you are still bound by the same rules as when you create subtypes in the Input Method’s meta-data XML file. The most important of these limits is that the name of the Subtype cannot be a String, it is the ID of a String resource when is localizable. You can still use a resource which include %s to be replaced with a localized version of the language name, but you cannot makeup a Subtype name on the spot, it has to exist in the resources bundled with the application.

Can I remove Subtypes?
In short… no.

The list of subtypes which is included in the InputMethod meta-data is fixed. There is no API to remove or disable them.

You can however remove any additional subtypes you added at runtime, simply re-add your additional subtypes minus the one(s) you want removed.

How to enable other/multiple subtypes

Changing the enabled Subtypes Programmatically
There are some parts of the Input Method Manager API which look useful for managing enabled subtypes. But in none of them seem to work full as expected, which is quite disappointing.

SetCurrentInputMethodSubtype

The first is InputMethodManager.setCurrentInputMethodSubtype(...), the method documentation sounds promising but this method always caused Android to throw a Security Exception when used. The exception is “java.lang.SecurityException: Package android does not belong to 10452”, in this case the “android” package is the system process which is being asked to change the Subtype and 10452 is the Linux User ID for my Input Method.  This method didn’t seem to work as advertised.

SetInputMethodAndSubtype

The InputMethodManager.setInputMethodAndSubtype(...) method allows the Input Method Service to change the current Input Method and Subtype. Inside a class which extends KeyboardService you can get the IBinder by calling getWindow().getWindow().getAttributes.token.

Unfortunately this method only works from within the KeyboardService implementation, it does not work from within an Activity included as part of the same application, this is probably due to the security concern that only the active input method should be allowed to change the input method.

This method also causes a strange situation where you can set the current Subtype to one which is not technically enabled. In other words, it does not technically enable the Subtype, but it does make it active.

This method does not work if any subtype, other than the implicit default subtype, is enabled.

Finally, this method wouldn’t allow multiple subtypes to be enabled at the same time.

Use the Subtype Settings Activity provided by Android
An application can display the Input Method Subtype Settings Activity using an intent like the code sample shown:

final Intent intent = new Intent(Settings.ACTION_INPUT_METHOD_SUBTYPE_SETTINGS);

intent.putExtra(Settings.EXTRA_INPUT_METHOD_ID, inputMethodInfo.getId());

intent.putExtra(Intent.EXTRA_TITLE, “Select Enabled Subtypes”);

context.startActivity(intent);

The Input Method ID which is added to the intent as an extra is used to tell the Activity which Input Method to show the subtypes for. Leaving it blank will show Subtypes for every enabled Input Method installed on the device (see “Getting the Input Method’s ID”).

In the example above the title of the activity can be changed by passing a string extra to the Intent with the key Intent.EXTRA_TITLE. The example above shows a hardcoded string, but it is recommended to use a String resource which has been localized.

The subtype settings activity for the Google IME is shown below.

subtypeSettings

The user can disable “Use system language”, the implicit default subtype, and then choose to enable specific subtypes from the list. If the user enabled multiple subtypes then the user will be offered them both when switching input methods.

It is important to note that prior to Android Lollipop (API 21) it was possible for users to get to this screen without the Input Method application directing them itself. Users could reach the Subtype Settings Activity directly by going to Android’s settings app, choosing “Languages & Input” and long pressing on the name of any enabled input method. This feature was removed from Android Lollipop, but the source code doesn’t indicate why.

Getting an Input Method’s ID
You can find an Input Method’s ID from its InputMethodInfo class, you can get a list of all the enabled InputMethodInfo objects by calling InputMethodManager.getEnabledInputMethodList() and checking the List for an instance of InputMethodInfo with the package name which matches the installed application using Context.getPackageName().

What’s missing from subtypes
In my opinion, there are some things missing from subtype.

Support for multiple languages in a single Subtype
One of SwiftKey’s key features is that users can type in up to three languages on the same keyboard on Android and up to two languages on the same keyboard on iOS, provided the three languages use the same script. Currently Subtypes don’t support multiple locales which make it hard to build a subtype which accurately describes how the keyboard will behave. This causes real problems when combined with the system spellchecker, its reliance on the Subtype locale to select the dictionary to use just isn’t compatible with Input Methods which support multilingual input.

Most Input Methods offer a selection of Subtypes, but these subtypes are technically set up incorrectly since they report a single input language to the system but are actually supporting the user in the input of multiple languages. The end result of this is that dictionaries on the device can become full of user-defined words which actually belong to another language.

APIs to control enabling subtypes

Although the Subtype Settings Activity does allow full control over which subtypes are enabled, it means the Input Method has to rely on the system for this functionality. The biggest problem here is that many Input Methods have complicated control over available languages and this Activity doesn’t meet all of their requirements. Opening up the API to allow control over enabled subtypes would allow Input Method developers to add this functionality into their own Language Settings Activities.

Subtype Names
It’s a bit restrictive to require the Subtype names to be a localisable String resource. I understand why the current system works like this, but it is preventing innovation. If multiple languages were supported in a single subtype, the developers may want to create a String for the Subtype name and handle localisation themselves, currently they cannot do that.

Documentation
This document goes some way to explaining how Subtypes work in Android, but it is the result of a lot of examining source code and writing experimental Input Methods, something most developers don’t have time for. The APIs are current and up to date, that’s not a problem, but more detailed documentation about how Subtypes are expected to work, what the System is going to do with subtypes and generally where Android is going in this area would probably help a lot.

Read more:

Why I became an iOS developer

How I became a software engineer

One response to “Implementing Android Input Method Subtypes”

  1. Игорь says:

    Hi. I selected 2 languages in settings. SwiftKeyboard looks like single subtype in Select language dialog. How can it be possible?

Leave a Reply

You must be logged in to post a comment.

This website uses cookies. By continuing to use this site, you consent to our use of cookies