Interaction manager for interaction on tactile displays based on the BrailleIO framework.

Make the usage of BrailleIO hardware key interaction a bit more easy. Furthermore, provide some basic functionalities for interactions, such as a Braille-keyboard.

Attention: this all works at this point only with BrailleDis 7200-like Pin-Matrix devices or with the ShowOffAdapter of BrailleIO.

Singleton construct and center class for interaction handling.

Have its own public available BrailleKeyboardInput for enabling Braille text input and basic text-work options.

Has an InteractionMode (Flag) that can be a combination of the following modes to identify how the interactions should be interpreted:

The InteractionManager is a mediator between interactions of an IBrailleIOAdapter from the BrailleIO framework and handlers for button-, button-combinations, gesture or Braille-text-inputs.

Therefor you have to register available input devices (BrailleIO. IBrailleIOAdapter) to the InteractionManager by using the method

public bool AddNewDevice(IBrailleIOAdapter device)

Automatically the InteractionManager registers to the devices’ input events, interpret them and forward them to listeners to handle them.

The ScriptFunctionProxy allows adding specialized interaction handlers that can be activated and deactivate based on the current application state or context.

The ScriptFunctionProxy forwards interaction events, such as button combinations etc., to registered specialized function proxies implementing the IInteractionContextProxy interface. Those specialized function proxies have to take care about their activation and deactivation based on the current application context. The specialized function proxies are ordered and are called in a cascading way. So the can handle the same input command but are called one after another based on the ordering (ZIndex – as higher as earlier it is called).

It has to be initialized with an InteractionManager instance to register to the available interaction events.

The BrailleKeyboard allows translating a Unicode sign or a character into a dot-pattern representation and vice-versa. Basically loaded is the Unicode Braille-char part that will be overwritten by the given *.cti translation table, based on the definitions of the liblouis project.

Set up a new BrailleKeyboard and load a Braille-Char to dot-pattern translation table

// in this example the German translation table is loaded
public readonly Control.BrailleKeyboard BrailleKeyboard = new tud.mci.tangram.TangramLector.Control.BrailleKeyboard(tud.mci.tangram.TangramLector.Control.BrailleKeyboard.GetCurrentDllDirectory() + @"/config/tables/de-chardefs8.cti");

Now you can ask for a character from a specific dot pattern

String c = BrailleKeyboard.GetCharFromDots("14"); // will return "c"

Or you can ask for the dot pattern of a specific character

String pattern = BrailleKeyboard.GetDotsForChar('g'); // will return "1245"

The BrailleKeyboardInput is some kind of virtual text-input field that can be filled with text, holds a cursor/caret for defining the input position and allows for other text-works operations.

The BrailleKeyboardInput is automatically registered to the Braille-keyboard events of the global singleton ScriptFunctionProxy instance:

        /// <summary>
        /// Occurs when a single BrailleKeyboard letter was entered.
        /// </summary>
        public event EventHandler<BrailleKeyboardEventArgs> BrailleKeyboardKey;
        /// <summary>
        /// Occurs when a complex braille keyboard command was entered.
        /// </summary>
        public event EventHandler<BrailleKeyboardCommandEventArgs> BrailleKeyboardCommand;

The input-cursor or caret is the position where text is entered if no text-input position is defined. The caret can be moved or set to a fixed position inside the text.

Actions can be registered to hook the caret moving functions (left, right, up, down, etc.) to adapt the handling if necessary.

The Braille keyboard available at the InteractionManager in combination with the ScriptFunctionProxy interprets button combinations from the BrailleDis like key settings to either Braille charters or Keyboard interaction commands.

The buttons "k1","k2","k3","k4","k5","k6","k7" and "k8" (of a Braille-keyboard-like button-layout) will be transformed into the corresponding dot in the Braille character dot-pattern.

Several standard key commands for text-work are defined. Therefor some additional function keys are necessary (commonly they are lying in between the both button groups for the left and the right hand are activated with the thumbs).

The BrailleKeyboardInput of the InteractionManager instance is global available by calling

BrailleKeyboardInput input = InteractionManager.Instance.BKI;

To activate the Braille interpretation you have to switch the to the Braille mode

InteractionManager.Instance.Mode = InteractionMode.Braille

ATTENTION you have to clear the BrailleKeyboardInput before allowing to write a new text. Otherwise, the currently set text will be extended.

// get the current text input
String text = InteractionManager.Instance.BKI.Input;

// clear the text input
InteractionManager.Instance.BKI.Reset();

The basic sequence of handling user interactions is displayed in the following diagram:

Set up basic global variables for the InteractionManager

readonly InteractionManager interactionManager = InteractionManager.Instance;

Set up a new BrailleIO.IBrailleIOAdapter and add it to the interaction manager as an input device

// see BrailleIO framework for further explanation
BrailleIOMediator io = BrailleIOMediator.Instance;

// set up a example/debug pin-matrix-device
io.AdapterManager = new ShowOffBrailleIOAdapterManager();
AbstractBrailleIOAdapterBase adapter = io.AdapterManager.ActiveAdapter as AbstractBrailleIOAdapterBase;

// add the adapter to the interaction manager as input device
interactionManager.AddNewDevice(adapter); 

For handling inputs by the interaction manger through the extensible ScriptFunctionProxy you can set up as follows:

Set up basic global variables for the ScriptFunctionProxy

readonly ScriptFunctionProxy functionProxy = ScriptFunctionProxy.Instance;

After getting the singleton instance, you have to connect the function proxy with the InteractionManager who delivers the inputs.

functionProxy.Initialize(interactionManager);

To add a specialized function handler for handling inputs an implemention of the IInteractionContextProxy interface can be added to the global ScriptFunctionProxy

IInteractionContextProxy specialProxy = null // you have to build one to handle your specific commands in a specific context

functionProxy.AddProxy(specialProxy);

For handling interaction events of the InteractionManager the interfaces for tud.mci.tangram.TangramLector.IInteractionContextProxy and tud.mci.tangram.TangramLector.IInteractionEventProxy have to be implemented.

For this purpose an abstract basic implement exists: tud.mci.tangram.TangramLector.SpecializedFunctionProxies.AbstractSpecializedFunctionProxyBase. Here you can override the functionalities you need.

public class ExampleSpecializedFunctionProxy : AbstractSpecializedFunctionProxyBase
{
	// override the handler for button combinations
	protected override void im_ButtonCombinationReleased(object sender, ButtonReleasedEventArgs e)
	{
		// check if this function proxy is active and should handle this event
		if (Active)
		{
			// handle a generic button combination of 4 keys reeased
			if (e.ReleasedGenericKeys.Intersect(new List<String> { "k2", "k3", "k4", "k5" }).ToList().Count == 4)
			{
				//TODO: handle whatever should be done 
				e.Cancel = true; // setting the cancel field to TRUE, stops the cascading forwarding to following further specialized function proxies!
			}
		}
	}
}

• DotNet_AudioRenderer - audio renderer for speech output
• DotNet_Extensibility - extension framework for easy extensibility
• DotNet_LanguageLocalization - localizing framework for different languages
• DotNet_Logger - logger to store logging information with timestamp
• Tangram_Interfaces - interfaces of the tangram framework for inter-project-compatibility
• BrailleIO - tactile display abstraction framework

-- TODO: build help from code doc

For getting a very detailed overview use the code documentation section of this project.