Technology moves fast! ⚡ Don't get left behind.🚶 Subscribe to our mailing list to keep up with latest and greatest in open source projects! 🏆


Subscribe to our mailing list

acat

Assistive Context-Aware Toolkit (ACAT)

Subscribe to updates I use acat


Statistics on acat

Number of watchers on Github 2742
Number of open issues 49
Average time to close an issue about 1 month
Main language C#
Open pull requests 7+
Closed pull requests 1+
Last commit over 2 years ago
Repo Created over 4 years ago
Repo Last Updated over 1 year ago
Size 173 MB
Organization / Authorintel
Latest Releasev1.00.0
Contributors1
Page Updated
Do you use acat? Leave a review!
View open issues (49)
View acat activity
View on github
Fresh, new opensource launches 🚀🚀🚀
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating acat for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

Official release of ACAT v1.00.0 now available here

Introduction

Assistive Context-Aware Toolkit (ACAT) is an open source platform developed at Intel Labs to enable people with motor neuron diseases and other disabilities to have full access to the capabilities and applications of their computers through very constrained interfaces suitable for their condition. More specifically, ACAT enables users to easily communicate with others through keyboard simulation, word prediction and speech synthesis. Users can perform a range of tasks such as editing, managing documents, navigating the Web and accessing emails.

ACAT was originally developed by researchers at Intel Labs for Professor Stephen Hawking, through a very iterative design process over the course of three years. Professor Hawking was instrumental to the design process and was a key contributor to the project design and validation. After Intel deployed the system to Professor Hawking, we turned our attention to the larger community and continued to make ACAT more configurable to support a larger set of users with different conditions.

Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction and many other features. ACAT is designed to run on Microsoft Windows* machines and can interface to different sensor inputs such as infrared switches, camera, push buttons, and more.

ACAT is written in C# using Microsoft Visual Studio 2012 and .NET 4.5 and runs on Microsoft Windows 7 or later. Word prediction functionality is powered by Presage, an intelligent predictive text engine created by Matteo Vescovi.

Project Website

Click here for the ACAT project website. Documentation can be found here

Licensing

ACAT is distributed under the Apache License, Version 2.0.

Who It's For

ACAT is useful for Microsoft Windows developers who are interested in developing assistive technologies to people with ALS or similar disabilities. Also for researchers who are working on new user interfaces, new sensing modalities or word prediction and wanting to explore these innovations in the this community.

acat open issues Ask a question     (View All Issues)
  • over 2 years New pre-release of ACAT v1.00.0 now available
  • almost 3 years Mouse Scanner issue and ACAT losing focus
  • almost 3 years New pre-release of ACAT v0.99.1 now available
  • almost 3 years License for Documentation
  • almost 3 years Where are the ACAT Vision's algorithms of facial gestures?
  • almost 3 years How to change the voice from male to female?
  • almost 3 years Why the software history was not kept?
  • almost 3 years Cannot build installer
  • almost 3 years New pre-release of ACAT (v0.99) now available
  • about 3 years LOCALIZATION
  • about 3 years Running ACAT as a user
  • over 3 years How to connect switch for F12
  • over 3 years No scanner - settings modification
  • over 3 years No face detection in Vision
  • over 3 years ACAT Vision : missing dll msvcp110.dll
  • over 3 years User Interface Translation!
  • over 3 years sensors and trigger system
  • over 3 years Need help in getting ACAT to run on Windows 7
  • over 3 years click2speak - alternative application
  • over 3 years Enter Key freeze program due to Presage timeout
  • almost 4 years Mouse control - Radar Scanning Mode
  • almost 4 years New release of ACAT now available
  • almost 4 years ACATsetup.exe not found in downloaded acat-0.9.1.zip
  • almost 4 years Localization.
  • almost 4 years acat_gestures.exe has stopped working
  • almost 4 years Support for iOS devices
  • almost 4 years More Language Support?
  • almost 4 years Port to GNU/Linux platform request
  • almost 4 years Fatal error. Error setting word prediction engine to [Presage Word Predictor]
  • almost 4 years Restarting the Presage engine
acat open pull requests (View All Pulls)
  • Add Appveyor.yml for enabling continuous build
  • DOC: Add url with MSDN review and screenshots.
  • Fix for #48
  • Fix broken headings in Markdown files
  • Update README.md
  • German Language Pack
  • Added sapces after "##" for heading to work
acat list of languages used
acat latest release notes
v1.00.0 ACAT Official Release v1.00.0

This is the official v1.00.0 release. The complete source code is included. Please report any issues/problems.

New in this release

  1. The English version includes an expanded Presage database for word prediction.
  2. Bug fixes from the previous release

Refer to the ACAT User Guide for details on the new features.

  1. Portuguese Language pack added. ACAT is now available in English, French, Spanish and Portuguese.
  2. Support for multi-modal gestures for off-the-shelf switch boxes with multiple switches. For instance, if a switch box has two switches, one can be mapped as the selector and the other to a command such as "undo the last editing change", or to "restart the scanning sequence". Mapping is done through the "Keyboard Actuator" setting in the "Actuators" option in the ACAT Config utility. The default trigger key (F12) can also be changed. All this can be done through the ACAT Config utility.
  3. A Phrases editor for adding/deleting/updating phrases that can be converted to speech. In the earlier versions of ACAT, this functionality was embedded in the Abbreviations editor. The Phrases editor can be accessed from the Settings menu. The editor also allows you to re-order phrases for faster access to the more frequently used phrases.
  4. ACAT applications are now DPI-aware. The interfaces looks sharper as it is now scaled to the native resolution of the monitor. The previous versions which were not DPI-aware resulted in fuzzy renditions of the UI elements as they depended on the Desktop Windows Manager (DWM) component of Windows to perform default scaling. The difference is more pronounced on high-resolution monitors (4K).
  5. Automatically adjusts to changes in screen resolution, for instance, if an external monitor or projector is connected.
  6. Contextual support added for the Windows Calculator (Win7, Win8 and Win10). The contextual menu mirrors all the buttons in the Standard calculator
  7. Contextual support added for the Windows 10 Photos app.
  8. Contextual support added for the Windows Edge Browser on Windows 10
  9. A new ACAT FAQ document has been added to the documentation. It has quick answers and tips for some of the most commonly asked questions.

Bugs found in the previous release have been fixed.

Documentation

All ACAT documentation and videos can be found here Getting Started Guide ACAT FAQ User Guide Developer's Guide

Videos

Prof. Stephen Hawking's Introduction to ACAT ACAT Features Overview ACAT Vision - Using a Webcam as Input Switch Using other Input Switches

Quick Installation Instructions

You must install the base English version ACAT before installing any of the ACAT Language packs. The Language packs are incremental installs over the base version. Below are quick steps to install ACAT. For detailed install/uninstall instructions, refer to the ACAT User Guide .

The base English version:

  1. If you have a previous version of ACAT already installed, first exit the application. If ACAT Dashboard is running, close it. Uninstall ACAT by going to Control Panel -> "Uninstall a Program". Right click on "Assistive Context-Aware Toolkit (ACAT)" and select "Uninstall". When prompted to remove ACAT and all its contents, select "No" if you are upgrading from a previous version and you want to retain your user settings. Otherwise select "Yes", which will delete all the contents under the ACAT installation folder
  2. Save ACATSetup.exe (at the bottom of this section) on your computer. This is the default English version
  3. Run ACATSetup.exe. Accept all defaults in the installation wizard.
  4. When installation is complete, a shortcut to ACAT Dashboard will be created on the Desktop. The Dashboard has shortcuts to all the ACAT applications.

French Language Pack:

  1. First download and install the base English version before installing the language pack.
  2. If you have a previous version of the ACAT French Language Pack already installed, uninstall it. Go to Control Panel -> "Uninstall a Program". Right click on "Assistive Context-Aware Toolkit (ACAT) - French Language Pack" and select "Uninstall".
  3. Save ACATPack-French.exe (at the bottom of this section) on your computer.
  4. Run ACATPack-French.exe. Accept all defaults in the installation wizard. This will install the French language extensions. You must install this in same destination folder as the base English version. The default is C:\Intel\ACAT
  5. You can switch between languages either by selecting "Language" from the ACAT Config utility, or by selecting the "Language" option from the Main menu in ACAT App

Spanish Language Pack:

  1. First download and install the base English version before installing the language pack.
  2. If you have a previous version of the ACAT Spanish Language Pack already installed, uninstall it. Go to Control Panel -> "Uninstall a Program". Right click on "Assistive Context-Aware Toolkit (ACAT) - Spanish Language Pack" and select "Uninstall".
  3. Save ACATPack-Spanish.exe (at the bottom of this section) on your computer.
  4. Run ACATPack-Spanish.exe. Accept all defaults in the installation wizard. This will install the Spanish language extensions. You must install this in same destination folder as the base English version. The default is C:\Intel\ACAT
  5. You can switch between languages either by selecting "Language" from the ACAT Config utility, or by selecting the "Language" option from the Main menu in ACAT App

Portuguese Language Pack:

  1. First download and install the base English version before installing the language pack.
  2. Save ACATPack-Portuguese.exe (at the bottom of this section) on your computer.
  3. Run ACATPack-Portuguese.exe. Accept all defaults in the installation wizard. This will install the Spanish language extensions. You must install this in same destination folder as the base English version. The default is C:\Intel\ACAT
  4. You can switch between languages either by selecting "Language" from the ACAT Config utility, or by selecting the "Language" option from the Main menu in ACAT App

Extending ACAT to other languages

  1. Download ACATResources.en.resx at the bottom of this section
  2. Use a resource editor to translate strings. SimpleResx works quite well and it has built in support to translate strings into a number of languages.
  3. Generate the resource .RESX file in the target language.
  4. Email the translated resource .RESX file to Sai Prasad (sai.prasad@intel.com)
  5. Identify source material to create the word prediction database. The source should be free of copyright or other violations. Send us a link to the source.
  6. We will generate the word prediction database for the target language
  7. We will create the ACAT Language pack for the target language and upload it to github

Note:

  1. ACAT doesnt include Prof. Stephen Hawking's voice for text-to-speech. It uses Microsofts Speech Synthesizer API and the default TTS voice on the target machine. (Refer to ACAT FAQ or to the ACAT User Guide on how to change the default voice). Developers can easily extend ACAT to support other speech engines.
  2. The UI look and feel is different from the one used by Prof. Stephen Hawking. Developers can create new UI's to support the needs of different users.
  3. ACAT Vision is not open-source yet. We plan to release it in upcoming releases

ACAT Vision

ACAT Vision which is the vision component of ACAT uses the webcam to track the user's face and to detect facial gestures (cheek twitch and/or eyebrow raise). The detected gestures are then used as triggers to actuate the UI. ACAT Vision supports cheek twitch and eyebrow raise gestures. Only cheek twitch is enabled by default. To use eyebrow raise, or to map gestures to ACAT command shortcuts, refer to either ACAT FAQ or to the Configure Actuators section in the ACAT User Guide.

If you have any issues or questions about ACAT Vision, please contact Lama Nachman (lama.nachman@intel.com)

Tips

  1. Make sure the face is positioned 1 to 2 feet away from the camera.
  2. During the calibration phase, try to keep as still as possible. Do not make any head or facial movements.
  3. Face detection works well in most lighting conditions but if the room is too dark or (artificial/sun) light is shining directly on the face then face detection will work quite as well. Make sure the face is reasonably illuminated.
  4. For gesture detection to work correctly, you must keep your head perfectly still and 1) move only your cheek muscle or your mouth for a cheek twitch gesture or 2) quickly raise and lower your eyebrow for an eyebrow raise gesture
  5. You can press Ctrl+Alt+Shift+W (in that order) at any time to hide or show the Vision window
  6. If the calibration is taking too long, you can force ACAT Vision to recalibrate. First make sure the ACAT Vision window is visible. Click on it to set focus and press the R key to reset face detection. This is useful if there was head movement and facial regions are no longer in the correct place

Using Off-the-shelf Switches

Off-the-shelf assistive switches can be also used to trigger ACAT. They usually work by sending a configurable keystroke to the computer whenever they are activated. To work with ACAT, they should be configured to send an F12 key press on activation. If they do not support the F12 key, refer to ACAT FAQ for instructions on how to change the default trigger key for ACAT. Some switches directly plug into the USB port of the host computer, and others plug into a USB switch box. If the switch box supports multiple switches, they can be individually mapped to different ACAT commands for richer multi-modal interactions. Refer to ACAT FAQ for details. Here are a few recommendations for off-the-shelf switches.

  1. A Bluetooth switch from Ablenet. It can be configured to send an F12 key whenever the switch is activated. Has two switches. One of them can be used as the primary switch to trigger ACAT by mapping it to F12. The other switch can be mapped to one of the ACAT commands. Instructions to configure the switches can be found here.
  2. A range of mechanical switches from Ablenet.
  3. Stealth Switch from StealthSwitchAT. Lets you program the switch with keystrokes (use F12 for ACAT) and plays them back when you press the switch. Supports multiple switch inputs.
v0.99.1

ACAT Release 0.99.1

This is a binary-only release. Source code for this version will be released soon. Please report any issues/problems and also provide feedback before the official release.

New in this release

Refer to the ACAT User Guide for details on the new features.

  1. Spanish Language pack added. ACAT is now available in English, French and Spanish.
  2. Multi-modal gestures. Commands (ACAT shortcuts) can be mapped to gestures to allow richer interactions. For example, the eyebrow-raise gesture in ACAT Vision can be mapped to "undo the last editing change", or to "restart the scanning sequence". Mapping is done through the "Actuators" option in the ACAT Config utility.
  3. A new Phrases application that displays a list of configurable phrases to convert to speech. The app can be launched from the ACAT Dashboard.
  4. Select preferred Text-to-speech voice by voice name or by gender. Voice selection can be done through the "Text-to-speech" option in the ACAT Config utility
  5. A new color scheme for scanners . Color scheme can be selected from the ACAT Config utility.
  6. A new parameter to configure the number of iterations for typing accented letters in non-English languages. Parameter can be controlled through the "Scan" settings in the ACAT settings menu.

Bug Fixes

  1. Selecting the @ symbol in the Punctuation scanner would cause ACAT to crash
  2. Page-up/Page-down buttons were disabled in the File Browser and Switch Windows scanner even if the window contained more than a pageful of entries
  3. On Windows 10, the "Switch Windows" option would display hidden or cloaked windows such as Photos, Calculator, Movies & TV even though they were hidden.
  4. "Show Desktop" in Switch Windows" would not work on Windows 10
  5. On some Windows 10 machines, ACAT Vision would crash in acat_gestures_dll.dll
  6. On Windows 10, right clicking on an icon in the Windows taskbar would cause ACAT to become unresponsive
v0.99 ACAT Release v0.99

ACAT Release 0.99

This is a binary-only pre-release. Please report any issues/problems and also provide feedback before the official release.

Release notes

Localization

ACAT can now be extended to other languages. English and French versions are included in this release.

  1. Text in all the ACAT scanners, menus and dialogs are now localized. Word prediction results are displayed in the selected language as well.
  2. Customized Alphabet scanner for the selected language which includes scanners for special characters.
  3. Ability to switch between languages from within the ACAT App.
  4. Language extensions available as ACAT Language Packs which can be separately downloaded and installed.

ACAT Vision

  1. ACAT Vision not a standalone EXE anymore. It is now integrated with ACAT as an extension. TCP/IP is no longer used to communicate with ACAT Vision. This should eliminate problems some users were facing with blocked TCP ports.
  2. A more seamless interaction through appropriate visual feedback to the user.
  3. Migrated to 32-bit. Will run on both 32-bit and 64-bit Windows OSs.
  4. Ability to select the preferred camera if multiple cameras are present
  5. Ability to select the preferred facial gesture that will act as the switch trigger.

UI

  1. Additional Alphabet scanner layouts and scanning sequences designed towards making typing more efficient.
  2. A new ACAT Dashboard application to launch all the different variations of ACAT App and ACAT Talk app. Shortcuts to the preferred ACAT Apps can be selectively created on the desktop.
  3. A new ACAT Config application (non-assistive) for direct control of a host of ACAT settings which are not available through the Settings menu in ACAT App.
  4. ACAT Talk App now directly accessible from within the ACAT Application.

Other

  1. Support added for MS Word 2013 and Outlook 2013.
Other projects in C#