Offline Speech Recognition With PocketSphinx

Follow the below steps to run AndroidPocketSphinxDemo project provided by Sphinx community:

1. Download the latest version of sphinxbase from the following link:
    http://sourceforge.net/projects/cmusphinx/files/sphinxbase/
    Extract the downloaded tar file and save it under a folder called 'sphinx' .

2. Download the latest version of pocketsphinx from the link:
    http://sourceforge.net/projects/cmusphinx/files/pocketsphinx
    Extract the downloaded tar file and save it under a folder named 'sphinx' .

3. After extracting sphinxbase and pocketsphinx, you will have to install both 
    in your system through the following commands,

•     cd <sphinxbase path or pocketsphinx path>
•     ./configure
•     make
•     make install

4. Make sure that you have installed Android NDK, if not installed, refer
    the link:
    http://developer.android.com/sdk/ndk/index.html

5. Download AndroidPocketSphinxDemo project from the link:
    https://github.com/cjac/cmusphinx/tree/trunk/PocketSphinxAndroidDemo

6. Copy the following files from <pocketsphinx path>/swig/edu/cmu/pocketsphinx
    to PocketSphinxDemo/jni/edu.cmu.pocketsphinx. The required files are:

•     Config.java
•     Decoder.java
•     Hypothesis.java
•     pocketsphinx.java
•     pocketsphinxJNI.java
•     SegmentIterator.java

7. Specify the path of 'sphinx' folder containing pocketsphinx and sphinxbase in
    'Android.mk' file. Then do Ndk build.

8. In the emulator/device in which you are going to install the project, create the
    folder called 'edu.cmu.pocketsphinx' under the path '/sdcard/Android/data/' .
   
9. Under the folder 'edu.cmu.pocketsphinx', create two folders called 'hmm' and
    'lm'. Under both 'hmm' and 'lm' create folder called 'en_US'.

10. Inside '/sdcard/Android/data/edu.cmu.pocketsphinx/hmm/en_US', copy the
      folder 'hub4wsj_sc_8k' present under   
      <pocketsphinx path>/model/hmm/en_US

11. Inside  '/sdcard/Android/data/edu.cmu.pocketsphinx/lm/en_US', copy the files
      'hub4.5000.dic' and 'hub4.5000.DMP' present under
      <pocketsphinx path>/model/lm/en_US

12. Now the project is ready to run on the emualtor/device.

Android Architecture

 

                                           ANDROID ARCHITECTURE

Android Applications: 
                                        Android Applications are written in java. Android will ship with a set of core applications including an email client, SMS program, calendar, maps, browser, contacts, and others.

Application Framework: 
                                             By providing an open development platform, Android offers developers the ability to build extremely rich and innovative applications. Developers are free to take advantage of the device hardware, access location information, run background services, set alarms, add notifications to the status bar, and much, much more.

Developers have full access to the same framework APIs used by the core applications. The application architecture is designed to simplify the reuse of components; any application can publish its capabilities and any other application may then make use of those capabilities (subject to security constraints enforced by the framework). This same mechanism allows components to be replaced by the user.

Underlying all applications is a set of services and systems, including: 

• An Activity Manager manages the lifecycle of applications and provides a common navigation backstack for applications, which are running in different processes.
• An Window Manager handles all application related windows. It is a java programming language abstraction on the top of lower services that are provided by the Surface Manager.
• Content Providers enable applications to access data from other applications (such as Contacts), or to share their own data.
• View System provides all the standard widgets to build an application, including lists, grids, text boxes, buttons, and even an embeddable web browser.
• Package Manager keeps track of the applications, which are installed in the device.
• The Telephony Manager contains a set of API necessary for calling applications.
• The Resource Manager is used to store localized strings, bitmaps, layout file descriptions and other external parts of the applications.
• Location Manager provides APIs to determine location and bearing of the underlying device (if available). 
• The Notification Manager is an Android system service that executes and manages all Notifications.

Android Runtime:
     

• Dalvik Virtual Machine: An interpreter-only virtual machine (no JIT), register based.
  
  •       Designed specifically for Android running in limited environment where the limited battery, CPU, memory and data storage are the main issues.
  •       Relying on underlying OS for process isolation, memory management and threading support.
  •       Executes Dalvik Executables (DEX) files which are zipped into an Android Package (APK).

          Note: Android gives an integrated tool "dx", which converts generated byte code from .jar to .dex, after this byte code becomes more efficient to run on the small processors.

• Core Libraries are written in Java Language and contains Collection classes, Utilities, IO and other tools.

Native Libraries:  

                                Android contains set of native libraries written in C/C++, which are responsible for stable performance of various components. For example,

• Surface Manager is responsible for composing different drawing surfaces on the mobile screen. It manages the access for different processes to compose 2D and 3D graphic layers.  
• OpenGL ES and SGL make a core of graphic libraries and are used accordingly for 2D and 3D hardware acceleration. Moreover it is possible to use 2D and 3D graphics in the same application in Android.
• The Media framework was provided by PocketVideo. It gives libraries for a playback and recording support for all the major media and static image files.
• FreeType libraries are used to render all the bitmap and vector fonts.
• Android uses SQLite for data storage. SQLite is a extra light rational management system, which locates a single file for all operations related to database.
• WebKit, the same browser used by Apple's Safari, was modified by Android in order to fit better in a small size screens.

Linux Kernel:

                          Android Architecture is based on Linux 2.6 kernel. It helps to manage security, memory management, process management, network stack, and driver model.The kernel also acts as an abstraction layer between the hardware and the rest of the software stack. Includes following drivers in it:

• Display Driver
• Camera Driver
• Flash Memory Driver
• Binder (IPC) Driver
• Keypad Driver
• WiFi Driver
• Audio Drivers

Some common features have been removed:

•     No GBLIC support
•     No native windowing system
•     Does not include the full set of Linux utilities