Wednesday, June 22, 2011

Interactive4J

About Interactive4J :

I have created this open source project aiming to provide simple easy APIs for Java developers to use interactive abilities in their Java Applications like speech recognition, handwriting recognition, use of web cam , sound record/play, decision trees , text to speech and many others, all with very simple APIs without any needed configurations or XML files,...etc. only simple method APIs.

Without further much enhancements to user experience in the standalone applications, users will prefer the web application for the huge features they provide, this open source enrich the user experience in such applications.

In the same time most of the developers will need to implement the typical and direct scenario in most of the supported features , Interactive4J provides this, in few cases developers need to customize the functionality and they will find project source code available so they can manipulate and use it directly as a reference implementation to the needed functionality.

This is to say moving these areas into easy usage will increase the interactivity of the application and the needs/dependency on them; which will push them more and more,so Interactive4J "When the easy way is the best way to do something!"

Features :

* Easy and simple APIs for developer with any experience to use them.
* Aim for Interactivity of Java Applications (for better user experience)
* Used for NextGen Java Application with higher user interactivity, if we didn't go for such enhancements in user experience , desktop application will fade soon in competition with web apps.
* Use some other open source projects and wrap them in a very simple way.
* Have a nice simple demo to show the supported features.
* Each functionality like text to speech , speech recongnition ,etc. have a very straightforward simple methods to use.
* Human abilites is the main purpose of the 1st release as see, hear , talk , translate, listen , recognize and think.
* Move some areas from scientific research into practical usage.
* Enhance the usage of Java in Educational puropose.
* Use the Standards whenever exist.
* Uses Sphinx-4 is a state-of-the-art speech recognition system written entirely in the JavaTM programming language
* Uses FreeTTS library for text to speech.
* Uses the library LTI-CIVIL which is part of FMJ (Freedom for Media in Java) for portability of WebCam usage.
* Uses Languages translation via open source "google-api-translate-java" for most possible interactivity with different users.
* Define a new JTextField component "JITextField" having JTextField + JButton for translation + JButton for text to speech
* Provides Genetic Algorithm and Decision Tree in a very simple and generic way for easy usage.
* Use Google Gson for JSON parsing and constructions.
* Use JMF (Java Media Framework) for creating movies from images and to play audio/video from the supported audio/video formats in JMF, click here to check them.

Releases :
Currently 2 separate releases one needs JMF support and one do not requires JMF support.

SourceForge URL :
http://sourceforge.net/projects/interactive4j/


How to Use it?



The demo represents some of the existing functionality in Interactive4J where you can simple initialize the feature you need and use it in a simple way..

Here is some of the important functionality in this release version, you must go through the Java-docs for more details about other not-listed features..

1) Text To Speech
The demo represents 2 ways of using this feature..

-Say it: where you pass the string you want the engine to say.
-Read it: where you do further manipulation to read word by word and highlight the current word.



The APIs is very simple : static method that take the string to say..No configuration files or any thing else!

JHSpeakUtils.speak(String);


2) Speech Recognition

Pass String[] with the speech recognition words! no thing else!

3 method control speech recognition:
- JHVoiceRecUtils speakUtils = new JHVoiceRecUtils(this,currentConfigPath);
To configure it to start (you should use the current configuration files without modification, but you can still modify them)
This = represents a class that implement the VoiceRecognitionListener where you get the recognized words in a method : setVoiceResult(String recognized)
- speakUtils.StartVoiceRecognition(commands);
Where you start the recongnition , commands here is a string[] contain all the words/letters you'd like to recognize.
- speakUtils.StopVoiceRecognition();
To stop the voice recognition.


3) Handwriting recognition

Construct the hand writing recognition with numbers or letter to be used and no thing else!

- handwriting = new JHHandWritingUtils(jPanel1, this,JHHandWritingUtils.MODE_NUMBERS,currentConfigPath);
You construct the handwriting utils with the panel you want the user to use it and listener of the results, plus mode (either number, capital letters or small letter), and finally the current configuration path (you shouldn't modify the given configurations.
- handwriting.startHandWritint();
To start the handwriting recognition..
- handwriting.stopHandWriting();
-To stop it

*So 3 modes is supported:
1-Numbers
2-Capital letters
3-Small letters

4) Record and play sounds/voices
3 methods:
- JHRecordingUtil recordingUtil=new JHRecordingUtil(maxSeconds);
construct the recorder with max seconds allowed.
- recordingUtil.startRecording("/recordSample.wav", 22000);
To start recording in a give file with sampling rate (this is configured being depend on the needed quality and it affect the recorded file size)

To play it : use the method:
recordingUtil.play("/recordSample.wav",1);
1 = one time only.
You have also stop method to stop it.

This is typically adapted for voice chat where you record a snapshot and send it and play the received snap voice and repeat the cycle..

5) Translate Text
One simple method provided to do the translation:
String trans = JHTranslateUtil.doTranslate(jTextArea1.getText(), Language.ENGLISH, Language.ARABIC);

Where you specify the text to translate and source/destination languages.

In the demo click on translate button translate text to Arabic and the 2nd click restore the original text.

6) Screen Capture
Only 1 simple method:
JHScreenCaptureUtil.takeScreenShot("/osama.jpg");
or
takeScreenAreaShot("/osama.jpg",x,y,width,height)

This facilitate creating desktop recording applications, if we conjugate it with voice/sound recording we can do such application in no time.

7) Web Camera Capture

Very easy and simple way encapsulate all complex logic behind to simple call with image file name to store the captured image into!

Simple single method you just need to specify the file name and the listener of this operation CaptureListener ..
mode=ONE_PHOTO;
webCam=new JHCaptureUtil(this, "/myWebCamPic.jpg");
webCam.start();


8) Web Camera Video Capture
This is actually a stream of jpg files displayed as if a video , this is the better approach for video chatting where you can send each picture separately, you can still construct a video file from these files.
mode=MULTI_PHOTO;
webCam=new JHCaptureUtil(this, "/myWebCamPic.jpg");
webCam.start();


9) Decision Tree
Use a simple way to design a decision tree and take the response to control whatever you need..
Here we used a game with decision tree to decided the PC movements..


10) Genetic Algorithm
Simple easy way to use the genetic algorithm in search problems in easy efficient way...prepare your problem and do search using 1 method!

// init population
Individual[] populations = new Individual[popSize];
for (int i = 0; i < popSize; i++) {
populations[i] = new Individual(numberOfGenes);
populations[i].randGenes();
}

// construct the evaluation function..
GeneticEvaluator geneticEvaluator = new GeneticEvaluator() {
public int evaluate(Individual individual) {
int fitness = 0;
......
return fitness;
}
public int evaluateSecondLevel(Individual individual) {
int fitness = 0;
......
return fitness;
}
};

//do genetic search now to get the best individual..by setting the configuration object with all details...
GeneticConfiguration config=new GeneticConfiguration();
config.setElitism(elitism);
config.setMaxFittness(maxFittness);
config.setMaxIterations(maxIterations);
config.setMutationRate(mutationRate);
config.setCrossoverRate(crossoverRate);
config.setPopulations(populations);
config.setGeneticEvaluator(geneticEvaluator);
config.setMulitFitness(true);
config.setChangeFitnessOnIteration(400);
Individual bestIndiv = JHGeneticUtil.executeJHGeneticUtil(config);

NOTE: this previous example uses multi-fitness functions where 2nd one is used after 400 iteration , this is not the typical genetic algorithm where you can switch this off by set it to false for the standard genetic algorithm implementation.

11) New Suggested JTextField style
This composed of JTextField plus 2 buttons; one for translation and one for text to speech..


12) JSON Utilities
Here you can convert from JSON into Object and vice versa using easily APIs provided by Google Gson.
Only you call toJson() and fromJson() according to conversion type.

13) Play Video files and Audio files
Built on JMF, play video/audio in the Swing easily by a single method:

JHPlayer createPlayer(JPanel pf, String media, ControllerListener listener)

14) Convert Series of images into movies

Build on JMF, , single method to create .mov file!

JHMovieMaker.createMovie(String movieName, int width , int height, Vector images, int frameRate)

For other features and functionality, go through the java-docs of the project.

Possible Applications and usages:

-Interactive Java Applications like Educational applications.
-Translation and multi-lingual applications.
-Book Readers and translators.
-Video chat applications.
-Screen capture software (outcome images or video)
-Command based applications like speech commands & handwriting; Interactive Browser is an example.
-AI applications.
-Monitoring applications like my home monitor application or Hidden parent eye application.

Some of these applications already implemented and you can find direct links @my blog for them.

14 comments:

  1. mr osama error running interactive4JDemo: com.lti.civil.CaptureException: java.lang.UnsatisfiedLinkError: no civil in java.library.path
    at com.lti.civil.impl.jni.NativeCaptureSystemFactory.createCaptureSystem(NativeCaptureSystemFactory.java:24)

    I've set the classpath for lti civil and add jar files. thanks in advance..

    ReplyDelete
  2. In the readme file, you will find the following section :
    the needed libraries for web camera module: -Djava.library.path="\......path......\lti-civil-20070920-1721\native\win32-x86"

    This varies according to the operating system.
    Please add this to the runtime JVM parameters and connect the USB camera and enjoy it!

    ReplyDelete
  3. thanks mr osama.. i can running the program..:)

    ReplyDelete
  4. Mr Osama,
    Currently i am working on an application where i need to store images which are captured from a web cam. I followed the sample code you ve post in a other page. but that JpegImageEncoder does not seem to be included in anywhere. I am using Java 6. do i need to switch to java 5 or is there anyway to continue with java 6?
    Please this is really urgent lot of work depend on this image store task

    ReplyDelete
  5. I am using Java 1.6 as well, there should be no issue on this.

    ReplyDelete
  6. I got the following error


    java.lang.NullPointerException
    at osa.ora.utils.JHCaptureUtil.onNewImage(JHCaptureUtil.java:119)
    java.lang.NullPointerException

    ReplyDelete
    Replies
    1. Mostly this means the camera is not connected correctly or not identified.

      Delete
    2. the camera has been detected

      Delete
    3. mode=ONE_PHOTO;
      webCam=new JHCaptureUtil(this, "/myWebCamPic.jpg");
      webCam.start();

      where I put this code?

      Delete
    4. In the action event of capture button :)

      Delete
  7. sir! on what library or jar file can i find this import

    import com.birosoft.liquid.LiquidLookAndFeel;

    because in your interactive4jDemo it cannot find this import. i tried adding almost all of the jar files in your interactive4j but still this error occur.

    ReplyDelete
  8. Sir i appreciate your effort , Can you help me in an application of speech recognition... im using sphinx 4... and i use it to give commands like "open calculator" and it opens also but it executes other commands also and opens "paint" and "player" also can you suggest something to me in this aspect. will be gratefull to you
    Regards Gursahib

    ReplyDelete
    Replies
    1. I have made similar staff in interactive java browser project.

      Delete