Sunday, January 31, 2016

NodeJS Quick Guide - Part 2

In this part we will use the express web application to develop a sample web application.

How to customize the application:
1) Change the port
We can change the default port to a new port or better use the runtime arguments to set it.
For example:
if (process.argv[2]!=null){
    var port = normalizePort(process.argv[2]);
} else {
    var port = normalizePort(process.env.PORT || '8000');
app.set('port', port);

This will let us use the application by providing the port as an argument as following:
set DEBUG=mytestapp:* & npm start 8002

2) Add images in the images folder:
Download any HR logo image and place it in the images folder.

3) Add or Edit Routes
We can edit the current pages or create a new routes.
Let's create header and footer pages and place all these files in the views folder
1) header.jade 
<html><head><title>#{title}</title></head><body><center><h1>#{title}</h1><img src="/images/hr-logo.jpg" width=200 height=100><hr><br>

2) footer.jade
<hr><div> (c) Copyright 2016 </div></center></body></html>

Now modify index.jade to be as following:
include ./header.jade
block content
  h1= title
  p Welcome to #{title}
p Enter an Employee number to see his/her data
</p><form name='users' action='/users' method='get'><input type='text' name='emp_num' /><input type='submit' value='Submit' /></form>
include ./footer.jade

As you can see in these files, we have mixed the html tags with jade syntax e.g. <p> or p
We use the format #{ ... } for variables.
We have included the header and footer files using include statement.
The form in index.jade submit the user inputs to the /users

The index.js should be modified a little bit to include the correct page title:
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res, next) {
  res.render('index', { title: 'HR Express' });
module.exports = router;

Let's now edit the user.js file to capture this input field and route it into details page.

var express = require('express');
var router = express.Router();
/* GET users listing. */
router.get('/', function(req, res, next) {
  res.render('emp', { title: 'HR Express', number: req.query.emp_num, name: 'Osama Oransa', position: 'PaaS Expert' });
module.exports = router;

You can see the query parameter is captured using req.query.parameter_name we have also set some variables, these variables should typically comes from the database tables.

Add emp.jade view with the following content:
include ./header.jade
block content
  h1= title
  p Welcome to #{title} - Employee Details
p Here is the Employee Data
P Number : #{number}
P Name : #{name}
P Position: #{position}
<a href='/'>Back</a>
include ./footer.jade

4) Test the application
We can now test the application by executing our command:
set DEBUG=mytestapp:* & npm start 8002

In the home page enter any employee number and click submit to view the details page.

5) Using Session object
To use the session object we need to install the following modules: connect and cookie-session.
npm install connect
npm install cookie-session

In app.js add the following to instantiate the session using cookies:
var connect = require('connect');
// store session state in browser cookie
var cookieSession = require('cookie-session');
    keys: ['secret1', 'secret2']

You can change the session keys to any unique keys.
Now in index.js modify the code to store a random guest name in the session.
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res, next) {
  if(req.session.guest==null) req.session.guest='Osama '+random(1,10);
  res.render('index', { title: 'HR Express','guest':req.session.guest });
module.exports = router;
function random (low, high) {
    return Math.floor(Math.random() * (high - low + 1) + low);

In users.js add the parameter as well to the variables:
/* GET users listing. */
router.get('/', function(req, res, next) {
  res.render('emp', { title: 'HR Express','guest':req.session.guest, number: req.query.emp_num, name: 'Osama Oransa', position: 'PaaS Expert' });

Finally in the header.jade add a p for the guest name just above the image:

p Welcome #{guest}

6) Test the application
Now test the application using 2 different browsers to see the effect of the session.

You can see the guest name is now below the title, test using different browsers.

7) Connect to the DB
Now to connect the database we have many options, we can connect on SQL or NO-SQL database for example to use Oracle Database you need to modify the code to query the database.
Oracle provides a DB driver for NodeJS can be found in the following URL:
To Install it, you'll need the following steps:
    - Python 2.7
    - C Compiler with support for C++ 11 (Xcode, gcc, Visual Studio or similar)
    - The small, free Oracle Instant Client libraries if your database is remote. Or use a locally installed database such as the free Oracle XE release
    - Set OCI_LIB_DIR and OCI_INC_DIR during installation if the Oracle libraries and headers are in a non-default location
 npm install oracledb
This will install it from the NPM registry.

Let's now modify the code to get the employee details from the DB (Oracle HR sample schema) as following:

var oracledb = require('oracledb');
    user          : "hr",
    password      : "hr",
    connectString : "localhost/XE"
  function(err, connection)
    if (err) { console.error(err.message); return; }
      "Sselect first_name||' '||last_name name, job_title "+
      " FROM employees,jobs"+
      " WHERE employees.job_id=jobs.job_id"+
      " and employee_id = :id ",
      [110],  // bind value for :id 

{outFormat: oracledb.OBJECT}, // outFormat can be OBJECT or ARRAY. The default is ARRAY
       function(err, result)
        if (err) { console.error(err.message); return; }

 We need to replace the [110] with our employee parameter: req.query.emp_num,
 And pass the result into our emp.jade page to render the employee details.
result[0].name & result[0].job_title
We need also to route the errors into error pages.
You can see here a lot of examples of how to use Oracle driver:

This is a simple application to show us how to use some of the NodeJS features in nutshell.

NodeJS Quick Guide - Part 1

NodeJS developed by Ryan Dahl in 2009 which runs using Chrome JavaScript engine plus some libraries for asynchronous network and I/O modules.
It targets building high performance and scalable server and client applications. Node.js uses Just-in-Time compilation with the V8 JavaScript Engine.

1) Installing NodeJS
The quickest and easiest way is to install it from the web site.

Once installed from the command line you can write node or npm to ensure everything is fine.

2) NodeJS
Uses the  java script language where you can write loosely typed event based asynchronous code.
Loosely typed: means the variable types can be determined at runtime not compilation time as in Java which gives power but could lead to potential errors as you already lost the compiler error scanning.
Event-based: means you need to fire action and wait for the response/output in a function that process this output so instead of waiting the DB operation to finish fire the query and wait the response in a function, this unlike the thread based languages such as Java where you need to wait for the response , the gain is nullifying the context switching and simplify the code, which gives higher performance in throughput of the application.
So every function that requests IO has a signature like function (... parameters ..., callback) and needs to be given a callback that will be invoked when the requested operation is completed i.e. Continuation-passing style.

Note that Node is not suitable for high computation operations and it needs good design to process such operations without impacting the application.
In spite NodeJS is a single threaded, and the potential feature for fork child-processes or threads is rolled back, some modules provide a way to run some threads for high computation consuming operations one example is Threads a gogo

3) NodeJS Modules:
Node is based on modules which is similar concept to packages in Java, the modules are used by the statement:
var http = require('http');    ==> This means the http variable will be used to access the exposed functions in the http module.
The http module as an example here can expose different functions now by the following statement: exports.function_name e.g. :
exports.stop = function(){
console.log('Inside the stop function');

One of the big issues in JavaSciprt is the global variables, using the modules and exports you can control what you need to expose and any global variable is now considered local to that module and won't interfere with any similar name variable in your application.

To install modules that you need in your application, you need to use npm where you specify if you need to install the module for your application or globally for all applications by using:

npm install -g express ==> this will install express a web application framework based on the open source connect project.
if you omit -g you will install the packages locally for your application.
The modules are loaded into the node_modules folder either in your application or globally.

4) Running a hello world application
The following sample application uses the http module to start a server:
var http = require('http');
exports.start = function(){
server=http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, World!\n');
}).listen(8082, '');
console.log('Server is now running:');

Save it into a file name; app.js
now run node
> node
> var app=require('./app');
> app.start();

Server is now running:

Open the browser on the port 8082 using localhost:8082 you can see hello world message is printed out.
In our application we have exported start(); function() and from the node interpreter we have loaded our application module and executed this exposed start() function.
Our application uses the http module to run the http server.
You can run wrap it in another application
var app = require('./app');

Save it as app-runner.js and now run it using:
node app-runner

another way is to use the package.json file which describes your application and dependencies
create the file package.json and add the following JSON content:
{ "name": "AppRunner",
  "version": "1.0.0",
  "scripts": {
    "start": "node ./app-runner"
  "dependencies": {

NOTE: to create the file you can execute npm init to create this package.json file.
It simply shows the application name, version, start script and any dependencies should be added to that section.
Now we can run the application using the command
npm start ==> which will search for that file and run the start command.
also npm install ==> will install all the required modules in the dependencies section in the local application directory.
Finally npm list shows the installed modules for this application.

5)  Using express web application framework
Express is a Web Application Framework for node:
- Install express and express-generator
npm install express -g
npm install express-generator -g

Now use it to create a sample web application
express mytestapp   ==> This will generate the web application framework in the mytestapp folder
cd mytestapp ==> you can see the main application app.js and package.json where you need to add any future dependencies.
npm install ==> to install all required modules locally.
set DEBUG=mytestapp:* & npm start ==> in windows to start the application server
Test the application in the browser using localhost:3000

The following contains the APIs Guide for using Express:

The generated sample application contain the following main components:

1) Package.json file
In that file we can see the start script is
"scripts": {
    "start": "node ./bin/www"

If we opened this file /bin/www , we can see that it uses the app.js, do the port configurations and the initialization of http server, which means if environment variable PORT is set, use it otherwise use the port 3000.
var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

We can change this port 3000 into any port e.g. 8082

2) View folder for views
We can add our views/pages here.
app.set('views', path.join(__dirname, 'views'));
The view engine to parse the views is specified as:
app.set('view engine', 'ejs');
app.set('view engine', 'jade');
You can use both options to build your view files.
Here is a quick guide for jade:
And here is a quick guide for ejs:

3) Public folder for static contents:
This folder for storing the images, stylesheets and client side java script files.

4) Router to router the requests.
var routes = require('./routes/index');
var users = require('./routes/users');
app.use('/', routes);
app.use('/users', users);

one is index.js for the / path which is placed in the routes folder and another one is users.js for the path /users and is placed as well in routes folder.
You can replace content in both files to see the impact of opening both URLs and

In part II, we will continue the application exploration by adding different pages and using parameters, forms, sessions and access the database to retrieve data.

Wednesday, January 27, 2016

App Engine's Datastore

     App Engine's datastore is built on top of Bigtable. To use App Engine we need to create account using our gmail, create identifier/key for our application and finally create our application in Eclipse.
You can install Google App Engine Plugin in any Eclipse e.g. in Luna use the following URL:

This represents all Google tools, select the App Engine tools.

To use Google datastore from inside our Java code we can use one of the following three methods:

A) Using Java Data Objects (JDO) 

JDO is a standard interface for accessing databases in Java, providing a mapping between Java classes and database tables.
Let's explore how to use JDO to access datastore:
We can follow the following simple steps:

1) Create the Entity Class(es) :
public class PersonLocation implements Serializable {
private static final long serialVersionUID = 1L;
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
public Long id;
public int personId;
public double lat;
public double lon;
public double accuracy;
+ settings, getters and constructors.

2) Get the persistent manager factory:

PersistenceManagerFactory pmfInstance = JDOHelper

3) Get PersistenceManager from the factory and use it for CRUD operations:
PersistenceManager pm = pmfInstance.getPersistenceManager();

4) Create Entity :
Create an object from the entity and persist it.
PersonLocation location = null;
try {
location=new PersonLocation();
} finally {
Once the entity persistent you can get the id.

5) Get Entity:
Either use the getObjectById as in the previous methods or execute datastore query to retireve the requied data.
location = pm.getObjectById(PersonLocation.class, id);

Or use the Query object, the query retrieves all entities of the given kind that satisfy all of the given filters, sorted in the specified order.
    public List getAllLocationsForEntity(int entityId){
PersistenceManager pm = pmfInstance.getPersistenceManager();
String queryStr = "select from " + PersonLocation.class.getName();
Query selectQuery=pm.newQuery(queryStr);
return (List) selectQuery.execute();

6) Update Entity: 
Get the entity, update the entity and persist the updates.
try {
// We do lookup for the required id
location = pm.getObjectById(PersonLocation.class, id);
} catch (Exception ex) {
throw new RuntimeException(ex);
} finally {

7) Delete Entity: 
Get the entity and delete it.
try {
// We do lookup for the required id
location = pm.getObjectById(PersonLocation.class, id);
} catch (Exception ex) {
throw new RuntimeException(ex);
} finally {

B) Using JPA

Similarly to using JPA:
1) In persistence.xml
2) Get the EntityManagerFactory:
 private static final EntityManagerFactory emfInstance =
3) Get the EntityManager from the EntityManagerFactory:
EntityManager em = emfInstance.createEntityManager();
4) Annotate your entity:
public class Employee {
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Key key;
    private String firstName;
    private String lastName;}
... setters, getters and constructors
5) Use the entity manager to do the JPA CRUD operations. 

C) Using the low-level Datastore API, or one of the open-source APIs developed specifically for Datastore, such as Objectify. 

This is recommended by Google as JDO and JPA was designed for use with traditional relational databases, and so has no way to explicitly represent some of the aspects of Datastore that make it different from relational databases, such as entity groups and ancestor queries. This can lead to subtle issues that are difficult to understand and fix.

Sample Code:
class Car {
    @Id String vin; // Can be Long, long, or String
    String color;

ofy().save().entity(new Car("123123", "red")).now();
Car c = ofy().load().type(Car.class).id("123123").now();

Sunday, December 13, 2015

Simplified Guide To Use MongoDB

MongoDB is one of the No-SQL database engines, it is Document Oriented Database.
Documents can represent any structure that can fit JSON format so for example the following is a car document :
        "_id" : 6,
        "Car" : "BMW",
        "Model" : "S5",
        "Price" : 3455,
        "Color" : "Blue",
        "Date" : "09/10/2015",
        "Full Options" : "YES",
        "Level" : 1

Simplified Steps to use MongoDB:

1) Install the database locally...

Download it and install it.
From the command line execute: mongod and specify the location of the database files (any valid path so it can be used for storing database files).
mongod --dbpath c:\MongoDB\Database
All database command lines exist in the bin folder under Server directory.

2) Create any test data ..

You can create .csv file or .json file the simplest and quickest way is to use .csv file.
For example:
_id,Car,Model,Price,Color,Date,Full Options,Level

As you can see the _id is the primary key in MongoDB (if you didn't specify it, MongoDB will generate it for these entries.

3) Import test data

From the command line run the following commands:
mongoimport -d catalog -c cars --file cars.csv --type csv --headerline
This will import our file into Catalog database (similar to Schema in SQL DB) in a collection named cars (similar to DB table in SQL DB)
Note: if you import json file you don't need to specify the --type or --headline

 4) Manipulate MongoDB data 

Execute mongo from the command line ...
The default will connect to the localhost running instance ...
To connect to our database (schema) that we have created for cars, execute:
 use catalog
Now we can see all existing collections i.e. tables by executing:
 show collections
which will display only "cars" collection.
To deal with "cars" we can use the following:    
To retrieve the count of the rows in this collection.
To select all cars records{"Level":1,"Color":"Black"})
To select cars with level 1 and color is black.{"Price":{$gt:2000}})
To select cars with price more than 2000.{"Price":{$lte:3000},"Color":/l/}).pretty()
To select cars with price less than 3000 and color contain a "l" letter inside and then format it in JSON readable format.{"Price":{$gte:1000}}).sort({"Price":1}).skip(1).limit(2).pretty()
To select all cars with price more than or equal 1000 and sort the return by price ascending and limit the output to 2 only after skip the 1st results (we can use -1 to sort the result descending).
Note: You can use findOne() instead of find() to return the 1st match only.
To insert a new record :{ "_id" : 6, "Car" : "BMW", "Model" : "S5", "Price" : 3455, "Color" : "Blue", "Date" : "09/10/2015", "Full Options" : "YES", "Level" : 1 })
 To update the price of this inserted record we can use:{_id:6},{$set:{"Price":5432}})
To delete a row in the collection use:{"_id":5})
To drop the whole collection: (don't execute it now) analyze a query performance we can execute explain plan as following:{"Color":"Red"}).explain()
We can create index to speed up the query by the following:{"Color":1}) 
This means we will create index for the Color field in ascending manner.
Re-execute:{"Color":"Red"}).explain() to see the effect of this index.

5) Use MongoDB from inside Java Code

1- Add Mongo required drivers to your project classpath ...
 - mongodb-driver-3.0.4.jar
 - mongodb-driver-core-3.0.4.jar
 - bson-3.0.4.jar

2- Write your code to connect to the Database as following:
MongoClient mongoClient = new MongoClient();
This connect to a local running mongoDB on a default a port.
MongoDatabase db = mongoClient.getDatabase("catalog");
This connect to our catalog database.
FindIterable iterable = db.getCollection("cars").
                find(new Document("Level", 1)).
        iterable.forEach(new Block() {
            public void apply(final Document document) {

This query our cars collection for Cars with level 1 and sort them ascending by Price, iterate over the output and print each Document (i.e. JSON string).
 To insert an entry use the same way:
String json = "{ \"_id\" : 5, \"Car\" : \"Kia\", \"Model\" : \"Picanto\", \"Price\" : 777, \"Color\" : \"Blue\", \"Date\" : \"09/09/2011\", \"Full Options\" : \"NO\", \"Level\" : 3 }";

You can convert Bean objects into JSON and vice versa using any existing APIs..

This is simplified overview that covers most of MongoDB in nutshell and let you master how to deal with this database in just 5 minutes.

Tuesday, December 23, 2014

Time To Plan For Your Next Year

As the new year is approaching it is the best time these days to look forward to your next year technical plan, now if you don't have a mentor please try to find a mentor for yourself.
Next thing you need to do is to decide which technical topic you want to learn, putting in your mind that in the technical field "Knowledge is our Future" ...
One of the worth mentioning offers here is Packt Pub campaign where they offer eBooks and videos for just 5$ so this is your chance to get the latest technology books and plan for your next year well.

Here is the press release I received from Packt Pub:

Packt’s $5 eBonanza returns

Following the success of last year’s festive offer, Packt Publishing will be celebrating the holiday season with an even bigger $5 offer.  
From Thursday 18th December, every eBook and video will be available on the publisher’s website for just $5. Customers are invited to purchase as many as they like before the offer ends on Tuesday January 6th, making it the perfect opportunity to try something new or to take your skills to the next level as 2015 begins.
With all $5 products available in a range of formats and DRM-free, customers will find great value content delivered exactly how they want it across Packt’s website this Xmas and New Year.
Find out more at

Wednesday, August 13, 2014

Android HTML 5 Parameter Passing Between Pages

Developing HTML 5 mobile application for Android is straightforward as we see before, one of the challenges in developing mobile applications using HTML 5 is passing parameters between pages, the main reason for that is some bugs that exist in some Android versions, let's see the possible ways to overcome this issue:

1) Using URL Parameters

In that case while you are routing your screen to another screen add the required parameters to the page URL.
For example:
In that example we have passed 2 parameters; "id" and "type" with the corresponding values.
This way fail due to Android bug and will mostly show "Page not found!" as it will search for the page that match the whole URL !

2) Using localStorage

In that way submitting the page will save the selected parameters into localStorage and the target page will reload these parameters in its onLoad() method.
For example:
< img src="images/index/84.png" onClick="goDetails(84)">

function goDetails(itemId){

In the target page, load the stored value:

< body onload="init();">

function init(){
      //do any required logic here ...

3) Using Android JavaScriptInterface

In this way we will use the Android code to store the selected parameters as instance variables so we can retrieve them from the target page.

For example:
function goDetails(itemId){
function init(){
And in your Activity add the following JavaScriptInterface methods:

public class MyJavascriptInterface {
    Context mContext;
    String id="114";
    MyJavascriptInterface(Context c) {
            mContext = c;
    public String getCurrentId() {
return id;
    public void setCurrentId(String current) {
id= current;

We used the "String id" to store the value..

And add it in the WebView we add our JavaScriptInterface with the prefix as following:
     webView.addJavascriptInterface(new MyJavascriptInterface(this), "Android");