Sunday, January 31, 2016

NodeJS Quick Guide - Part 2

In this part we will use the express web application to develop a sample web application.

How to customize the application:
1) Change the port
We can change the default port to a new port or better use the runtime arguments to set it.
For example:
if (process.argv[2]!=null){
    var port = normalizePort(process.argv[2]);
} else {
    var port = normalizePort(process.env.PORT || '8000');
app.set('port', port);

This will let us use the application by providing the port as an argument as following:
set DEBUG=mytestapp:* & npm start 8002

2) Add images in the images folder:
Download any HR logo image and place it in the images folder.

3) Add or Edit Routes
We can edit the current pages or create a new routes.
Let's create header and footer pages and place all these files in the views folder
1) header.jade 
<html><head><title>#{title}</title></head><body><center><h1>#{title}</h1><img src="/images/hr-logo.jpg" width=200 height=100><hr><br>

2) footer.jade
<hr><div> (c) Copyright 2016 </div></center></body></html>

Now modify index.jade to be as following:
include ./header.jade
block content
  h1= title
  p Welcome to #{title}
p Enter an Employee number to see his/her data
</p><form name='users' action='/users' method='get'><input type='text' name='emp_num' /><input type='submit' value='Submit' /></form>
include ./footer.jade

As you can see in these files, we have mixed the html tags with jade syntax e.g. <p> or p
We use the format #{ ... } for variables.
We have included the header and footer files using include statement.
The form in index.jade submit the user inputs to the /users

The index.js should be modified a little bit to include the correct page title:
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res, next) {
  res.render('index', { title: 'HR Express' });
module.exports = router;

Let's now edit the user.js file to capture this input field and route it into details page.

var express = require('express');
var router = express.Router();
/* GET users listing. */
router.get('/', function(req, res, next) {
  res.render('emp', { title: 'HR Express', number: req.query.emp_num, name: 'Osama Oransa', position: 'PaaS Expert' });
module.exports = router;

You can see the query parameter is captured using req.query.parameter_name we have also set some variables, these variables should typically comes from the database tables.

Add emp.jade view with the following content:
include ./header.jade
block content
  h1= title
  p Welcome to #{title} - Employee Details
p Here is the Employee Data
P Number : #{number}
P Name : #{name}
P Position: #{position}
<a href='/'>Back</a>
include ./footer.jade

4) Test the application
We can now test the application by executing our command:
set DEBUG=mytestapp:* & npm start 8002

In the home page enter any employee number and click submit to view the details page.

5) Using Session object
To use the session object we need to install the following modules: connect and cookie-session.
npm install connect
npm install cookie-session

In app.js add the following to instantiate the session using cookies:
var connect = require('connect');
// store session state in browser cookie
var cookieSession = require('cookie-session');
    keys: ['secret1', 'secret2']

You can change the session keys to any unique keys.
Now in index.js modify the code to store a random guest name in the session.
var express = require('express');
var router = express.Router();
/* GET home page. */
router.get('/', function(req, res, next) {
  if(req.session.guest==null) req.session.guest='Osama '+random(1,10);
  res.render('index', { title: 'HR Express','guest':req.session.guest });
module.exports = router;
function random (low, high) {
    return Math.floor(Math.random() * (high - low + 1) + low);

In users.js add the parameter as well to the variables:
/* GET users listing. */
router.get('/', function(req, res, next) {
  res.render('emp', { title: 'HR Express','guest':req.session.guest, number: req.query.emp_num, name: 'Osama Oransa', position: 'PaaS Expert' });

Finally in the header.jade add a p for the guest name just above the image:

p Welcome #{guest}

6) Test the application
Now test the application using 2 different browsers to see the effect of the session.

You can see the guest name is now below the title, test using different browsers.

7) Connect to the DB
Now to connect the database we have many options, we can connect on SQL or NO-SQL database for example to use Oracle Database you need to modify the code to query the database.
Oracle provides a DB driver for NodeJS can be found in the following URL:
To Install it, you'll need the following steps:
    - Python 2.7
    - C Compiler with support for C++ 11 (Xcode, gcc, Visual Studio or similar)
    - The small, free Oracle Instant Client libraries if your database is remote. Or use a locally installed database such as the free Oracle XE release
    - Set OCI_LIB_DIR and OCI_INC_DIR during installation if the Oracle libraries and headers are in a non-default location
 npm install oracledb
This will install it from the NPM registry.

Let's now modify the code to get the employee details from the DB (Oracle HR sample schema) as following:

var oracledb = require('oracledb');
    user          : "hr",
    password      : "hr",
    connectString : "localhost/XE"
  function(err, connection)
    if (err) { console.error(err.message); return; }
      "Sselect first_name||' '||last_name name, job_title "+
      " FROM employees,jobs"+
      " WHERE employees.job_id=jobs.job_id"+
      " and employee_id = :id ",
      [110],  // bind value for :id 

{outFormat: oracledb.OBJECT}, // outFormat can be OBJECT or ARRAY. The default is ARRAY
       function(err, result)
        if (err) { console.error(err.message); return; }

 We need to replace the [110] with our employee parameter: req.query.emp_num,
 And pass the result into our emp.jade page to render the employee details.
result[0].name & result[0].job_title
We need also to route the errors into error pages.
You can see here a lot of examples of how to use Oracle driver:

This is a simple application to show us how to use some of the NodeJS features in nutshell.

NodeJS Quick Guide - Part 1

NodeJS developed by Ryan Dahl in 2009 which runs using Chrome JavaScript engine plus some libraries for asynchronous network and I/O modules.
It targets building high performance and scalable server and client applications. Node.js uses Just-in-Time compilation with the V8 JavaScript Engine.

1) Installing NodeJS
The quickest and easiest way is to install it from the web site.

Once installed from the command line you can write node or npm to ensure everything is fine.

2) NodeJS
Uses the  java script language where you can write loosely typed event based asynchronous code.
Loosely typed: means the variable types can be determined at runtime not compilation time as in Java which gives power but could lead to potential errors as you already lost the compiler error scanning.
Event-based: means you need to fire action and wait for the response/output in a function that process this output so instead of waiting the DB operation to finish fire the query and wait the response in a function, this unlike the thread based languages such as Java where you need to wait for the response , the gain is nullifying the context switching and simplify the code, which gives higher performance in throughput of the application.
So every function that requests IO has a signature like function (... parameters ..., callback) and needs to be given a callback that will be invoked when the requested operation is completed i.e. Continuation-passing style.

Note that Node is not suitable for high computation operations and it needs good design to process such operations without impacting the application.
In spite NodeJS is a single threaded, and the potential feature for fork child-processes or threads is rolled back, some modules provide a way to run some threads for high computation consuming operations one example is Threads a gogo

3) NodeJS Modules:
Node is based on modules which is similar concept to packages in Java, the modules are used by the statement:
var http = require('http');    ==> This means the http variable will be used to access the exposed functions in the http module.
The http module as an example here can expose different functions now by the following statement: exports.function_name e.g. :
exports.stop = function(){
console.log('Inside the stop function');

One of the big issues in JavaSciprt is the global variables, using the modules and exports you can control what you need to expose and any global variable is now considered local to that module and won't interfere with any similar name variable in your application.

To install modules that you need in your application, you need to use npm where you specify if you need to install the module for your application or globally for all applications by using:

npm install -g express ==> this will install express a web application framework based on the open source connect project.
if you omit -g you will install the packages locally for your application.
The modules are loaded into the node_modules folder either in your application or globally.

4) Running a hello world application
The following sample application uses the http module to start a server:
var http = require('http');
exports.start = function(){
server=http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, World!\n');
}).listen(8082, '');
console.log('Server is now running:');

Save it into a file name; app.js
now run node
> node
> var app=require('./app');
> app.start();

Server is now running:

Open the browser on the port 8082 using localhost:8082 you can see hello world message is printed out.
In our application we have exported start(); function() and from the node interpreter we have loaded our application module and executed this exposed start() function.
Our application uses the http module to run the http server.
You can run wrap it in another application
var app = require('./app');

Save it as app-runner.js and now run it using:
node app-runner

another way is to use the package.json file which describes your application and dependencies
create the file package.json and add the following JSON content:
{ "name": "AppRunner",
  "version": "1.0.0",
  "scripts": {
    "start": "node ./app-runner"
  "dependencies": {

NOTE: to create the file you can execute npm init to create this package.json file.
It simply shows the application name, version, start script and any dependencies should be added to that section.
Now we can run the application using the command
npm start ==> which will search for that file and run the start command.
also npm install ==> will install all the required modules in the dependencies section in the local application directory.
Finally npm list shows the installed modules for this application.

5)  Using express web application framework
Express is a Web Application Framework for node:
- Install express and express-generator
npm install express -g
npm install express-generator -g

Now use it to create a sample web application
express mytestapp   ==> This will generate the web application framework in the mytestapp folder
cd mytestapp ==> you can see the main application app.js and package.json where you need to add any future dependencies.
npm install ==> to install all required modules locally.
set DEBUG=mytestapp:* & npm start ==> in windows to start the application server
Test the application in the browser using localhost:3000

The following contains the APIs Guide for using Express:

The generated sample application contain the following main components:

1) Package.json file
In that file we can see the start script is
"scripts": {
    "start": "node ./bin/www"

If we opened this file /bin/www , we can see that it uses the app.js, do the port configurations and the initialization of http server, which means if environment variable PORT is set, use it otherwise use the port 3000.
var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

We can change this port 3000 into any port e.g. 8082

2) View folder for views
We can add our views/pages here.
app.set('views', path.join(__dirname, 'views'));
The view engine to parse the views is specified as:
app.set('view engine', 'ejs');
app.set('view engine', 'jade');
You can use both options to build your view files.
Here is a quick guide for jade:
And here is a quick guide for ejs:

3) Public folder for static contents:
This folder for storing the images, stylesheets and client side java script files.

4) Router to router the requests.
var routes = require('./routes/index');
var users = require('./routes/users');
app.use('/', routes);
app.use('/users', users);

one is index.js for the / path which is placed in the routes folder and another one is users.js for the path /users and is placed as well in routes folder.
You can replace content in both files to see the impact of opening both URLs and

In part II, we will continue the application exploration by adding different pages and using parameters, forms, sessions and access the database to retrieve data.

Wednesday, January 27, 2016

App Engine's Datastore

     App Engine's datastore is built on top of Bigtable. To use App Engine we need to create account using our gmail, create identifier/key for our application and finally create our application in Eclipse.
You can install Google App Engine Plugin in any Eclipse e.g. in Luna use the following URL:

This represents all Google tools, select the App Engine tools.

To use Google datastore from inside our Java code we can use one of the following three methods:

A) Using Java Data Objects (JDO) 

JDO is a standard interface for accessing databases in Java, providing a mapping between Java classes and database tables.
Let's explore how to use JDO to access datastore:
We can follow the following simple steps:

1) Create the Entity Class(es) :
public class PersonLocation implements Serializable {
private static final long serialVersionUID = 1L;
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
public Long id;
public int personId;
public double lat;
public double lon;
public double accuracy;
+ settings, getters and constructors.

2) Get the persistent manager factory:

PersistenceManagerFactory pmfInstance = JDOHelper

3) Get PersistenceManager from the factory and use it for CRUD operations:
PersistenceManager pm = pmfInstance.getPersistenceManager();

4) Create Entity :
Create an object from the entity and persist it.
PersonLocation location = null;
try {
location=new PersonLocation();
} finally {
Once the entity persistent you can get the id.

5) Get Entity:
Either use the getObjectById as in the previous methods or execute datastore query to retireve the requied data.
location = pm.getObjectById(PersonLocation.class, id);

Or use the Query object, the query retrieves all entities of the given kind that satisfy all of the given filters, sorted in the specified order.
    public List getAllLocationsForEntity(int entityId){
PersistenceManager pm = pmfInstance.getPersistenceManager();
String queryStr = "select from " + PersonLocation.class.getName();
Query selectQuery=pm.newQuery(queryStr);
return (List) selectQuery.execute();

6) Update Entity: 
Get the entity, update the entity and persist the updates.
try {
// We do lookup for the required id
location = pm.getObjectById(PersonLocation.class, id);
} catch (Exception ex) {
throw new RuntimeException(ex);
} finally {

7) Delete Entity: 
Get the entity and delete it.
try {
// We do lookup for the required id
location = pm.getObjectById(PersonLocation.class, id);
} catch (Exception ex) {
throw new RuntimeException(ex);
} finally {

B) Using JPA

Similarly to using JPA:
1) In persistence.xml
2) Get the EntityManagerFactory:
 private static final EntityManagerFactory emfInstance =
3) Get the EntityManager from the EntityManagerFactory:
EntityManager em = emfInstance.createEntityManager();
4) Annotate your entity:
public class Employee {
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Key key;
    private String firstName;
    private String lastName;}
... setters, getters and constructors
5) Use the entity manager to do the JPA CRUD operations. 

C) Using the low-level Datastore API, or one of the open-source APIs developed specifically for Datastore, such as Objectify. 

This is recommended by Google as JDO and JPA was designed for use with traditional relational databases, and so has no way to explicitly represent some of the aspects of Datastore that make it different from relational databases, such as entity groups and ancestor queries. This can lead to subtle issues that are difficult to understand and fix.

Sample Code:
class Car {
    @Id String vin; // Can be Long, long, or String
    String color;

ofy().save().entity(new Car("123123", "red")).now();
Car c = ofy().load().type(Car.class).id("123123").now();