The Power Behind the JMS

In every discussion about middleware, the idea of a ESB tool, Web Services and EJBs are always in focus. But we can't forget that the JMS can bring up a reliable and loosely coupled way to have an asynchronous communication.

Of course, every scenario demands different combination of components of software to be used. And the success formula is directly related on the way you design your final architecture.

In 2007 I was called to solve some bad issues one big retail company was having in their environment. Basically, they had one of the biggest consulting companies developing their solutions in a JEE environment, with lots of integrations. But some decisions they made on the design phase was causing lots of troubles, and giving a huge headache for the retailer's IT Manager.

Besides some basic issues they couldn't resolve, like adjusting the heap size for the application server, the major problems weren't that easy to address. That's delicate. You have to tell they spent money in a not well designed architecture, and that they will fail with that is not a straight task. So, I decided to have a different approach, showing how to fix the current scenario, but also recommending changes to support better performance in the future. And one of the points was related to the JMS.

The design for that project was using JMS to exchange data between one ERP and the developed java system responsible to populate other systems. That was the biggest bottleneck. The performance was critical, and it was failing on delivering some of the messages.

So, first thing to analyze was the amount of data transmitted in a daily basis. More than 40GB. Oh, looks bad... So, I picked up the biggest file. Around 10MB. Basically, if you work with persisted messages, the data segments within JMS have a predefined size, and a predefined amount of files that can or cannot rotate when they're full. These values can be changes, of course. But they left the default values there. So, when a 10MB message was about to be stored in the data segments within the JMS, with less then 10% of it's size, it was failing.

The final document presented the two scenarios. For the first one, "to make it work as it is", the recommendation was to set a huge data segment size. And the second one, was to make the JEE application responsible for the integration to act like a FTP Hub, being fed with JMS notifications:

• For the messages with less then certain amount of data, lets say 100K, the message would contain the real data, and the process would be the same;
• Bigger that that, it would make use of FTP to get the data and send it to the target system, database or file system.

Some other recommendations over the use of JMS were included on the final document, like making concurrent processes instead of having just one process running. But one of the issues they were having was related to the target JMS availability. In fact, they were thinking in the opposite way. The application that is about to send the messages was running in a application server. So, it should send to a JMS Queue within the same application server. So, if the target consumer is not available, you won't stop. In fact, the consumer is responsible to go to the source and get the messages. With this scenario, you won't have any trouble. It works perfectly!

I also created a PoC to show them what we was talking about, and how to implement it. When I arrived there, JMS was being questioned, and they were also considering to substitute it by web services, because it was failing. With that kind of architecture, Web Services would become the "big bad wolf" sooner or later. Anyways, when I finished my assignment, JMS was in a good picture again. 

In summary, JMS is a really powerful tool. But you have to apply it with some premisses:

• Is the communication asynchronous?
• What kind of information is being loaded into the messages?
• Is it confidential information? Not a good idea to use it on queues!
• What is the amount of data, per message? Focus on the biggest messages;
• And the final question is: do you really need JMS? I mean, I saw one design where JMS was supposed to be used as a logger, for one standalone application. The responsible for that wasn't unable to answer this question with any good and reasonable arguments.

During the architecture design, we should try to come up with discussions related to the peaks on daily/monthly processing, and dependencies. You might need to consider these information on your final design, to further concurrency setup and strategy to adopt for the distributed transactions.

JMS can be a really nice addition to your projects, but the project should require it, and not the opposite.

A bit of AWS

I was always reading about Amazon Web Services, liked the idea of having remote services to build a complete cloud solution, but left it in a side note to try it later.

Well, today I decided to try it. Created my AWS free account, and got really impressed with the amount of features and components available to create the solutions!

My intention here today is not to explain what is AWS, since there are lot of web sites, including the AWS one, with tutorials and videos explaining it. I'd like to show what I've done, leave a general picture on what I did, how I did, and the hidden tricks on the configurations.

I decided to focus on the main features, and from that I'm talking about a Linux instance on the cloud, a relational database, an application server and a network configuration to make it accessible from my local environment.

I tried two different approaches, and so I'll explain these two scenarios, just to show the freedom you have to architect and specify your projects on the cloud using AWS.


Scenario 1

Within this scenario, I'll show how to create and use an EC2 instance. In other words, I'll pick up a Ubuntu instance on the cloud. There are also other Linux options, and even Windows. The setup is straight, not a big deal. But let's keep it simple.

Go to the Management Console, and click on top of EC2. From there, hit on Launch Instance button.

While creating the instance, you might have created a Key Pair. Don't loose your local key, and set the permission for just you to read:

chmod 400 <your key name>.pem

Go to the Network Interfaces, and find your Public DNS.

To access the Ubuntu via SSH, you must use the ubuntu user, your local key pair, and the public DNS:

ssh -i <your key name>.pem ubuntu@<your public DNS>

Done! Now you're accessing your remote ubuntu instance!

From there, I installed tomcat7:

sudo apt-get update
sudo apt-get install tomcat7

Just to make sure it's running:

ps -ef | grep tomcat

Now you have to open port 8080, make it available from outside world:

• First you have to go to Security Groups, and on Inbound tab, add the port 8080;
• Apply changes and go back to Instances;
• Right-click your instance and click Connect - get the IP for the Public DNS;
• Open another tab on your browser and look for the port 8080 on your Public DNS.

If you get the message It Works! from tomcat, you're good to go! You now can access your linux instance remotely, and access the port 8080 without any problem.

I'm not going far on this scenario. I just wanted to show how to create the OS instance, access it from your local machine. I just set up the tomcat to show that is necessary to configure the authorization to access the specified tomcat port externally.

Even though it's simple, from there you can expand your environment as you need and want. For example, having the basic environment created, you can just install a database and a version control component (SVN, GIT) on your Ubuntu, add Jenkins, update the configurations and quickly you can have an integrated development environment.

Replicate the environment, and you can create the Q&A and Production environments. Implement promotion processes and have a full environment ready.


Scenario 2

For this scenario, I decided to pick up different components available on AWS. The database and the application server. Note that you won't need the EC2 instance for this scenario.


Application Server - Elastic Beanstalk

Go to Elastic Beanstalk and follow the steps to create a new Application Server Instance. Easily you have a new App Server running and accessible! I selected Tomcat 7.

Comparing to having it tied to the OS, it's extremely interesting the option to have it independent from the EC2 Ubuntu instance. And working together with the Eclipse Plugin, you can directly deploy to it.


Database - RDS

Go to RDS and click on Launch a DB instance button.

With several options between MySQL, Oracle and SQL Server, I picked up MySQL - it's just for a test. Basically you can access it from your local machine using a SSH connection. It's pretty easy to use from MySQL Workbench. If you have a MySQL instance running in your box, it's easier to turn it off. Within a linux, just type a command like:

mysqladmin -u <root> -p<password> shutdown

To configure the SSH tunnel on MySQL Workbench, click on New Server Instance:

• The Address you can get from your BD Instance. When you click on the details, check the address on Endpoint;
• Follow the next steps, and done! You have now access to your database on the cloud.

Don't forget to download the Connector/J from MySQL website. This is the official JDBC connector for MySQL.


Create a sample table

From MySQL Workbench, create a simple table to be used as an example:

create table grosseries (
  code    int,
  name    varchar2(50),
  date    datetime
);

Test the connection from your local Eclipse

Create a Java Project, a class called Connect, and use a code like the following one:

Just heads up on the JDBC URL:

• To get the server name, go to your instance and click on top of the row. The details will be presented. Get the name from the EndPoint field;
• The database name is the same you created during the installation.

Now it's just a matter to run it and check if you got any results on your console.


Install the Eclipse Plug-in

In order to create a Eclipse project and deploy on the Elastic Beanstalk, it is necessary to install the AWS Plugin for Eclipse:

• On Eclipse, select the option Install New Software under the menu option Help;
• Type http://aws.amazon.com/eclipse on Work With field and click on Add button;
• Type a name for the installation - I used AWS Plugin and hit the Ok button;
• Select the top option and follow the next steps until the plugin is installed;
• Restart Eclipse and configure the plugin:
• Open the AWS Management Perspective;
• On EC2 Instances tab, hit on the link to login with your account;
• To create/view your Access Key, and get your Account Name and Secret Access Key, go to: https://portal.aws.amazon.com/gp/aws/securityCredentials
• Type your information, and connect - you must see your EC2 Instance listed below.

Test the Deployment to the Elastic Beanstalk

I'll not list all steps to create a java project here. Feel free to create anything web based, so you can access from a web browser/

The idea here is to create a local project on your Eclipse and then deploy to AWS Elastic Beanstalk. In order to accomplish that, the following rules have to be considered:

• Create a AWS Java Web Project on Eclipse;
• Use a local Application Server for local tests;
• When the application is ready to go to AWS, follow the steps below:
• On the Project Explorer, right-click your project and select the option Deploy to AWS Elastic Beanstalk, under the Amazon Web Services option;
• Select the correct server and version and hit Next;
• Type the application name and environment name and hit Next;
• Set the option Deploy with a key pair;
• Hit Finish - it took me some time to deploy my application.

Now go to your Elastic Beanstalk on AWS and check the results.

Note that you can also just create manually an application on AWS Elastic Beanstalk and upload the war file.


Conclusions

I made it simple. But the capabilities are impressive! You can create a high available environment with load balancing, setup regular backups, build huge solutions and count with more resources as you need them, monitor... Anyways, I can create a enormous list of features and benefits, but the message will be just one: you can find anything you need to build a consistent solution on the cloud with AWS. Of course, there's a price to be paid in the end. So, my 50 cents, start with the minimum, and grow as you need.