PUB/SUB Taken To Next Level Enterprise Event Bus (EEB)

Aliases

Publisher-Subscriber, pub-sub, Enterprise Event Bus, EEB, Notification Broker

Context

You have distributed applications (from different vendors running on variety of platforms) , integrated with middleware infrastructure. Applications interact with each other in a traditional/synchronous way by sending messages over the Enterprise Service Bus (ESB)/Broker to get the job done.

Problem

The synchronous infrastructure is causing delays leading to unproductive wait times and dissatisfaction from the users.  There is a need for applications to propagate information (State changes, Interesting events) to other interested applications asynchronously without knowing the details about their identity?

Forces

Following are driving forces behind Enterprise Event Bus (EEB)

• Applications (publishers/subscribers) can be dynamically added/removed from the EEB.
• Applications should be able to publish various Events.
• Applications should be able to consume various Events for which it is interested, Should not receive other events.
• EEB should support the following
  • Store and Forward:
  • Throttling:
  • Translation:
  • Filtering:
  • Push and Pull:

• Various Protocol :
• Governance and Monitoring:

Solution

Enhance the existing middleware infrastructure to allow registering/deregistering publisher/subscriber, devise a mechanism to send the event (notification) to intended listeners with the help of topics. One of the two WS-* specifications (WS-Notification: Spearheaded by IBM, WS-Eventing: Spearheaded by Microsoft) may be considered while implementing the approach.  We will useJBoss Fuse Services works to implement this approach.

Architecture

Design is pretty simple, we have got set of components (services and processes) to provide functionality, exposed over various bindings, data store (Relational and column oriented) to store the configuration and eventing data, and mechanism (In this case Apache Camel) to route/translate events.

Level 0 

Level 1

Level 2

 Here is the ER diagram to start with. With this setup store and forward can be implemented easily.

Why JBoss Fuse Service Works?

• Exposing any end point (SOAP, REST, JMS AMQP, etc.) is just a matter of few clicks.
• Monitoring and Governance support
• Built in support to integrate with BPMN, Drools and Camel.
• Can be deployed to OSGI as well as server
• JEE support
• I wanted to try something different 🙂

Example

A mini EEB using Fuse Service works for demonstration purpose can be cloned from GitHub, instructions to bring it up and running is provided in the README.md file.

Here is the switchyard diagram for the Notification feature. It hardly took me any time to implement this feature, same can be extended further to provide other features easily.

Resulting Context

When considering this approach following benefits and liabilities should be evaluated.

Benefits

• Supports the business demands for better service
• Highly decoupled applications with more flexible and agile structure
• Highly Secured

Liabilities

• Increased maintenance
• Decreased performance

Conclusion

EEB can help enterprise build highly decoupled, flexible, secured and event driven applications.

References

• Enterprise Event Bus with Fuse Service Works
• IBM’s Article on WS-Notification
• Thomas Erl on Notification and Eventing
• Oracle on Event Driven SOA
• JBoss on WS-Notification
• Javaworld’s Article on Event Driven SOA
• Pub-Sub with WSO2
• Event-Driven Architecture: How SOA Enables the Real-Time Enterprise
• Pattern-Oriented Software Architecture: A Pattern Language for Distributed Computing, 4th Volume
• Microsoft’s Integration Pattern Blog
• Pattern-Oriented Software Architecture For Dummies
• Do More with SOA Integration: Best of Packt
• WS-Notification implementation using CXF and ActiveMQ

Light weight, Extensible, loosely Coupled and Modular Application development

Introduction

Software applications are judged by many facets, for example easy maintainability, easily altering software behavior via configuration, speedy incorporation of new feature without affecting existing features , how performant the application is , etc.

If there is a question of building such software, you would get different answer based on whom you approach, Spring Evangelist would say “Spring is the framework for all you need, Dump the JEE standards since JEE Lost the competition and Spring Won” while JEE adherent would respond  saying It’s not Spring anymore, it is the summer of JEE7 since the age of framework is over hence move from spring to JEE

However many a times we neither need a heavy framework nor a heavy application server to create a highly configurable/modular/Testable light weight, loosely coupled Application, and yes it is possible, it is neither a science fiction nor a fairytale.

Just utilizing Core Java Features, Sand-witched in Design Patterns, stuffed with Design principle and sprinkled with Good Programming Idioms.

​All that I am trying to convey is that we have to use right tool at the right place, it is all about knowing when to use a hammer and a screw driver.

Problem Scenario

Let’s say we are going to build an IAM (Identity and Access Management) Application, which is in fact absolutely essential tool for every organization regardless of size, it requires the following (but not limited to) functionality:

• Password Management
• Self Service
• Delegated Admin
• Provisioning
• Audit and Compliance
• Federation and Single Sign on using SAML, OAuth, OpenId etc
• Role Based Access Control (RBAC)
• Social Login
• SOA/Rest API security etc.

Design Elements

We will use the following:

• Interface Segregation Principle : Many specific interfaces are better than a single, general interface
• Open Closed Principle (OCP) : Classes should be open for extension, but closed for modification
• Dependency Inversion Principle: Depend upon abstractions. Do not depend upon concrete classes.
• Single Responsible Principle: Classes should change for only a single reason.
• Inversion of Control : Service Locator pattern using ServiceLoader API
• Simple Factory Programming idiom

To build an application, this would be:

• Modular
• Loosely Coupled
• Highly Configurable
• Highly Testable
• Easily extensible

Here are the main participants, which gives 30,000 feet overview:

• Provider SPI/API : Using these components Services are exposed
• Configuration: This is typically a JSON configuration file, which defines the services consumed by the application.
• Service Loader: This is typically the Application Session Factory, which loads the services based on the config and make it available to the clients via Session Component.

Provider SPI/API

There are three main interfaces, ProviderSpi, ProviderFactory and Provider, Entire application would be central around these interfaces, as you would be seeing, we would be extending and implementing these interfaces to get the job done. Note, you can find the source code here

public interface ProviderSpi {

    String getName();
    Class<? extends Provider> getProviderClass();
    Class<? extends ProviderFactory> getProviderFactoryClass();
}

public interface ProviderFactory<T extends Provider> {

	public T create(AppSession session);
	public void init(Config.Scope config);
	public void close();
	public String getId();
}

public interface Provider {

	void close();
}

Config holds the user configuration, and AppSession provides a way to get any particular instance of Provider to get the job done. Let’s see two more interfaces important in this aspect.

public interface AppSessionFactory {

	AppSession create();
	<T extends Provider> ProviderFactory<T> getProviderFactory(Class<T> clazz);
	<T extends Provider> ProviderFactory<T> getProviderFactory(Class<T> clazz, String id);
	List<ProviderFactory> getProviderFactories(Class<? extends Provider> clazz);
	void close();
}

public interface AppSession {

	AppTransactionManager getTransaction();
	<T extends Provider> T getProvider(Class<T> clazz);
       <T extends Provider> T getProvider(Class<T> clazz, String id);
       <T extends Provider> Set<String> listProviderIds(Class<T> clazz);
       <T extends Provider> Set<T> getAllProviders(Class<T> clazz);
       void enlistForClose(Provider provider);
       AppSessionFactory getAppSessionFactory();
       void close();
}

Here is our provider-api module

Let’s extend provider-api, to do something meaningful, and hence let’s create model-api, to keep things simple, let’s add only one model called User.

Here are the corresponding classes/Interfaces

public class UserSpi implements ProviderSpi {

    public String getName() {
        return "user";
    }

    public Class<? extends Provider> getProviderClass() {
        return UserProvider.class;    
    }

    public Class<? extends ProviderFactory> getProviderFactoryClass() {
        return UserProviderFactory.class;
    }
}

public interface UserProviderFactory extends ProviderFactory<UserProvider> {

}

public interface UserProvider extends Provider {

    UserModel addUser(UserModel user);
    boolean removeUser(UserModel user);
    UserModel getUserByUsername(String username);   
    void close();
}

public interface UserModel {

    String getUsername();
    void setUsername(String username);
    boolean isEnabled();
    void setEnabled(boolean enabled); 
    String getFirstName();
    void setFirstName(String firstName);
    String getLastName();
    void setLastName(String lastName);
    String getEmail();
    void setEmail(String email);  
}

Now this is the Java feature called ServiceLoader API

Here is two of the implementations of model-api, one is model-mem which would store the details in memory, while other one would store in database using jpa called model-jpa, you can have a third implementation called model-mongo, which would store the details in mongodb.

Configuration

The main config file is shown below, based on this configuration things would be driven.

Service Loader Component

Service Loader component is an implementation of AppSessionFactory, the main purpose of this component is to load the available service based on the configuration. For this purpose it uses JDK Service Loader API in combination Config Object. Here is the snippet code, which shows what is going on in the AppSessionFactory. The important piece is in the init method.

public class DefaultAppSessionFactory implements AppSessionFactory {

	
	private Map<Class<? extends Provider>, String> provider = new HashMap<Class<? extends Provider>, String>();
	private Map<Class<? extends Provider>, Map<String, ProviderFactory>> factoriesMap = new HashMap<Class<? extends Provider>, Map<String, ProviderFactory>>();

	public void init() {
		for (ProviderSpi spi : ServiceLoader.load(ProviderSpi.class)) {
			Map<String, ProviderFactory> factories = new HashMap<String, ProviderFactory>();
			factoriesMap.put(spi.getProviderClass(), factories);

			String provider = Config.getProvider(spi.getName());
			if (provider != null) {
				loadConfiguredProvider(spi, provider, factories);
			} else {
				loadUnconfiguredProvider(spi, factories);
			}
		}
	}

	private void loadUnconfiguredProvider(ProviderSpi spi, Map<String, ProviderFactory> factories) {
		for (ProviderFactory factory : ServiceLoader.load(spi.getProviderFactoryClass())) {
			Config.Scope scope = Config.scope(spi.getName(), factory.getId());
			factory.init(scope);

			factories.put(factory.getId(), factory);
		}

		if (factories.size() == 1) {
			String provider = factories.values().iterator().next().getId();
			this.provider.put(spi.getProviderClass(), provider);

		} 
	}

	private void loadConfiguredProvider(ProviderSpi spi, String provider, Map<String, ProviderFactory> factories) {
		this.provider.put(spi.getProviderClass(), provider);

		ProviderFactory factory = loadProviderFactory(spi, provider);
		Config.Scope scope = Config.scope(spi.getName(), provider);
		factory.init(scope);

		factories.put(factory.getId(), factory);
	}
………
}

Demo

As can be seen here in our main class, we are doing two things, first inserting user and then scheduling some tasks, depending upon the Config, either it would store in memory or in database.

public class Main {

	public static void main(String[] args) {
		ConsoleApplication application = new ConsoleApplication();
		application.insertUser();
		application.setupScheduledTasks();
	}
}

This time user provider is ‘mem’

After changing the user provider to ‘jpa’

Conclusion

What I have demonstrated is the very basic infrastructure which can be extended further to take it to different levels, and can be used in any layer, whether it is the Web, Middle or data access layer. In fact what I have demonstrated is the design in Keycloak Application (Identity and Access Management Application from Jboss), Here are some of the modules

With this approach we end up creating lots of interfaces and hence different components depend upon abstractions instead of concrete classes, your system would become open of extensions and closed for modifications, since any specific implementation can be worked on separately and included in the system without affecting existing system plus your system would become highly modular (since jar is the unit of modularity)

References

• Keycloak Website
• Source code of this blog

Blazingy Fast JavaScript Development

I have been trying to keep myself out of UI Development for years until recently after the emergence of Typescript. Developing UI stuffs in Typescript is such a breeze that, there is nothing like unpredictable, I no longer hit my keyboard too hard, my colleagues are no more afraid coming to my desk while I do UI development and most importantly I spend more time with my family than doing late night outs in the office trying to uncover a unknown mysterious secret. 🙂

Things were never been such fascinating earlier, Kudos to Microsoft for understanding the developer frustrations and filling the long standing void in JavaScript.

Why Typescript

Here are couple of reasons that looks appealing to me.

Object Oriented support

One of the main goals of Typescript is to make it as compatible as possible to ECMAScript6, and hence it supports most (if not all) of the Object oriented concepts/Constructs, specially Classes, Interfaces, Modules, Generics, Static Typing  etc. The following code snippet demonstrate most of the concepts. It looks more natural and closer to what an Object Oriented developer could imagine. This is the good point about typescript, it borrows most of the concepts and syntax from existing languages and hence the learning curve would be closer to nil.

Here is the equivalent generated JavaScript, there are so many commotions that an Object oriented developer(For example a Java Developer) would be uncomfortable with

IDE support

Most of the IDEs (Eclipse, Webstorm, Visual Studio) have support for Typescript, either built in (Visual Studio) or as plugin. Which makes the life easier since IDE would act as enforcer and guide developers in need.

Build tool support

All major JavaScript build tools Grunt , Gulp and Brunch have typescript support, In fact the example source code demonstrate how we can marry Grunt and Typescript together very easily.

Syntactic sugars

Most prominent is the Arrow function or Lambda expressions. Every JavaScript developer would have experienced losing the this object while running functions asynchronously, specially in callbacks. Suddenly you find your this object to be changed by something unbelievable, and you keep nodding your head whole night in front of your monitor.  Simple work around is to preserve the this object in some other variable, for example:

Equivalent typescript code would be

External Library Integration

In the above example, you must be wondering, where is CanvasRenderingContext2D defined ? This is where type definition(.d.ts) files comes into picture, Typescript has definition for every core JavaScript feature/functionality.

A million dollar Question: What about the external Libraries (JQuery, Backbone, Jasmine, AngularJs etc)? ,  It can be defined based on the functionality it provides and that is why Definitely Typed repository exists. If you explore this repository you will find the d.ts for almost every thing you can think of. For some reason if it does not exists, it can be created easily.

Typescript Alternatives

There is no shortage of languages that compile to JavaScript. CoffeeScript is perhaps the best known example of one of these languages. Typescript is just one of the many languages that compile to JS, some of the popular among them are:

• Google’s Traceur : Compiles EC6 (Syntactic niceties for example Class, Module, Arrow Functions, Promises, Default Parameters etc.) to EC5 JavaScript,
• Google’s Dart :  Completely new language to address  JavaScript issues, cleans up its semantics, adds static types and class based and object oriented feature with out being dogmatic.
• Coffescript : Neater syntax which is based on other dynamic languages, there by makes it easier to write.

Applications Using Typescript

• Office365 and Visual studio online heavily uses Typescript
• Entire hawtl.io UI is written using Typescript

Sample Application

This Sample Typescript application is loaded with lots of features, It basically demonstrate Typescript feature usage in collaboration with existing libraries and tools.

• Generics
• Module, Classes and Interfaces
• Unit Testing With Mocking and Spying Using Jasmine
• Grunt Usage
• Project directory structure organization

Feel free to clone it and play with it, instructions are provided in the Github repository.

Conclusion

We have been witnessing proliferation of single page applications that are built using JavaScript, It would be a tedious task to maintain/Write these applications with just pure JavaScript. Languages which compiles to JavaScript (Typescript for example) and provides good syntactic sugar, structures and constructs becoming immensely helpful.

References

• Expert Javascript
• Pro Typescript
• Typescript Essentials
• Typescript Revealed
• Mastering Javascript Design Patterns
• Jasmine Javascript Testing
• Source code of this blog