Oracle Acquires Sun

I cannot stop myself digressing from our core topic of integration with news "Oracle acquires Sun".

Subject to necessary approvals itseems that one more company had been added to Larry's shopping list. I had been guessing this for last 4 years. As I come from DBA background on Solaris platform, I know how much is the love between two technologies and its followers.

From technical perspective its perfect marriage. Some how I am not convinced with IBM acquiring Sun. I had seen how Rational acquisition worked earlier with IBM.

With Oracle's expertise in making acquistions into successfull ventures we should not have any doubt on the success of current acquisition.

We had great upside where Oracle would continue to invest in Java community, and it would finally have edge over IBM.
Think Java, Think Oracle.
On the downside we need to clearly see Oracle's strategy on mySql database which been default database for many startups.

Batch Process & SOA

Its been a while that I had written in my blog. Its more a factor of my time consumed in my deliveries to my clients than any thing else. Off late there was a situation where I had to study application and data integration aspects of operational data store system for one of my clients.

I learned that teams working in the project always thought about real time synchronization of data and application, when there are situations where we can go for batch processing. Some how batch process seems to have lost its charm. It seems to be attached with legacy systems.

I feel batch processes remains vital to today’s real-time enterprises.

Behind the real time systems that power the real time enterprises, such as Customer Order Fulfillment, Account Management, Supply Chain Scheduling and Optimization, Financial Trading Systems are regularly-updated back office business systems.

Today, batch processes remain essential for one key reason:

It is simply not efficient to regenerate a complete forecast or business plan every time the business processes creates a single event such as an incoming customer order.Real time enterprises do require systems that can support dynamic processes; however, it is best to reserve that capacity for aspects of data or processes for the most volatile high-velocity markets.

While the need for batch processes hasn’t changed, the nature of batch processing today has certainly evolved. For instance, scheduled batch processes remain relevant for time-related processes such as end-of-period reporting. Where as real time enterprises may require more flexibility for adapting to temporary or permanent market fluctuations, or to planned or unplanned changes in underlying IT infrastructure.

Companies are faced with new requirements for managing compliance with increasingly stringent regulatory mandates. This may dictate the need for policy-driven workflows with the capability for dynamically triggering batch processes as and when specific scenarios arise. For instance, a Sarbanes Oxley violation might trigger a rollback and new round of updating core financial systems outside of the normal schedule.

SOA presents enterprises with the opportunity to expose information and processes as Self-Contained Services that can communicate and interoperate with each other in a standard, loosely coupled fashion. Although the common impression is that services expose business processes, data processes, or application functionality, they are also well-suited for exposing the very processes that drive batch-oriented workloads. SOA enables the business to build flexible compositions of services that implement either business or IT processes in a loosely coupled manner, which has important ramifications for IT service delivery, and the batch processes that are part of it.

If we retrospect batch technologies; they had evolved from script-based automation to rules or policy-driven workload automation. IT has long sought to automate the running of batch jobs, initially with tools that automatically chained them together in a schedule. As batch windows shrunk, IT organizations embraced job management technologies enabling them to optimize their use of limited resources over shorter timeframes. Workload automation is the culmination of innovations in job automation, adapting it to the needs of the real time enterprise with policy-driven, configurable workflows. Compared to traditional job automation methods, configurable workflows are far more flexible and maintainable.

Yet, the objective has changed.

Instead of optimizing time windows or resource consumption in the data center, today the mission of batch processing has broadened its support of the business. In the real time enterprise, today’s batch operation is morphing from a static, often standalone process to a dynamic component of an application or composite business process. The goals could include accelerating responsiveness, improving corporate transparency, and enabling IT to adopt a more business-centric and compliance-driven approach to managing its own operations.

The key to achieving these goals is consistency and agility.

With traditional, hard-coded or manual approaches, ensuring uniformity of batch processes was often a hit or miss proposition. Batch, and its successor workload automation, has evolved over time, with the latest innovations adding policy or rules-driven workflows that transform batch jobs into repeatable work flows.

Ultimately, SOA provides the architectural underpinnings that bring this vision to fruition: By invoking batch processes as a service, they become ingrained components of the composable business processes that power the real time enterprise. For instance, via Service enablement, workload automation can become extensions of applications ranging from supply chain management to IT service desk.

Embedding workload automation as a service could also encourage cultural changes that bring together IT operations, software development, and the business. Typically, batch jobs are either scheduled or “thrown over the wall” from the business to operations. When batch workloads or workflows exposed as services become extensions of the application, developers who design or configure the application can now factor in the demand on IT infrastructure by specifying the logic or rules under which enterprise systems can request batch Services. Workload automation is the key to making the batch window a first class citizen of the real time enterprise, which dictates expanding the role of workflow automation from an IT to a business-driven tool.

Consequently, the goal of workload automation changes as well. Instead of IT optimizing its own data center update workflows, applications or business processes are now in the driver’s seat, triggering workloads when specific business events or scenarios occur. This requires a business process-focused workflow engine for job automation, and a business rules engine for enabling policy-based workflow management. In a business-driven workflow, requests from applications, database stored procedures, or business processes trigger workloads (which may consist of one or more batch jobs) as a result of explicit business rules or policies.

Under these scenarios, workloads no longer strictly focus on optimizing database updates in the data center. Instead, they have become intrinsic components of demand-driven supply chain business processes. Batch processes would be exposed as services that you might want to access in real time. The idea of a batch is that the process is pre-defined (so, a batch is not a flexible process), but it would be exposed as a service that might be consumed in a flexible process. In traditional workloads, a batch might be scheduled as part of a daily activity, but a Service-oriented Batch could be executed On Demand. From a SOA perspective, the service contract and policy would have to specify how many times a batch can be executed in a single day, how long the batch takes to operate, and any other service or data dependencies.

Although typically associated with functionality that is exposed from software applications and/or composed business processes, SOA can also expose IT infrastructure processes such as workload automation as Services. Using SOA, business events trigger specific job types. In effect, the job becomes part of a composite, Service-Oriented Business Application. The SOA infrastructure coupled with the standards used in implementing SOA facilitate a true loose coupling where business logic, business, rules, and job specifications can each change without impacting the other components. For instance, the workload automation system could invoke a batch update service as a result of BI system triggering a scheduled Extract-Transform-Load (ETL) operation to a data warehouse. If the user changes their ETL system itself or some of the transformation routines, that change is kept internal to the ETL Service and should not impact the Workload Service.

In today’s ascendance of the real time enterprise has not eliminated the need for effective batch processing. By leveraging Service-Oriented Architecture (SOA), enterprises can evolve batch workloads from standalone data center operations to components that are intrinsic to composite, Service-Oriented Business Applications. Where workload automation and batch processing were once arcane tasks relegated deep in the organization, they become critical Services delivered as part of the Enterprise Architecture (EA).