Using workflow management system for some sysadmin tasks

Processes in large environments

In large production environments practically everything must be controlled by oracularly defined processes. These computing environments are too complex to know by some system admins or even group of them. This complexity requires dividing operation into separate group of tasks carried out by different groups of specialists.
Orchestration of operation is achieved by processes defined on the basis of recommendations and rules formulated in ITIL V3. These processes are described in series of documents which must be well-known by groups concerned.
This document describes a possible solution which can help employees to keep up with these processes, considerably speed up some of them, minimize human errors and avoid situations when processes disintegrate due to changes in human resources.

Implementation of processes

Processes defined only in written form have got a serious drawback: Each worker involved into the process must know in details at least part of the documented process which must be followed in his job. Moreover, he should know the neighboring and connecting processes too in order to be really effective and to be capable to make decisions in complex situations. This can lead to temporary disintegration of processes and/or considerable slowdown of them in case of personal changes in groups.
To avoid such situation each process should be technically implemented in some process management system which could drive and document the process and provide actors of the process with supporting information.

Processes not controlled by standard tools

There are many situations in everyday operation when some process should be followed but the given process is not well-defined. Even more, many of those processes are out of scope of main tools used for operation processes (eg. ticketing system, CMS, etc.) therefore there is no tool for driving and documenting them. These processes usually based on some information exchange by mail, personal connections between workers and working experience of some senior colleagues.
These processes are usually long-lasting, uncertain and frequently lead to confusing outcome. We can lose considerable time and resources till the point where the process comes to a well described state and under control of one of standard tools.

Task automation during processes

Most of processes in IT operation involve human interaction with systems operated. Many of those processes are very typical and practically follow a simple well-defined algorithm. Automation of those steps is beneficial from the point of view of execution time and avoiding human errors.
There are two classes of such process steps:

  •  Process steps not leading to any considerable change of the target system.
    Such steps are usually information gathering ones about the target system, or some minor changes not influencing the service provided for the customer of the system.
  •  Process steps directly influencing the behavior of the system and the service provided.

First class of process steps can be automated easily. The second class requires more attention and a specialist must check the process step before it is executed on the target system.

Integration with the existing environment

In a large IT environment it is crucial how different elements of that system are integrated. However, in such environment entities of the system shouldn’t be directly interconnected. Direct interfacing system entities leads to an unmanageable web of interconnections and makes any change to the system extremely complex.
The key to manageable system integration is the data held by separate system entities describing the whole system and processes in it. These data and system entities holding that will form the environment in which all other system entities must exist actively. Usually all core systems are able to serve their data in well-known, standard form. They are also able to receive data from other systems in standard form.

Each new system entity must be able to acquire data actively from this environment and use that data for its internal purposes. Results of processes in those system entities must be presented (or sent if the other system is not prepared to actively get the data) in well-known standard forms for other systems.

The Bonita workflow management system

The goal formulated above can be achieved using the system named “Bonita Open Solution” (http://www.bonitasoft.com/)
Bonita is a workflow management system with a lot of features, covering the whole cycle of planning, developing, deploying and using implemented business processes. Two main software packages provided by Bonita are the development environment called “Bonita Studio” and the Bonita server bundle. The latter provided in two variants: one based on Tomcat servlet container and the other on JBoss. Both of them are available for Linux and Windows servers too.

The Bonita development environment

Implementation of a business process begins with its planning, design and development. Bonita Studio is an integrated development environment based on Eclipse. It supports all steps of process design up to exporting the result to a package installable on a Bonita Server or on a stand-alone Tomcat or JBoss server.

Bonita has a lot of built-in features and also it is highly extensible through its powerful APIs. Available APIs include Java-based API, EJB2, EJB3 and REST for custom application development and easy embedding.

Logical planning of the process

Process design in Bonita begins with modeling. Process modeling in Bonita is done in a very intuitive way, in fact with sketching up steps of the process and transitions between them. It is possible to design processes with several conditional paths of steps, generate and capture events in some conditions, initiate sub-processes or parallel ones and synchronize execution of them.

Connectors in Bonita

Maybe the most powerful feature in Bonita is the set of built-in connectors. These connectors provide access to external systems, data sources and programming environments.

These connectors make it possible to easily integrate processes developed in Bonita with other system entities and data sources forming the environment in which the implemented workflow will run.

Actor selectors in Bonita

Each step of a process and the process itself requires definition of person(s) who is eligible and authorized to carry out steps of the process. There is a special type of connectors in Bonita called Actor Selectors which are used to select those users authorized to carry out process steps. Each step can have its own set of actors.

One can create a particular Actor Selector(s) for the process designed by specifying correct parameters for these templates statically or using Groovy script.

Connectors and Actor Selectors can be extended and/or new, custom ones can be created using the development environment of Bonita Studio.

Data and programming in Bonita

Each business process is about handling some kind of data. Bonita is able to handle all major types of data and data structures. The programming language provided and recommended for data handling in Bonita is the Groovy script.
Scripts are able to work with data defined on different levels: process level, process step level, transient data for a step, etc. Data storage is persistent in its scope. Therefore each script defined in that scope can use that data. Scripts are executed at specific events of the process (entering into a step, end of a step, etc.) and in order of listing them at the particular event.

The user interface

The outcome of the design process in Bonita Studio is a web application. It is clear that such an application has to have a user interface to interact with. There are several types of web forms that can be created Bonita. Two main types of those forms are data entry forms and view forms.
First type is the user interface one will use during work in the case, created using pre-designed process. The second one used for displaying the status of the case and as an introduction at the very beginning of it. View forms do not allow data input and control of the case.

The test case

Monitoring team frequently receives questions among others about status of the monitoring on some servers. Such information can be important for application administrators, OS administrators and others mainly when something has changed in their environment.
To answer such questions a monitoring specialist has to log on to monitoring servers and sometimes also to the monitored node itself, run some commands manually, copy output of those commands into a mail message and send it back to the requestor.
This process from the very beginning – sending request from application administrator to monitoring team – till its end – receiving answers from monitoring team – can take hours or even days if the monitoring team is busy with higher priority tasks.
On the other hand, these questions are similar from procedural point of view and answering them does not require any change on mentioned systems. It is clear that this particular process is a good candidate for implementation in Bonita.

First steps of process implementation

As for each process, implementation of this one also begins with very simple modeling of it. At this stage there is no need to think about data, scripts or whatever technical details. That would only distort the logic and flow of the business process itself.
In Bonita studio the development process begins with sketching up the logical scheme of the process: steps of it and transitions between them.

Steps and transitions

Main building blocks of a process are steps and transitions. Bonita is able to handle several kinds of process steps: service task, human activity and call activity. In this example each step is a human activity. Each activity contains entry form for user interaction, connectors and scripts for data gathering and handling and actor selector.
Transitions between steps are not unique as it is usually happens in real life. Therefore, each transition has a condition or a decision table (more than one condition coupled some way) in order to decide which way to go further.
The process has very simple forks: If the user wants to save results displayed on forms, an additional step is included into the process. (Save result x)

Running commands on remote systems – the Script Connector

In order to automate processes we must be able to run commands and scripts on remote systems. In fact, Bonita provides a built-in connector for running commands and scripts on the host operation system where the Bonita server runs. Thanks to extensibility of Bonita it is possible to create customized connector for specific needs. However, in this example the Script Connector provided by Bonita was used. The connector uses local shell environment for running commands.

In order to avoid interactive password authentication on the remote server the public key based authentication provided by SSH system was used.

In our particular case the Bonita server runs on Linux and the target system where we want to run our commands is a HP Unix. So it is straightforward that the well-known SSH is the optimal choice.

The command for the Script Connector:
ssh user@<IP address> /some/command ${node_Gorup_Selected}
Notice the part of the command typed in red. That is a variable evaluated in the previous step of the process.

Handling output of connectors

The connector captures of the output of the command executed. The result can be placed into a variable previously declared in the scope of the process step or above:

The result from our command is a multi-line text string. Below is a design screen showing the design of an entry form and the command preparing the output of the script for using in a drop-down list on the data entry form presented to the end-user.

In more complex cases a Groovy script connector can be used for preparing data. This Groovy code prepares the script output for displaying in a table on the user interface. The input of that script is a multi-line string produced by a remote command. Each line contains equal number of strings separated by “|” character. The script colorizes the output depending of the content of the column #6:

result_Table = []
def result_Line = []
if (readLines(script_Output)[0] ==~ /^NODE:.*/){
for (def i = 0; i < (readLines(script_Output).size()); i++){
result_Line[i] = tokenize(readLines(script_Output)[i])[1]
}
if (result_Line[6] == "CONTROLLED") {
result_Line[0] = "<font color=\"#006400\"><b>" + result_Line[0] + "</b>"
}
else {
result_Line[0] = "<font color=\"#8B0000\"><b>" + result_Line[0] + "</b>"
}
result_Table[0] = result_Line
}
else{
for (def i = 1; i < (readLines(script_Output).size()); i++) {
result_Line = []
result_Line = tokenize(readLines(script_Output)[i],"|")
if (result_Line[6] == "CONTROLLED") {
result_Line[0] = "<font color=\"#006400\"><b>" + result_Line[0] + "</b>"
}
else {
result_Line[0] = "" + result_Line[0] + ""
}
result_Table[i - 1] = result_Line
}
}

Decision table for transitions

Processes in real life always has got different paths to follow depending on results of its steps or in case of errors made in some of them. Bonita makes it possible to define multiple transitions between process steps. Naturally, not all transitions started from a step may be suitable at a time. Usually a single one should be active except when the process is forked into several sub-processes. To achieve that each transition has a condition at which it activates. In case of dependency of the transition on several conditions a decision table can be composed using graphical interface in Bonita Studio:

Actor selection

Bonita Actor Selectors can define people or groups allowed to open and work with process steps.

There are several Actor Selector templates in Bonita. Current Actor Selectors composed from those templates by specifying correct parameters for them.

Bonita has its own user database. It is not mandatory to use that user repository however, that repository is built-in a way that it can describe in details groups, their leaders, substitution rules, positions in regard to responsibilities. In enterprise versions of the Bonita Open Solution a module for synchronizing Bonita’s user DB with external Sources like LDAP is available.

In the current implementation of the test process an external source, the LDAP server similar to one deployed in my company is used.

The resulting application

The process implemented automates information gathering about status of monitoring on the server selected during the process. The process can be deployed in form of a web application to the Bonita server and can be used by application and server administrators without involving the monitoring team. This way considerable amount of working time can be saved on both side and quality of the operation improves thanks to the easy and fast process of the information delivery.

Conclusions

The workflow management system Bonita is a great tool for implementation of administrative and automated operation-like processes. It can help to solve problems:

  • Typical technical processes requiring human interaction this time. (Information gathering about status of the monitoring, everyday “morning check” of systems, etc.)
  • Information exchange processes preceding creation of a change process
  • Requests of employees to HR
  • Decision preparation, change negotiation, information gathering processes on different organizational unit levels

Thanks to its features Bonita can be easily integrated into the company’s environment. It can be used at first time without requiring big resources from finances or from workforce.

Based on conversation with application administrators, engineering fellows and others a lot of fields of application are seen already. Each of them could save considerable working time and improve quality of operation.

Advertisements

Author: bithajcsar

I am a system architect with decades of experience behind. During that time I learned that only the continuous development of knowledge and skills can keep a specialist on the cutting edge of his profession. And only those will move the World forward! Another thing I noticed that the stupidly used ICT (Information and Communication Technology) moved people far from each-other, when the goal of the whole thing should be the opposite. A modern computer may provide too many "instant experience and happiness" and chain users - mainly kids - to their monitors. Just a step away from the Matrix! After all those thoughts I decided to share my knowledge and experience in a form understandable for anyone who wants to know more about devices and systems we use every day. My primary targets are kids. They are curious and they can learn so fast that it sometimes seems unbelievable for us. Maybe they will just know a bit more about ICT and will use that in a more clever way. But who knows? Maybe they will go further and later will move the World forward. I hope that I will be able to help them to find a right way and go much further we could imagine now.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s