Saturday, May 23, 2015

Migrating process instances in IBM Business Process Manager V8

Introduction
Over time processes need to be refined, modified and changed, which leads to the need to deploy new versions of the process applications that encapsulate the new business processes. This presents challenges for businesses in dealing with long-running processes and deciding how to handle processes that are using the previous process application version. IBM Business Process Manager V8 (IBM BPM) provides three options for managing the migration of running process instances:
  • The Leave option means that the version is not updated and the process will run until it completes as determined by the originally deployed process.
  • The Migrate in-flight instances option migrates currently running instances to the version of the selected snapshot. Wherever the running instances are in the flow of the process or service, the new version is implemented for the next item or step. This function allows processes to be modified "in-flight" and to adapt to process changes during runtime. Process owners can open and edit in-flight processes as they are running, change process data, dynamically activate and cancel tasks, trigger escalations, and trigger events.
  • The Delete option, available only on development servers, removes all instances of the old process without completing them.
This article focuses on the migrate option and provides guidance on managing the migration, as well as insight into the rationale for choosing the migration option.
Migrating process instances scenario
Migrating a process is the most complicated of the three options because it requires an understanding of the impact of the process change. This article provides a step-by-step example of a migration process, and describes what happens to running instances when the process flow has changed and left dangling or orphaned navigation steps. Figure 1 illustrates the concept of migration and shows at a high level what we are trying to achieve in choosing the migrate option for upgrading the process.

Figure 1. Migrating a process instance to new version
Migrating process a                    instance to new version 
The versioning of processes in IBM BPM V8 is governed by the following characteristics:
  • Native version control of the process
  • Snapshots to allow you to go back in time to view, deploy and run the previous version of the process application
  • Migration of processes between instances
  • In-flight modification to adopt changes
Migrating in-flight process instances involves updating running processes from the original process to the new version of the process so that they complete as if they were started as the new version, The migration complexity occurs when the new version modifies process flows and activities in the steps that the running instance has not yet completed.
The business scenario
The migration example we'll use in this article leverages the Standard HR Open New Position process from the quick-start tutorial included with the IBM BPM V8 installation. The Hiring Sample process application contains a single business process definition called Standard HR Open New Position. The Open New Position process captures the steps and decision points for submitting and processing a request to fill a position. Figure 2 shows the business process diagram:

Figure 2. HR Open New Position Process
HR Open New Position                     Process 
HR open new position process activities summary
  1. A hiring manager submits a requisition either to fill a new position or to replace a departing employee in an existing position.
  2. The hiring manager determines the position requirements and prepares a requisition for submission to the Human Resources (HR) department.
  3. If the request is to fill an existing position, the requisition is routed directly from the hiring manager to HR, which can then search for job candidates.
  4. If the request is to fill a new position, the requisition is routed to the General Manager (GM) for approval. The GM evaluates the requisition and either approves it or rejects it.
  5. If the GM approves the requisition, it is routed to HR, which can then search for job candidates. If the GM rejects the requisition, the hiring manager is notified and the requisition is terminated.
Preparing the example application
In order to prepare the example application to demonstrate the migration of running process instances, you need to first ensure that the business process definition (BPD) is exposed. You need to expose the BPD to particular participant groups to establish who can:
  • Start instances of the process in Process Portal.
  • View data for instances of the process in reports in Process Portal.
To define this, complete the following steps:
  1. In the IBM Process Center console, click the Process Apps tab.
  2. Select Open in Designer.
  3. In the Designer library, click Processes to view a list of available process definitions.
  4. From the list, double-click the Standard HR Open New Position BPD to display the process diagram.
  5. In the IBM Process Designer view, open the BPD that you want to expose.
  6. Click the Overview tab.
  7. In the Exposing section, shown in Figure 3, configure the exposure settings to expose different aspects of the process to specific groups of users, then click Ctrl+S to save. The example enables all users to start the process instance in IBM Process Portal.

    Figure 3. Exposing business process definitions
    Exposing business                     process definitions in Overview tab
Table 1 shows the settings that can be enabled in the Exposing section.

Table 1. Settings that can be enabled in the Exposing section
Exposure settingAction
Expose to startClick Select to choose the participant group whose members can start instances of this process in Process Portal. Members of the selected participant group can start instances of the process from the Launch tab in Process Portal.
Expose business dataClick Select to choose the participant group whose members can view ad hoc reports that include data for this process in Process Portal.
Expose performance metricsClick Select to choose the participant group whose members can view data for this process in the Process Performance dashboard in Process Portal.
Starting the process instance
Run the Standard HR Open New Position process in the Process Designer Inspector to see a step-by-step execution of a process model.
The Hiring Sample process application contains a single business process definition called Standard HR Open New Position. In Process Designer, in the Diagram tab of the Standard HR Open New Position, follow the steps below to start the process instance:
  1. Click the Standard HR Open New Position process diagram to ensure that it is selected.
  2. Click the run icon, shown in Figure 4, to launch an instance of the process.

    Figure 4. HR Open New Position process diagram
    HR Open New Position                     process diagram
  3. When you are prompted to switch to the Inspector, click Yes. The process diagram displays in the Inspector.
  4. Run the process from the Inspector as follows:
    1. Click the active instance in the left pane to display the new Submit job requisition task in the right pane of the Process Instances view.
    2. Click the run icon in the toolbar to start the task, as shown in Figure 5.

      Figure 5. HR Open New Position process diagram in Inspector
      HR Open New Position                     process diagram in Inspector

    The Submit job requisition task generates the Create Job Requisition Coach in a web browser. In IBM BPM V8, the Coaches are completely redesigned to contain Coach Views. Coach Views are reusable user interfaces that you can create and customize.
  5. Acting as the hiring manager, complete the requisition information, as shown in Figure 6, and click Submit.

    Figure 6. Fill in job requisition information
    Job                             requisition data dialog
  6. Set the Position Type to New.
  7. Click Next.
  8. Acting as the hiring manager, review the information in the Confirm Position Details Coach, shown in Figure 7, and clickSubmit.

    Figure 7. Hiring manager reviews and confirms requisition
    Hiring manager review                             and confirmation and requisition data
  9. Click the refresh icon refresh icon on the toolbar to update the Process Instances view. You can see that the Submit job requisition task is now closed. Because the hiring manager submitted a request to fill a new position, the General Manager must approve the requisition before it is routed to HR. Thus, the process instance moves to the next activity, and generates the Approve or reject requisition task for the General Manager, as shown in Figure 8.

    Figure 8. HR Open New Position process diagram showing process received
    HR Open New Position                             process diagram showing process received
  10. Click the run icon. In some cases, you may be prompted for a user account or a password to run the task.
  11. Select Approved, add a comment if desired, then click Submit.

    Figure 9. Final approval and submission of HR Open New Position 
    Final approval and                             submission of HR Open New Position
  12. You can also run the process from Process Portal. The BPM Process Portal has been redesigned to provide a highly collaborative work experience with increased social capabilities. To test the process in Process Portal, do the following:
    1. Select Start => All Programs => IBM => BPM Advanced 8.0 => Profiles => ProcCtr01 => Process Portal.
    2. Log on to Process Portal at http://localhost:9080/portal using administrator credentials (user ID and password of admin.
    3. Click the name of the process to create an instance of the BPD, in this case Standard HR Open New Position orStandard Hiring Sample, as shown in Figure 10.

      Figure 10. My Task list is Portal
      My Task list                                     is Portal

      The Submit requisition task opens.
    4. Acting as the hiring manager, complete the requisition information.
    5. Set the Position Type to New and click Next.
    6. Because the hiring manager submitted a request to fill a new position, the General Manager must approve the requisition before it is routed to HR, as shown in Figure 11.

      Figure 11. Approve or reject requisition
      Approve or                                     reject requisition and submit

      The process instance moves to the next activity, and generates the Submit requisition task, as shown in Figure 12.


      Figure 12. Submit requisition in My Tasks list
      Submit requisition in My Tasks list
Creating a snapshot
Snapshots record the state of library items in a process application or track at a specific point in time. A developer can capture a BPD at a specific point during development, for example to capture development milestones.
You can create new snapshots for process applications or toolkits from the Designer view and from the Process Center console. Snapshot management, such as installing, exporting, and archiving, is performed in the Process Center console. Here, you'll create a snapshot of the Standard HR Open New Position Process in IBM Process Designer, which will serve as a means to control the iteration of the process. In our example, the snapshot is called V1 because it is the first iteration of the process instance.
  1. Select Snapshot to take a snapshot of the process as shown in Figure 13.

    Figure 13. Create a snapshot
    Create a                     snapshot
  2. Click the plus sign next to Create New Snapshot.
  3. Give the snapshot a name, an acronym and a description, and click Create.
  4. Once the snapshot is created, you can view it in the Process Apps under Snapshots, as shown in Figure 14. Notice that it is not yet deployed.

    Figure 14. Current process on Process Server
    Current process on                             Process Server
  5. You can deploy process application snapshots to connected process servers or to offline process servers using the standard deployment service that is created for each process application. You can also customize the deployment service to add calls and scripts that perform specific functions when a process application is deployed on a server in another environment. Table 2 describes the various server environments.

Table 2. Process Server environment
EnvironmentDescription
DevelopmentBuild and refine your process applications in IBM Process Designer. Create your process models and implement the steps in those models using the Designer. Using the Inspector, demonstrate your development progress in playback sessions so that you can quickly evaluate and refine your prototype. Using the Process Center console, deploy your process applications in test and production environments.
TestUsing the Process Center console, deploy your process applications on the Process Server in your test environment to implement formal quality assurance tests. You can use the Inspector to help verify and resolve issues.
ProductionWhen all issues reported from formal testing are resolved, use the Process Center console to deploy your process applications on the Process Server in your production environment. You can use the Inspector to investigate and resolve any issues reported in your production environment.

  1. On the Process Apps tab, click the process application you want to deploy, and then click Snapshots.
  2. The Snapshots list displays all available snapshots and the status of each. Click Install next to the snapshot you want to deploy.
  3. In the Install Snapshot to Server dialog box, select the server or servers to which you want to deploy the snapshot and then click Install, as shown in Figure 15. In our example, we'll deploy to WS (IBMBPM).

    Figure 15. Install snapshot to WS (IBMBPM) server
    Select to install snapshot to                     WS (IBMBPM) server
  4. As shown in Figure 16, the V1 snapshot is now installed on WS (IBMBPM).

    Figure 16. V1 snapshot installed on WS (IBMBPM) server
    V1 snapshot                     installed on WS (IBMBPM) server
The V1 snapshot instance is now installed on the WS (IBMBPM) Process Server instance. It currently has no instances. The next step is to create instances to test the latest snapshot version on the runtime server.
Creating process instances
Next we want to create some process instances using the Authoring Environment Inspector. For example, Figure 17 shows four process instances deployed on the WS test process server with the V1 snapshot.

Figure 17. Process instances in V1 snapshot
Process instances                     in V1 snapshot 
The four process instances are visible in the V1 snapshot on the WS (IBMBPM)server in the Process Apps Snapshots tab, as shown in Figure 18.

Figure 18. Process instances in V1 snapshot on Process Apps Snapshots tab
Process instances in V1                     snapshot on Process Apps Snapshots tab 
You can also view the process instances in the Process Portal, as shown in Figure 19.

Figure 19. V1 snapshot viewed in the Process Portal
V1 snapshot viewed in the Process Portal 
Creating and deploying a new version
To differentiate snapshot V1 from V2, we've simplified the process in V2 by removing the General Manager approval, making this process simple and subject to orphaned tokens.
Repeat the steps in Creating a snapshot to create a V2 process instance, or snapshot, as shown in Figure 20. Figure 20 illustrates a greatly simplified version of the process that will help illustrate the problem of orphaned tokens. A token becomes orphaned when its associated activity is removed from a BPD of a migrated snapshot. The easiest way to identify and manage orphaned tokens is to generate a policy file and use it to specify whether each potential orphaned token should be moved or deleted during instance migration. We'll cover this in Managing orphaned tokens.

Figure 20. Simplified Standard HR Open New Position without GM approval
Simplified Standard                     HR Open New Position without GM approval 
Table 3 shows the migration options for the new snapshot.

Table 3. Migration options for installed snapshot
Online migration optionOffline migration optionDescription
Leave running instances on current version of the snapshotLeaveThe instances currently running continue to completion using the previously deployed version of the snapshot.
Migrate running instances to new version of the snapshotMigrateCurrently running instances are migrated to the new snapshot you are deploying. Wherever the running instances are in the flow of the process, the new version is implemented for the next item or step.
Delete running instances of current version of the snapshotDeleteThe instances currently running are immediately stopped and do not continue to completion. All records of the running instances are removed from the process server.
Note: Delete is not available on production servers.

When deploying the V2 snapshot, select the Leave option, as shown in Figure 21, so that the active instances continue to run using the old V1 snapshot. Then click Install. Later, you can go to the Process Admin console to perform the instance migration and specify the orphan token policy file.

Figure 21. Select Leave option and install snapshot
Install snapshot                     dialog showing Leave selection 
As you can see in Figure 22, the new snapshot has no instances yet.

Figure 22. V2 snapshot on WS server with no process instances 
V2 snapshot on WS server with                     no process instances 
Figure 23 shows the V1 snapshot with six process instances; two are already completed and four are active.

Figure 23. V1 process instances on WS server
V1 process instances 
Figure 24 shows three currently active process instances on the V2 snapshot.

Figure 24. V2 active process instances
V2 active process instances 
As depicted on Figure 25 shows the V1 and V2 snapshots deployed on WS (IBMBPM) with three active instances on V2 and 4 active instances on V1.

Figure 25. V2 process instances
V2 process instances 
Select Server details for more information on the WS (IBMBPM) server, as shown in Figure 26.

Figure 26. Server details
WS (IBMBPM) server                     details 
Figure 27 shows a the Hiring Sample on server WS (IBMBPM) with the two snapshots, V1 and V2. Click Configure Server to open the Process Admin Console of the WS Server. You can use the Process Admin Console to administer and configure runtime settings for snapshots that are installed on a process server.

Figure 27. Hiring Sample instances for V1 and V2
Hiring Sample instances for V1                     and V2 
In Figure 28, you can see that the server is WS and there are three processes started. Click Process Inspector.

Figure 28. Process Admin Console of WS (IBMBPM) server
Process Admin                     Console of WS (IBMBPM) server 
When you click Installed Apps in the Process Admin Console, you can see the list of snapshots of process applications that have been installed on the current Process Server. In each process application snapshot, only the processes that have been exposed are shown. For each process, you can see the number of instances currently running. As you can see in Figure 29, V1 is the active and default snapshot and V2 is simply shown as active.

Figure 29. V1 and V2 process instances on WS (IBMBPM)
V1 and V2 process instances                      on WS (IBMBPM) 
On a process server, the first snapshot you install is considered the default version. This means that the items in it run when an event or other trigger that applies to more than one version of a process or service is received. When you install subsequent snapshots, you can use the Make Default Version option, shown in Figure 30, in the Process Admin Console to ensure that the snapshot you want to run is the default. Note that this option is available only if you have more than one snapshot on the server.

Figure 30. Make Hiring Sample (HSS) – V2 (Active) the default version
Make Hiring Sample (HSS) – V2                     (Active) the default version 
V2 is now the active and default snapshot, as shown in Figure 31.

Figure 31. Hiring Sample (HSS) – V2 on WS (IBMBPM) shown as Active and Default
Activating HSS – V2 on WS                     (IBMBPM) shown as active and default 
Deactivating an installed application
Use the Process Admin Console to deactivate and, if necessary, stop snapshots that are installed on a process server. All installed snapshots except the default version can be deactivated.
Verify that the Hiring Sample (HSS) – V1 is active and click Deactivate Application, as shown in Figure 32, to deactivate the V1 snapshot. This deactivates the snapshot, allowing all currently running instances to complete. Deactivated snapshots remain installed, but you cannot start a new instance of the exposed processes or services.

Figure 32. Deactivate an installed application
Deactivate                     an installed application 
Managing orphaned tokens
IBM BPM V8 provides a new set of capabilities for managing orphan tokens, which result from migration of in-flight process instances to a new process application snapshot that does not have some of the activities that existed in the previous snapshot. The new capabilities include analysis of two snapshots to identify steps that can potentially result in orphan tokens, as well as the ability to define orphan token handling policies and a set of REST APIs for moving or deleting orphan tokens.
An orphaned token is a pointer that is associated with an activity that was removed from a business process definition (BPD). You can use a policy file, a REST API, or the Process Inspector to manage orphaned tokens.
Some examples of changes between snapshots are:
  • Cosmetic, such as formatting to a Coach (human service)
  • Changes in business variables
  • Modification of flow, such as gateways or swim lanes
  • Changes to undercover agents (UCAs) on the default snapshot
Always compare snapshots before instances are migrated to identify the potential locations of orphaned tokens.
Table 4 shows the options for where to move orphan tokens when installing version two of a snapshot.

Table 4. Where to move orphan tokens
Source Activity LocationTarget in Same ProcessTarget in Parent ProcessTarget in Child SubprocessTarget in Child Event SubprocessTarget in Child Linked Process
ProcessYesN/AYesNoNo
SubprocessYesYesYesNoNo
Event subprocessYesNoYesNoNo
Linked processYesNoYesNoNo

You can run the wsadmin command as shown in Listing 1 to get a list that you can use to spot potential migration problems so that when you migrate to the new version of the snapshot, you have some confidence that it will work. When the file is generated, it will identify all the potential orphaned tokens. Go through the file and identify how you want to handle each token. During migration, if a policy file is specified, the migration uses that file to determine whether to delete or move orphaned tokens.

Listing 1. wsadmin scripting command to sign into the process application
wsadmin -conntype SOAP -port 8880 -host localhost -user admin 
-password admin -lang jython

In the example in Listing 1, localhost is the name of the host in the wsadmin.properties file that is specified by thecom.ibm.ws.scripting.port property. Make sure to substitute your own port, host name, user name, and password when creating a connection and don't forget to specify the language (jython, in this example).
The following example illustrates how to establish a SOAP connection to the Process Center server and then generate an orphaned token policy file for a process application with acronym HSS, which is the acronym that comes from the default Standard HR Open New Position process.
All wsadmin commands for managing process applications must be run in connected mode and the server must be running. Use the -contype argument to indicate what connection type you want to use (SOAP or RMI).

Listing 2. wsadmin scripting command for checking orphan tokens
AdminTask.BPMCheckOrphanTokens('[-processAppAcronym HSS 
-sourceSnapshotName "V1" -targetSnapshotName "V2" -outputFile 
C:\TokenPolicyFile_HSS_V1_to_V2.xml]')

In Listing 2, the BPMCheckOrphanTokens command compares two snapshots and detects the possibility of orphaned tokens before installing a new snapshot and then identifies whether to delete or move each token. The parameters used are as follows:
  • processAppAcronym: A required parameter that identifies the process application that is to be installed; in this case, HSS.
  • sourceSnapshotName: A required parameter that provides the snapshot name from which instances will be migrated (the old version); in this case, V1.
  • targetSnapshotName: A required parameter that provides the snapshot name to which instances will be migrated (the new version); in this case, V2.
  • outputFile: A required parameter that provides the file path to the directory where the orphan token policy file will be generated and a name for that file; in this case, C:\TokenPolicyFile_HSS_V1_to_V2.xml. Use this XML file during the migration of active process instances.
Figure 33 shows the outcome of the wsadmin command that generated the policy file, extracted from the C:\TokenPolicyFile_HSS_V1_to_V2.xml.

Figure 33. Check orphan tokens
Check orphan                     tokens - XML file snippet 
Migrating the in-flight instances
The deployment process checks to see whether the target server is currently running any instances of the BPDs business process definitions included in the deployed snapshot. If it detects one or more running instances on the target server, you are asked whether you want to migrate those running instances to the new snapshot.
  1. From the Process Admin Console, select Installed Apps.
  2. From the list of snapshots, select the one to which you are migrating data, and click Migrate Inflight Data, as shown in Figure 34, then select the snapshot from which you want to migrate data.
    You can lessen the risk that your instance will not complete by carefully managing tokens that are associated with actions that have been deleted. If you are using a policy file to manage potential orphaned tokens, provide the path to that policy file before you click Migrate.


    Figure 34. Migrate in-flight data
    Select to migrate                     in-flight data
The next section shows migration of a very simple process that is used for generation of user tasks.
Migrating in-flight data from a snapshot
You can use the Process Admin Console to migrate in-flight data to a snapshot. After you install a new snapshot that replaces a snapshot with instances that are still running, you might want to migrate the currently running instances to the new snapshot. To do this, use the Migrate Inflight Data dialog to migrate currently running instances to the new version of the selected snapshot. Wherever the running instances are in the flow of the process or service, the new version is implemented for the next item or step.
  1. Click on the V1 snapshot to mark this as the snapshot from which to migrate data.
  2. Select the policy file created in the last section (C:\TokenPolicyFile_HSS_V1_to_V2.xml and click Migrate to migrate to V2, as shown in Figure 35.

    Figure 35. Migrate In-flight data from snapshot
    Migrate In-flight                     data dialog
  3. If the migration does not produce any orphaned tokens, the file is used but no error tokens are generated. Figure 36 shows the migration summary in this case. You can see that the Process Application, Process Instance Migration and Orphan Token Policy generated no errors. Click Close after verifying that the results showed no errors. In the next section, we'll give you an example where orphan tokens are identified and describe possible actions.

    Figure 36. Process instance migration summary
    Process instance                     migration summary
  4. Verify that the processes have been migrated to V2. In Figure 37, you see that there are 6 process instances, with 3 completed and 3 still active.

    Figure 37. V2 process instances on WS server
    V2 process instances on WS                     server
Handling orphaned tokens
Remember that orphan tokens result from migration of in-flight instances to a new BPD snapshot. They are pointers associated with an activity that is no longer a part of the BPD.
You can only delete or move a token that is a leaf node in the execution tree; any parent orphaned tokens will be handled implicitly. For example, suppose an activity implemented as a subprocess is deleted in a new snapshot. Deleting an orphaned token in the subprocess will also delete the orphaned token on the parent activity.
For a parallel gateway, both branches must complete to complete the process successfully. Therefore, if you choose to delete an orphaned token on one branch of a parallel gateway, the process using the parallel gateway will never be able to complete.
When you move a token, you are actually deleting it from one activity and creating a new copy of it attached to a different activity. This behavior creates a limitation if you are using multiple instances of nested business objects. For example, if you have an activity that has three tokens associated with it and you move those tokens to a second activity, only one token is created on the second activity.
Figure 38 shows an example of failed tokens that can't be activated.

Figure 38. Failed token error in process
Failed token error in process 
Figure 39 shows an example of a failed BPDIBD token from a generated XML file.

Figure 39. Failed token extracted from XML file
Failed token extracted                     from XML file 
Conclusion
Running processes can be modified to include new actions and steps, as well as have these actions affect any future processes. Users can choose whether to apply changes to the individual processes or update the process model, which forces other processes to also adopt the change. In this article, you learned about two features in Business Process Manager V8: migrating in-flight process instances and orphaned token management that can help you deal with managing process migration complexities for long-running processes.