Using the Concurrency Quota policy

Learn how to monitor web service concurrency performance by specifying a concurrency limit (quota).

For information about using policies in the context of the Community Manager developer portal, see Business Policies.

Table of Contents

• Introduction
• Creating a Concurrency Quota policy
• Concurrency Quota policy options
• Configuration
• Activating a policy
• Attaching a policy
• Concurrency Quota policy behavior
  • Policy behavior—summary
  • Runtime behavior: default
  • Runtime behavior with grid services enabled (com.soa.grid)
  • Response time effect on concurrency quota counter
• Concurrency Quota policy: use cases for Policy Manager
• Concurrency Quota policy: use cases for Community Manager

Introduction

A Concurrency Quota policy is a Quality of Service (QoS) policy that allows you to:

• Specify a limit (quota) for the number of concurrent requests allowed to the API/service or contract.
• Configure a fault message that is returned to the consumer, and an alert that is generated, if a request exceeds the predefined quota.

You can define a fault message by specifying the following values:

• Name (Fault Message)
• SOAP fault element that holds errors and status information for a SOAP message (Fault Element Name)
• URL that points to the fault code schema (Fault Namespace URI)

You can also define Alert Codes in the Alerts > Alert Codes section and enable and select them in the policy definition. For more information, see Using Alert Codes.

If the specified concurrency limit is exceeded, Policy Manager returns a fault and sends an alert.

Creating a Concurrency Quota policy

The first step in creating a policy is to define the basic policy information.

To add a QoS policy

1. Go to Workbench > Browse > Organization, and select Policies > QoS Policies. The Policies Summary is displayed.
2. Click Add Policy.
3. Choose the policy type and click Next.
4. Specify a name (required) and description (optional) and click Finish. At the Completion Summary, click Close. The Add Policy Wizard creates a draft policy instance that you can then configure on the Policy Details page.

For more information, see Add Policy.

Concurrency Quota policy options

The policy includes the configuration options listed below.

• Concurrency Limit: Allows you to specify a concurrency limit (quota) that represents the maximum number of concurrent requests allowed. If the limit is exceeded, a fault message is returned to the consumer and an alert is logged.
• Fault Message: The name of the SOAP fault message to be returned if the quota is exceeded (<faultString>).
• Fault Element Name: The name of the SOAP fault element that holds errors and status information for a SOAP message.
• Fault Namespace URI: The URL that points to the fault code schema.
• Fault Namespace Prefix: Allows you to enter the fault namespace prefix.
• Time to Clear Alert(s): Allows you to enter the length of time for which the throughput needs to be within the limit before a second alert is sent to signify that the consumer is once again within the quota.
• Set Reminder Alert: Enables the Reminder Alert function. This function sends the alert that is selected on the Alert Code drop-down list. The alert is sent at the Reminder Alert Interval until the consumer is within the quota.
• Reminder Alert Interval(s): Allows you to enter an interval (in seconds) at which Reminder Alerts are sent until the consumer is within the quota.
• Alert Code: A drop-down menu that displays a list of custom alert codes that you have defined. If you have "Send Reminder Alert" selected, alerts are sent based on the alert interval defined using the selected Alert Code. For more information on defining an Alert Code, see Using Alert Codes.
• Clear Alert Code: A drop-down menu that displays a list of custom alert codes that you have defined. If you have "Send Reminder Alert" selected, alerts are sent when the alert condition no longer exists.
• Reminder Alert Code: A drop-down menu that displays a list of custom alert codes that you have defined. If you have "Send Reminder Alert" selected, alerts are sent based on the specified interval when the condition continues to be exceeded.

Configuration

Let's take a quick walkthrough of the Concurrency Quota policy configuration process to get you started.

Step 1: Add policy

In Policy Manager, to create a Concurrency Quota policy instance, go to Policies > QoS Policies and choose Add Policy.

Step 2: Modify policy

When you click Modify to make changes to the Concurrency Quota policy on the Policy Details page, the initial policy looks like this:

Configure the policy options based on your requirements and click Apply.

Step 3: Activate policy

When the policy configuration is complete, activate the policy so that you can use it.

Step 4: Attach policy

After you've saved and activated your policy, you can attach it to a contract or service.

Step 5: Test policy and view monitoring data

After you've attached the Concurrency Quota policy to a contract or service, send a request to your service and go to the Services > Monitoring section to view the results for Logs, Real Time Charts, and Historical Charts.

Activating a policy

When you create and configure a policy, the policy is in Draft state. When the policy configuration is complete, activate the policy: click Activate Policy and then confirm. See Activate a Policy.

A policy in Draft state is not available for general use. Once you activate the policy, it is in Active state and is available for use.

Attaching a policy

To use the policy, go to the Policies folder in the respective organization and attach the policy.

You can attach the Concurrency Quota policy to:

• A contract
• A service

  Note: Although the Community Manager developer portal user interface allows attaching a policy at the operation level, the Concurrency Quota policy works correctly only when attached to the service, not to an individual operation.

Depending on where the policy is attached, the concurrency counter, which tracks the number of open requests, is activated for the service or for the contract.

Concurrency Quota policy behavior

In this section:

• Policy behavior—summary
• Runtime behavior: default
• Runtime behavior with grid services enabled (com.soa.grid)
• Response time effect on concurrency quota counter

Policy behavior—summary

When the Concurrency Quota policy is in use, there is a counter to determine whether the concurrency limit has been reached.

If the policy is attached at the contract level, the counter name is derived from the contract key. If the policy is attached at the service level, the counter name is based on the service key.

When this policy is attached to a service or contract, the Network Director does the following:

1. When a message request is received:
   1. Checks to see if there is an existing counter. If there is no counter for the API contract or service, it creates a counter with a value of 0, following a specific naming convention for the contract or service.

      The counter represents the number of threads active for the API based on its context (contract or service, depending on where the policy is attached), when the limit has not been exceeded.

   2. Increments the counter by 1.

   Conditional: If the pre-set limit is exceeded, the Network Director initiates alert processing as defined in the policy.

2. When the corresponding response is received, the Network Director sends the response back to the client. At this point, transaction processing is complete. The Network Director then decrements the counter by 1.

Runtime behavior: default

By default, the Concurrency Quota policy settings apply only to messages processed by the container, even if the container is in a cluster.

For example, if the policy allows 16 concurrent requests, and the Network Director container receives an additional request while 16 requests are still open, alert processing is initiated.

Runtime behavior with grid services enabled (com.soa.grid)

With grid services enabled, the counter is shared across all Network Director containers in a cluster.

For example, if the Concurrency Quota policy is set to 16 requests, and there are four Network Director containers, the limit is 16 requests between all four Network Director containers.

For more information about enabling grid services, see Akana Grid Services.

Response time effect on concurrency quota counter

The counter is incremented for each request, and decremented when each response is complete. Therefore, if the response time is longer, it is more likely that the error limit will be reached since the counter is not decremented until the response is sent back to the client and the handlers are closed.

Let's say for example you attach this policy to two APIs, one of which has a significantly slower response time. With the same number of messages to both APIs, the API with the slower response time is likely to reach the concurrency limit sooner than the API with the faster response time.

In addition, if the policy is attached to an API that has multiple operations, and one of the operations has a higher response time, taking longer to process, the total number of requests for this one operation could keep the counter value high for a longer duration. This could result in calls to the other operations being rejected, even though the other operations have a much faster response time.

Concurrency Quota policy: use cases for Policy Manager

This section provides a list of Policy Manager-specific usage scenarios for the Concurrency Quota policy.

It includes:

1. Scenario 1: Generate alert when concurrent requests received
2. Scenario 2: Generate reminder alert when concurrent requests received
3. Scenario 3: Generate alert when using contracts
4. Scenario 4: Generate clear alert when concurrency exceeds specified interval

Scenario 1: Generate alert when concurrent requests received

Generate an alert and send an email to the administrators if a service receives more than 10 concurrent requests.

1. Create a physical service in the Policy Manager Management Console using Create Physical Service.
2. Provide service details and click Finish.
3. Using Virtualize Service, virtualize and host the physical service on Network Director (ND1), and assign a name (for example, Vs1).
4. Navigate to Organization > Policies > QoS Policies and use Add Policy to create a Concurrency Quota policy.
5. Configure the Concurrency Quota policy with 10 Concurrency Limit and a custom Fault message.

   

6. Activate the policy, and attach it to the Vs1 service in the Service Details > Policy Attachments > QoS section.
7. Send 20 concurrent requests from the application/client to the Vs1 service.
8. When the concurrency limit has been exceeded, the Fault Message is returned to the service consumer and an alert is logged at the service level.

   

9. The details in the usage logs show the error being displayed as configured in the policy.

   

10. In the alert code, an email must be configured that will be sent to the administrator(s).
11. Timeline

    

Scenario 2: Generate reminder alert when concurrent requests received

Generate a remainder alert and run a Management Script if more than 10 concurrent requests per 10 seconds are received repeatedly by a service in the 100 seconds.

1. Create a physical service in the Policy Manager Management Console using Create Physical Service.
2. Provide service details and click Finish.
3. Using Virtualize Service, virtualize and host the physical service on Network Director (ND1), and assign a name (for example, Vs1).
4. Navigate to Organization > Policies > QoS Policies and use Add Policy to create a Concurrency Quota policy.
5. Configure the Concurrency Quota policy with 10 Concurrency Limit and a custom Fault message. Set Remainder Alert interval to 10 seconds and Clear alert time interval to 100 seconds.

   

6. Activate the policy, and attach it to the Vs1 service in the Service Details > Policy Attachments > QoS section.
7. Send 20 concurrent requests from the application/client to the Vs1 service. Results:
   1. When the concurrency limit has been exceeded, the Fault Message is returned to the service consumer and an alert is logged at the service level.
   2. The details in the usage logs show the error being displayed as configured in the policy.
8. Keep sending 20 concurrent requests from the application/client to the Vs1 service. Results:
   1. During the first 10 seconds no alert is shown.
   2. If the concurrency hits are seen after 10 seconds, a remainder alert is thrown which is valid for the next 10 seconds.
   3. After the first interval of 10 seconds, if the concurrency hits are still observed, there is another remainder alert.

      

   4. Within the clear interval, for every 10 seconds, this alert is thrown if validation crosses
   5. In the Alert code, a script can be configured to run (for example, to store any sort of information related to the alert).

Scenario 3: Generate alert when using contracts

Generate an alert if an application hits a contract when sending more than 15 concurrent requests to various services using the contract

1. Create a physical service in the Policy Manager Management Console using Create Physical Service.
2. Provide service details and click Finish.
3. Using Virtualize Service, virtualize and host the physical service on Network Director (ND1), and assign a name (for example, Vs1).
4. Navigate to Organization > Policies > QoS Policies and use Add Policy to create a Concurrency Quota policy.
5. Configure the Concurrency Quota policy with 15 Concurrency Limit and a custom Fault message.
6. Activate the policy, and attach the Concurrency Quota policy to a contract which serves Vs1 and Vs2 services.

   

7. Send 25 concurrent requests from the application/client to Vs1 and Vs2 services all together
   1. When the concurrency limit has been exceeded, the Fault Message is returned to the service consumer and an alert is logged at contract level.
   2. The details in the usage logs show the error being displayed as configured in the policy.

Scenario 4: Generate clear alert when concurrency exceeds specified interval

Generate a clear alert and send an email to the administrator when concurrency is no more exceeded in the 100 seconds interval

1. Create a physical service in the Policy Manager Management Console using Create Physical Service.
2. Provide service details and click Finish.
3. Using Virtualize Service, virtualize and host the physical service on Network Director (ND1), and assign a name (for example, Vs1).
4. Navigate to Organization > Policies > QoS Policies and use Add Policy to create a Concurrency Quota policy.
5. Configure the Concurrency Quota policy with 10 Concurrency Limit and a custom Fault message.
6. Activate the policy, and attach it to the Vs1 service in the Service Details > Policy Attachments > QoS section.
7. Send 20 concurrent requests from the application/client to the Vs1 service.
   1. When the concurrency limit has been exceeded, the Fault Message is returned to the service consumer and an alert is logged at the service level.
   2. The details in the usage logs show the error being displayed as configured in the policy.
   3. In the next 100 seconds, if the policy is not violated, a clear alert is sent.

      

   4. In the clear alert code, an email must be configured that will be sent to the administrator(s).

Timeline (Scenarios 2 and 4)

Concurrency Quota policy: use cases for Community Manager

This section provides a list of Community Manager-specific usage scenarios for the Concurrency Quota policy.

It includes:

1. Add / configure Concurrency Quota policy
2. Scenario 1: How do I verify Concurrency Quota policy at the service level?
3. Scenario 2: How do I verify Concurrency Quota policy at the contract level?
4. Scenario 3: How do I verify Concurrency Quota policy at the license > term level?

Add / configure Concurrency Quota policy

If you are using a Concurrency Quota policy with Community Manager, you must first add the policy to the Community Manager Tenant Organization in Policy Manager. Once that's done, it is available for selection in the Community Manager developer portal.

1. Go to the Policy Manager Management Console > Tenant Organization > Policies > QoS Policies and click Add Policy.

   

2. Specify Policy Name, Policy Type (Concurrency Quota policy), click Finish, and then click Close.

   

3. On the Concurrency Quota policy Details screen, click Modify.

   

4. Provide the policy details and click Finish.

   

5. Activate the policy.

   

   

Scenario 1: How do I verify Concurrency Quota policy at the service level?

1. Launch Community Manager.
2. Attach the Concurrency Quota policy to the API implementation.

   

3. Request API access. Verify that the contract is activated.

   

4. Go to Test Client and send continuous requests. This policy condition is met only when hitting concurrent requests (i.e., sending multiple requests in 1 second). This behavior is shown below using SoapUI and Test Client.
5. Start one load test to send continuous requests from SoapUI. Also send requests from Test Client one by one. When multiple requests are received, the policy condition is met and you'll see a fault message in the response window.

   

6. Alerts are displayed in the Community Manager API > board (as shown below).

   

Scenario 2: How do I verify Concurrency Quota policy at the contract level?

1. Attach the Concurrency Quota policy at the Contract Level and activate the contract.

   

2. Now Go to Test Client and send continuous requests. This policy condition will meet only when hitting concurrent requests (i.e., sending multiple requests in 1 second).
3. This behavior is shown below using SoapUI and Test Client.
4. Start one load test to send continuous requests from SoapUI. Also send requests from Test Client one by one. When multiple requests are received, the policy condition is met and you'll see a fault message in the response window.

   

5. Alerts are displayed in the Board for the API.

   

6. Alerts will also be displayed in the Policy Manager Registry.

   

Scenario 3: How do I verify Concurrency Quota policy at the license > term level?

Note: When defining a license, there are two options: API-Wide Mapping and Operation-Specific Mapping. The Concurrency Quota policy works at the license term level only for API-wide mapping. Operation-specific mapping is not supported.

1. Edit License and attach Concurrency Quota policy.

   

2. Enable licensing for the API, with API-wide mapping:
   1. Go to API > Details > Manage Licensing. In the Manage Licensing page, check Enable Licensing for API. Leave the default mapping setting, which is API-Wide Mapping.
   2. Assign the scope that is associated with the license that has the Concurrency Quota policy as a term, to the entire API.

      

      Note: The Concurrency Quota policy works only with the API-Wide Mapping setting. Operation-specific mapping is not available.

      For detailed instructions, see API Licenses and Scopes (developer portal help).

3. Next, test the Concurrency Quota policy at the License term level. This assumes that the same scope (as above) is used for Scope mapping, and the same license is selected while requesting API access.
4. Go to Test Client and send continuous requests. This policy condition is met only when hitting concurrent requests (that is, sending multiple requests in 1 second).
5. This behavior is shown below using SoapUI and Test Client.
6. Start one load test to send continuous requests from SoapUI. Also send requests from Test Client one by one. When multiple requests are received, the policy condition is met and you'll see a fault message in the response window.

   

7. Alerts are displayed in Community Manager, in the Board for the API.

   

   

   

   In the API Monitoring Logs, the following logs will display for concurrency faults.

   

Related Topics