Using the HTTP Message Validation Policy

Learn how to use the HTTP Message Validation policy to validate HTTP messages based on criteria contained in the API description document that was used to define the API (Swagger, RAML, or WADL).

For information about using policies in the context of the Community Manager developer portal, see Business Policies.

Table of Contents

Introduction

The HTTP Message Validation Policy allows you to validate HTTP messages based on criteria specified in the API description document that was used to create the API in the Community Manager developer portal (Swagger, RAML, or WADL). For information on supported versions, see What types of API description document can I use to define my API?

It includes:

  • Validation of form, query, and header parts.
  • Complete validation of JSON message payloads based on JSON Schemas contained in the Swagger or RAML documents used to define the API.

For information about the JSON Schema rules for HTTP message validation, see:

For information about defining an API description document, refer to the applicable specification. For example, for Swagger 2.0: https://swagger.io/specification/v2/.

UK Open Banking support

This policy also supports conformance to the UK Open Banking specification (version 3.1) with regard to error messaging. When the UK Open Banking option is enabled for version 3.1, the error messages returned to the API client application are formatted according to the Open Banking 3.1 specification, section titled Standard Error Code(s).

In version 2019.1.0 and later, you can customize the URL field returned in the error message, as defined in the specification. For details, see Documentation URL below.

Creating an HTTP Message Validation Policy

The first step in creating a policy is to define the basic policy information.

To add an operational policy

  1. Go to Workbench > Browse > Organization and select Policies > Operational Policies. The Policies Summary is displayed.
  2. Click Add Policy.
  3. Choose the policy type and click Next.
  4. Specify a name (required) and description (optional) and click Finish. At the Completion Summary, click Close. The Add Policy Wizard creates a draft policy instance that you can then configure on the Policy Details page.

For more information, see Add Policy.

Configuring the HTTP Message Validation Policy

Once you've defined the basic policy information, you can configure the technical details that determine how the policy works when it's attached to a service.

To configure an HTTP Message Validation Policy

  1. Go to Workbench > Browse > Organization and select the Policies > Operational Policies folder. The Policies Summary is displayed.
  2. Find the policy on the list and double-click to go to the Details page for the policy.
  3. In the second panel, click Modify to access the Specify HTTP Message Validation Policy Options page.
  4. Choose one or more validation scopes to determine which messages in a transaction should be validated by the policy. Request is the default. For details, see Validation Scope below.
  5. Choose one or more processing options. For details, see Processing Options below.
  6. Click Finish.

After you've configured your policy, you can activate it, then attach it to a web service, operation, or binding.

HTTP Message Validation Policy options

Specify HTTP Message Validation Policy Options

On this page, you can specify:

Refer to the field descriptions below.

Validation Scope

Identifies which messages in a transaction should be validated by the policy. Choose one or more of the following (Request is the default value):

  • Request: All input messages, coming in from client applications, are validated.
  • Response: All normal output messages, being returned from the API endpoint, are validated.
  • Fault: Error responses returned from the API endpoint are validated.

Processing Options

This section has options to enable or disable several features of the message validation and reporting of detected errors.

Fail early
If this option is enabled, the policy stops processing the message after the first validation error. If not enabled (the default), the policy continues to test all remaining validation rules and report any other errors in the message.
This option enables you to reduce the overhead of message validating by terminating validation at the first error. A typical use is in a production environment where processing speed is particularly important.
Note: If the policy encounters a significant issue, such as a type validation error, it fails even if this setting is cleared.
Key benefit: Enabled: efficiency. Disabled: more detailed validation errors.
Enable format validation
If this option is enabled, the policy applies stricter validation by enabling the format JSON Schema keyword. For example, the rules for date-time formats and for email addresses are strict. If disabled (the default), less strict validation is applied.
Per the JSON Schema specification, this validation is optional. For more information, see https://tools.ietf.org/html/draft-wright-json-schema-validation-01#page-14: Section 8: Semantic Validation with "Format". For valid values, see Format keyword options below.
Key benefit: Enabled: stricter adherence to JSON Schema rules. Disabled: less processing overhead.
Verbose errors
If this option is enabled, detailed validation error messages are returned to the API client application. If disabled (the default), a generic summary error message is returned.
Note: The Gateway container log always contains all message validation error details.
Key benefit: Enabled: more detailed error messaging. Disabled: less processing overhead.
Coerce strings
With this option enabled, the policy treats strings as numeric, Boolean, or null values as appropriate for validation purposes. For example, if a string looks like a number and the validation rules expect a number, the rule will pass. By default, all validation is applied to values exactly as they are presented.
Key benefit: Enabled: efficiency. Disabled: stricter validation.
Exact decimals
Controls how decimal numbers are compared. Normal comparison of decimal numbers treats values such as 2.25 and 2.250 as being equal. If this option is enabled, the number of significant digits becomes a factor in the comparison of decimal numbers; values such as 1.1 and 1.100 are not considered equal, since they do not have the same number of significant digits. With this option disabled, decimal numbers are compared based on their actual value without regard to the number of trailing significant zeroes.
Key benefit: Enabled: stricter validation. Disabled: more relaxed validation.
Treat [null] as [ ]
With this option enabled, the policy treats an array of null ([null]) the same as an empty array ([]). An empty array in JSON schema is often defined as a null array. By default, strict JSON Schema rules are applied, so these two arrays are not equal.
Key benefit: Enabled: more relaxed validation. Disabled: stricter adherence to JSON Schema rules.
UK Open Banking
Enable this option for conformance to the UK Open Banking specification with regard to the format of error messages returned to the client application. Check the box and then specify the version. Versions up to 3.1 are supported, as shown below.
When this option is enabled for version 3.1, the error messages returned to the API client application are formatted according to the Open Banking specification version 3.1; see Open Banking 3.1 specification, section titled Standard Error Code(s). By default, when the option is not enabled, only a generic summary error message is returned.
Note: The API Gateway container log always contains all message validation error details.
Version: 2019.1.0 and later
Documentation URL: If you choose Version 3.1, an additional field appears, Documentation URL:
Specify JOSE Policy v2 Option for Open Banking version 3.1: Documentation URL
In this field, you can specify the documentation URL to include as part of Open Banking-compliant error messages returned by the policy. The default value is the URL for the UK Open Banking Read/Write Data Specification version 3.1.1: https://openbanking.atlassian.net/wiki/spaces/DZ/pages/1000702294/Read+Write+Data+API+Specification+-+v3.1.1. You can change the value to another valid URL, which will be used in the Url field in error messages returned by the HTTP Message Validation policy, as specified in the Error Response Structure section of the specification.
Allow additional properties by default
Version: 2019.1.0 and later
Determines the default behavior when an additionalProperties property in a Schema object is not specified in a Swagger schema:
  • Enabled (the default value): all additional properties are allowed.
  • Cleared: additional properties are not allowed for that schema.
For more information about Swagger support of the additionalProperties property in a Schema object, refer to the specification: Swagger 2.0 or OAS 3.0.
Log additional properties
Version: 2020.1.3 and later
Indicates the behavior if the Allow additional properties by default field is enabled and the request message includes additional properties that are not defined in the API schema (from the uploaded API definition document, or as defined in the Community Manager developer portal in the API Designer):
  • Disabled (the default value) additional properties are allowed, and no alert is generated.
  • Enabled: all additional properties are allowed, and an alert is generated.
Note: If there is a Basic Auditing or Detailed Auditing policy on the API, the additional properties are also logged in the usage logs (unknown_validation_properties) if this setting is enabled.
If the Allow additional properties by default setting is disabled, or the API definition schema itself disallows additional properties, this setting does not have any effect.
Default: Disabled.
Ignore extra JSON in payload
Version: 2019.1.5 and later
Determines the behavior if a JSON payload includes extra content. By default, an error is returned if any extra content is encountered after the initial JSON object in the payload.
If you disable this option, and there is extra content after the initial JSON object in the payload, it is ignored.
Default: Enabled.
Allow All Headers
With this option enabled (the default), all headers are allowed, and are validated if they are specified in the schema for the calling operation.
If you want to specify one or more headers that should not be allowed, clear the Allow All Headers box and then, in the Headers not allowed table, specify the bad headers. Then specify the action to be taken if bad headers are found. Options:
  • Strip unallowed headers checked: All headers listed in the Headers not allowed table are stripped, and the message is sent on without them.
  • Strip unallowed headers cleared: If any headers listed in the Headers not allowed table are found, an error is thrown and the message is aborted.

If you want one or more headers to be excluded from validation, add them in the Headers excluded from validation table.

An example is shown below.

Specify HTTP Message Validation Policy Options for headers

Note: certain headers, such as the Accept header, are vital to the HTTP message and should never be included in the list of headers that are not allowed.

Format keyword options

Refer to the tables below for the validation specification for format keyword options.

Examples

Below are some examples of type within the format keyword.

value Validation spec
int32 32-bit signed integer
int64 64-bit signed integer
float Single-precision floating point
double Double-precision floating point
binary All strings validate true
password All strings validate true
byte Standard Base64 encoded strings
date yyyy-mm-dd
date-time RFC 3339
email RFC 5322, section 3.4.1
hostname RFC 1034, section 3.1
ipv4 RFC 2673, section 3.2
ipv6 RFC 2373, section 2.2
uri URIs (absolute) as per RFC 3986
uri-reference URI Reference (URI or relative URI) as per RFC 3986
base64 Standard base64 encoding for RFC 3548 or RFC 4648
base64url Standard base64url with URL and Filename Safe Alphabet (RFC 4648 sec 5 Table 2: The "URL and Filename safe" Base 64 Alphabet)
regex

Regular Expressions (java.util.regex.Pattern rules)

Note: If you're using regular expressions, refer to Using regular expressions for information on syntax and limitations.
time ISO-8601 extended local time format.
uuid RFC 4122

Options not yet implemented/supported

The following options for the format keyword are not yet available.

value Validation spec
json-pointer (not yet supported)
uri-template (not yet implemented)

Activating a policy

When you create and configure a policy, the policy is in Draft state. When the policy configuration is complete, activate the policy: click Activate Policy and then confirm. See Activate a Policy.

A policy in Draft state is not available for general use. Once you activate the policy, it is in Active state and is available for use.

Attaching a policy

To use the HTTP Message Validation Policy, go to the Policies folder in the respective organization and attach the policy to a web service, binding, or binding operation.

Using regular expressions

If you're using regular expressions, remember that the JSON Schema Validation specification (https://tools.ietf.org/html/draft-wright-json-schema-validation-01) includes the following stipulations regarding use of regular expressions:

  • It supports a specific version of regular expression syntax, ECMA 262 (see the ECMA 262 specification).
  • There are certain specific limitations, as laid out in the standard.

For details, refer to the applicable section of the specification: section 3.3, regular expressions.

HTTP Message Validation Policy: examples

This section includes a couple of request and response examples, with explanations how the HTTP Message Validation Policy would work in these scenarios. It includes:

Sample request JSON

In the example below, from a Swagger definition, properties are defined for the request. The policy matches the incoming API request against this definition, and returns an error if any of the fields don't match the definition. In this example, the properties are named to indicate the type of acceptable input.

For example, the policy would reject a request to this API if a mandatory field is missing, if the specified data type doesn't match, or if a field value doesn't match specified constraints.

{
  "mandatoryString": "String",
  "mandatoryDate": "2018-07-04T11:53:00.123Z",
  "mandatoryBoolean": true,
  "mandatoryNumber": 12345,
  "mandatoryInteger": 123,
  "mandatoryNull": null,
  "mandatoryEnum": "one",
  "optionalString": "String",
  "optionalDate": "2018-07-04T11:53:00.456Z",
  "optionalBoolean": true,
  "optionalNumber": 12345,
  "optionalInteger": 123,
  "optionalNull": null,
  "optionalRestrictedLengthString": "abcdefgh",
  "optionalMultipleOf": 20,
  "optionalInclusiveNumberRange": 10,
  "optionalExclusiveNumberRange": 99,
  "optionalAllOf": 30,
  "optionalArray": ["string1","string2","string3"],
  "optionalObject":{
  "objectProperty1": "one",
  "objectProperty2": 2,
  "objectProperty3": 3
  },
  "optionalContainsArray": ["string1","string2","2018-07-04T11:53:00.123Z"],
  "optionalIP": "256.123.123.123",
  "optionalUKPostcode": "W3 1WW",
  "optionalZipcode": "90210",
  "optionalEmail": "me@me.com",
  "anotherProperty":"value"
}

Sample success response

In the above example, a request that matches the API definition would pass validation, and the response below is returned from the back-end API.

{
  "message": "Request validated successfully. Timestamp Thu Jul 05 2018 05:34:24 GMT-0700 (PDT)"
}

Sample error response

In the example below, the API definition for a PUT operation included validation that the body exists. When an API call is sent with no message body, this error response is returned.

{
  "errorCount": 1,
  "errors": [
    {
      "seq": 1,
      "errorCode": "base.missing.instance",
      "errorMsg": "Required message part is missing",
      "path": "body:body#"
    }
  ]
}