AWS API Gateway – outbound proxy passthrough to external API/s for logging and observability and traceability

When your application platform enages with external APIs (microservices) it can be useful to proxy this access via AWS API Gateway to provide logging and observability capabilities on these requests. The following outlines the approach at a high level, with reference to implementation using AWS Cloudformation for your ‘infrastructure as code’.

This is specifically for API requests to external API endpoints that are not hosted on your AWS environment/s.

Solution overview

Firstly a private rest API is defined on the AWS API Gateway [‘MyExternalAPI’]. The definition includes a policy allowing ‘Invoke’ permission for any request source within the VPC, a default root resource was automatically created for the AWS API Gateway in your AWS Account.

You create an AWS API Gateway Resource where the path is set to {proxy+}. This means that any path is considered valid by the API gateway. This resource definition is given below. In this code extract, ‘MyExternalAPI’ is the logical name of the rest API.

  MyExternalResource:
    Type: 'AWS::ApiGateway::Resource'
    Properties:
      RestApiId: !Ref MyExternalAPI
      ParentId: !GetAtt MyExternalAPI.RootResourceId
      PathPart: '{proxy+}'

Having defined all paths as valid, we then set all methods as valid. This is done by defining a root method and a proxy method. Both are set to use the ‘ANY’ method, meaning any of GET, POST, DELETE etc. The reason two methods need to be defined is to firstly allow the ‘ANY’ method to be accepted by the root resource of the API gateway and then also to allow the ‘ANY’ method to be accepted by the proxy resource. When defining the proxy method, it must be explicitly allowed to use the proxy path parameter received from the root resource (RequestParameters). Furthermore, when defining the integration of the method, method request parameters must also be mapped to those of the integration request (integration.request.path.proxy).

A good resource describing how to set up an AWS API Gateway Proxy using AWS CloudFormation can be found at this link.

The definition of the proxy method is shown below.

  ProxyMethod:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizationType: NONE
      HttpMethod: ANY
      RestApiId: !Ref MyExternalAPI
      ResourceId: !Ref MyExternalResource
      RequestParameters:
        method.request.path.proxy: true       # "The request parameters that API Gateway accepts." Accept everything, since this is a proxy.
      Integration:
        ConnectionType: INTERNET
        CacheKeyParameters:
          - 'method.request.path.proxy'
        RequestParameters:
          integration.request.path.proxy: 'method.request.path.proxy'     # "The request parameters that API Gateway sends with the backend request." Send everything from the method proxy to the integration proxy.
        IntegrationHttpMethod: ANY
        PassthroughBehavior: WHEN_NO_MATCH
        Type: HTTP_PROXY
        Uri:  !Join ['', [!Ref KNURL, '{proxy}']]
        IntegrationResponses:
          - StatusCode: 200

July 16, 2020