Asynchronous APIs

In this quickstart, you'll learn how to use Hookdeck as your asynchronous API infrastructure, proxying asynchronous API calls through to your internal API. You'll achieve this by creating a Hookdeck Source with configured authentication to consume asynchronous API requests — a request where the client just needs to know the request has been accepted — at scale from many API clients and deliver those API requests events to your own infrastructure via Hookdeck.

Asynchronous API infrastructure

You will walk through these steps:

  1. Ensure you have some prerequisites in place before you begin
  2. Set up a Connection within Hookdeck, including a Source configured with authentication
  3. Receive test asynchronous API requests on Hookdeck, delivering the events to a Hookdeck Mock API Destination
  4. Use the Hookdeck CLI to receive the API requests on your local development server via Hookdeck
  5. Update the destination configuration to deliver API requests to your production environment

1. Before you begin

Sign up for a Hookdeck account and create a Hookdeck Organization.

Get the API key from your project secrets if you prefer to use the Hookdeck from the terminal via cURL requests.

Install the Hookdeck CLI and run hookdeck login.

2. Create a connection

A Connection consists of a Source, optional Rules, and a Destination. It defines the flow of the API request event, from receipt through to delivery.

Run the following command to create a Connection within your Hookdeck project:

curl --location 'https://api.hookdeck.com/2023-07-01/connections' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer {YOUR_PROJECT_API_KEY}' \
--data '{
  "name": "asynchronous-api-quickstart",
  "source": {
    "name": "inbound-api",
    "verification": {
      "type": "api_key",
      "configs": {
        "header_key": "X-API-KEY",
        "api_key": "{UNIQUE_CLIENT_API_KEY}"
      }
    }
  },
  "destination": {
    "name": "internal-api",
    "url": "https://mock.hookdeck.com"
  }
}'

The cURL command creates a Connection named asynchronous-api-quickstart, a Source with the name inbound-api, and a Destination with the name internal-api and url with the value https://mock.hookdeck.com.

The source is configured to use the API key verification method with the header_key set to X-API-KEY and the api_key set to {UNIQUE_CLIENT_API_KEY}. Replace {UNIQUE_CLIENT_API_KEY} with a unique value for your API client.

Hookdeck supports other forms of source authentication.

The response will be formatted as follows:

{
  "id": "web_2B4zoMEJZ1Gu",
  "team_id": "tm_EZobjDgZytYu",
  "disabled_at": null,
  "updated_at": "2024-01-15T16:35:30.667Z",
  "created_at": "2024-01-15T16:35:30.880Z",
  "paused_at": null,
  "name": "asynchronous-api-quickstart",
  "rules": [],
  "description": null,
  "destination": {
    "id": "des_AQmhcxuKaaMF",
    "team_id": "tm_EZobjDgZytYu",
    "url": "https://mock.hookdeck.com",
    "disabled_at": null,
    "updated_at": "2024-01-15T16:35:30.667Z",
    "created_at": "2024-01-15T16:35:30.671Z",
    "rate_limit": null,
    "rate_limit_period": "second",
    "cli_path": null,
    "path_forwarding_disabled": false,
    "name": "internal-api",
    "http_method": null,
    "auth_method": {
      "type": "HOOKDECK_SIGNATURE",
      "config": {}
    },
    "description": null
  },
  "source": {
    "id": "src_93mdm4h47f7zqr",
    "team_id": "tm_EZobjDgZytYu",
    "disabled_at": null,
    "updated_at": "2024-01-15T16:35:30.667Z",
    "created_at": "2024-01-15T16:35:30.680Z",
    "name": "inbound-api",
    "allowed_http_methods": [
      "POST",
      "PUT",
      "PATCH",
      "DELETE"
    ],
    "custom_response": null,
    "description": null,
    "url": "https://hkdk.events/93mdm4h47f7zqr",
    "verification": {
      "type": "api_key"
    }
  },
  "full_name": "inbound-api -> asynchronous-api-quickstart"
}

The cURL command has created a connection. The source.url value is the URL used to receive the asynchronous API requests.

The destination.url value is the URL where Hookdeck will deliver the API request event payloads to. In the cURL request, you set the Destination URL to https://mock.hookdeck.com, which is a destination you can use for testing that receives the HTTP request and returns a 200 response.

The source.verification is confirmed to be of type api_key, but the credentials are not shown in the response.

From the Connections section of the Hookdeck dashboard, click the + Connection button.

In the Create Connection page, under Configure a Source, enter inbound-api.

Under Configure a Destination enter the Name internal-api and set the Destination Type to Mock API.

Under Advanced Source Configuration enable the Source Authentication toggle, select API Key as the authentication method, enter X-API-KEY as the API Key Header, and choose a value for the API Key.

In this quickstart, the API Key value is referred to as {UNIQUE_CLIENT_API_KEY}.

Click the + Create button to create the Connection and close the Connection Created dialog that appears by clicking the Close button.

Open the Connections section for your project in the Hookdeck dashboard to see the visual representation of the connection you created.

Quickstart asynchronous API connection within the Hookdeck dashboard

To support multiple authenticated API clients, create a new Source, and reuse the Destination.

3. Make an asynchronous API request to your Hookdeck Source

Now it's time to simulate an API client.

Make a POST request to the Source URL from the previous section, using your configured API key. Replace {YOUR_SOURCE_URL} and {{UNIQUE_CLIENT_API_KEY}} in the following cURL command:

curl --location '{YOUR_SOURCE_URL}/track' \
--header 'Content-Type: application/json' \
--header 'X-API-KEY: {{UNIQUE_CLIENT_API_KEY}}' \
--data-raw '{
    "event_type": "mouse.click",
    "click_x": "339",
    "click_y": "318",
    "timestamp": "2024-01-15T22:39:29.964Z"
}'

The response will look similar to the following:

{
  "status": "SUCCESS",
  "message": "Request successfully handled. Request ID: req_pSO2eEXILyN4CxxphL3I",
  "request_id": "req_pSO2eEXILyN4CxxphL3I"
}

The status with a value of SUCCESS indicates that Hookdeck has successfully ingested the HTTP request.

Hookdeck supports returning custom HTTP responses to inbound requests.

Open the Requests section of Hookdeck dashboard to see the request within the UI.

Requests within the Hookdeck Dashboard

Next, open the Events section of the Hookeck dashboard. Click on the Event to expand the section to see the Attempts made to deliver the event to a Destination URL.

Events within the Hookdeck Dashboard

Notice that the /track Path is preserved in the URL. Path forwarding is an advanced configuration option for a Source and is enabled by default. This feature enables you to make requests to any path on your API without any configuration changes within Hookdeck.

4. Receive an asynchronous API request on your localhost using the Hookdeck CLI

To receive an asynchronous API request locally, first start your local development server:

Run the following to get the code and install the app's dependencies:

npx degit hookdeck/quickstarts/typescript/inbound hookdeck-inbound
cd hookdeck-inbound
npm i

Then, run the application:

npm run dev

You will see output as follows:

> inbound@1.0.0 dev
> nodemon index.ts

[nodemon] 3.0.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: ts,json
[nodemon] starting `ts-node index.ts`
🪝 Server running at http://localhost:3030

Get the Hookdeck quickstart code:

git clone https://github.com/hookdeck/quickstarts.git
cd quickstarts/python/inbound

Create a new virtualenv, activate it, and Install the application dependencies:

virtualenv venv
source venv/bin/activate
pip install -r requirements.txt

Run the app:

python app.py

The output will look as follows:

python app.py
 * Serving Flask app 'app'
 * Debug mode: on
INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
INFO:werkzeug:Press CTRL+C to quit
INFO:werkzeug: * Restarting with stat
WARNING:werkzeug: * Debugger is active!
INFO:werkzeug: * Debugger PIN: 530-477-256

Get the Hookdeck quickstart code:

git clone https://github.com/hookdeck/quickstarts.git
cd quickstarts/go/inbound

Run the app:

go run .

The output will look as follows:

go run .
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /*path                  --> main.main.func1 (3 handlers)
🪝 Server running at http://localhost:3030[GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.
Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.
[GIN-debug] Listening and serving HTTP on :3030

Next, use the Hookdeck CLI to create a connection to route events from the inbound-api Source to the local development server. For the next step, you'll need to know the port your dev server is listening on.

Assuming your dev server is running on port 3030, run the following command:

hookdeck listen 3030 inbound-api

When prompted for a path, enter /.

When prompted for a label, enter local.

You will see a command output similar to the following:

? What path should the webhooks be forwarded to (ie: /webhooks)? /
? What's your connection label (ie: My API)? local

Dashboard
👉 Inspect and replay webhooks: https://dashboard.hookdeck.com?team_id=tm_SAN9hnAiGLbI

inbound-api Source
🔌 Webhook URL: {YOUR_SOURCE_URL}

Connections
local forwarding to /

> Ready! (^C to quit)

Open the Connections section for your project in the Hookdeck dashboard to see the Hookdeck CLI within the visual representation of the connection.

CLI within a connection in the Hookdeck dashboard

With your dev server running and the Hookdeck CLI set up to listen to events from the inbound-api source, you can test receiving API requests to your localhost:

curl --location '{YOUR_SOURCE_URL}/track' \
--header 'Content-Type: application/json' \
--header 'X-API-KEY: {{UNIQUE_CLIENT_API_KEY}}' \
--data-raw '{
    "event_type": "mouse.click",
    "click_x": "339",
    "click_y": "318",
    "timestamp": "2024-01-15T22:39:29.964Z"
}'

After running the cURL command, you will see the request logged by the Hookdeck CLI:

2023-11-19 20:36:53 [200] POST http://localhost:3030/track | https://dashboard.hookdeck.com/cli/events/evt_YhWPqX1QNKZrrrcybI

You'll also see the request logged by your development server:

{
  webhook_received: '2024-01-16T11:59:19.515Z',
  body: {
    event_type: 'mouse.click',
    click_x: '339',
    click_y: '318',
    timestamp: '2024-01-15T22:39:29.964Z'
  }
}

INFO:app:webhook_received 2023-11-19T20:40:29.767Z {
  event_type: 'mouse.click',
  click_x: '339',
  click_y: '318',
  timestamp: '2024-01-15T22:39:29.964Z'
}
INFO:werkzeug:127.0.0.1 - - [19/Nov/2023 20:36:53] "POST /track HTTP/1.1" 200 -

2024/01/16 12:03:09 webhook_received: {
  "event_type": "mouse.click",
  "click_x": "339",
  "click_y": "318",
  "timestamp": "2024-01-15T22:39:29.964Z"
}

5. Deliver asynchronous API requests to your production API

This step assumes you have an API deployed that you want to receive the proxied asynchronous API request.

Next, update the Destination URL for your asynchronous-api-quickstart to your production API base URL.

Run a PUT request to update your connection:

curl --location --request PUT 'https://api.hookdeck.com/2023-07-01/connections' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer {YOUR_PROJECT_API_KEY}' \
--data '{
  "name": "asynchronous-api-quickstart",
  "source": {
    "name": "inbound-api"
  },
  "destination": {
    "name": "internal-api",
    "url": "<YOUR_API_URL>"
  }
}'

You will see a response similar to the following:

{
  "id": "web_oPymMJ0isyuS",
  "team_id": "tm_SAN9hnAiGLbI",
  "disabled_at": null,
  "updated_at": "2023-11-20T12:42:56.433Z",
  "created_at": "2023-11-18T16:52:44.096Z",
  "paused_at": null,
  "name": "asynchronous-api-quickstart",
  "rules": [],
  "description": null,
  "destination": {
    "id": "des_wNYxQXpkYHcM",
    "team_id": "tm_SAN9hnAiGLbI",
    "url": "https://api.example.com/",
    "disabled_at": null,
    "updated_at": "2023-11-20T12:42:56.250Z",
    "created_at": "2023-11-18T16:52:43.903Z",
    "rate_limit": null,
    "rate_limit_period": "second",
    "cli_path": null,
    "path_forwarding_disabled": false,
    "name": "internal-api",
    "http_method": null,
    "auth_method": {
      "type": "HOOKDECK_SIGNATURE",
      "config": {}
    },
    "description": null
  },
  "source": {
    "id": "src_KUdNgMByl0uP",
    "team_id": "tm_SAN9hnAiGLbI",
    "disabled_at": null,
    "updated_at": "2023-11-20T12:42:56.252Z",
    "created_at": "2023-11-18T16:52:43.905Z",
    "name": "inbound-api",
    "allowed_http_methods": ["POST", "PUT", "PATCH", "DELETE"],
    "custom_response": null,
    "description": null,
    "url": "{YOUR_SOURCE_URL}",
    "verification": {
      "type": "api_key"
    }
  },
  "full_name": "inbound-api -> asynchronous-api-quickstart"
}

From the Connection section, click on the internal-api Destination. Change the Destination Type to HTTP and update the URL to reference your production API base URL and click Save .

Connections -> Update Destination URL in Hookdeck Dashboard

With the connection updated, you can now use Hookdeck as your asynchronous API infrastructure, with all asynchronous API requests ingested and proxied to your deployed production API.

Make an API request to the Hookdeck {YOUR_SOURCE_URL} and you will receive the API request on your own API production.

As before, you can check this in both the Hookdeck dashboard Requests and Events sections.

What you have learned

By following this quickstart, you set up a Connection with Hookdeck and tested receiving and authenticating asynchronous API requests with Hookdeck, with API request events delivered to Hookdeck's Mock API endpoint. You then ran a local development server and used the Hookdeck CLI to receive API requests in your local environment. Finally, you updated the Connection to deliver asynchronous API requests to a live deployed API.

Where next?

You've just scratched the surface of using Hookdeck as your asynchronous API infrastructure. So, try out the following features:

  • Set up a rate limit on your Destination to manage the number of requests your API has to manage per second.
  • Securely receive API requests in your own infrastructure by setting up Hookdeck webhook signature verification.
  • Define your retry logic for those occasions when your deployed API isn't available or can't accept an API request event.
  • Add support for multiple authenticated API clients by creating a new Source for each API client, using unique API keys, but reusing the destination.
  • Follow the quickstart for sending webhooks, and add callback support to your asynchronous APIs.
  • Add a new connection that uses the same Source, but use a Filter to route requests to a different Destination
  • Add a transformation to ensure all payloads confirm to a standard format for your API.