API Query Dataset

The API Query Dataset refers to converting an HTTP JSON API into a dataset through API queries.

Create Dataset

Please follow the steps below to create an API Query Dataset.

Creation Entry

On the Dataset interface, click on New Dataset and select API Query.

Supported Request Configurations

The request method supports GET or POST.

1. Request Header, by configuring the Header, you can pass custom HTTP request headers, such as commonly used Content-Type configured as application/json, or by configuring Authorization as Basic SOME-BASE64 to support some simple authentication functions.

2. Query Parameters, generally used to specify the query object or configure pagination. For example, some interfaces specify the campaign ID via ?compainId=123, and can also be combined with the pagination configuration below to achieve pagination functionality. For example, if an interface supports pagination via ?page=1, it can be combined with the pagination parameter PAGE specified in the pagination settings to achieve pagination, defining ?page={{$PAGE}}.

3. Request Body Definition, similar to query parameters, can also specify the request body that needs to be passed for the specific query.

4. Pagination Settings, supports two pagination methods: pagination by page number ?page=1&n=10, and pagination by offset offset=20&limit=10.

5. Authentication methods support Basic username and password authentication, or OAuth2 authorization code mode.

JSON Field Splitting

After the data acquisition configuration is completed, JSON parsing configuration is required. The syntax of JSON parsing is consistent with the syntax of the JSON splitting function, supporting splitting of the first-level JSON. Each split needs to have an alias, and aliases are not allowed to be duplicated. The split field types only support numbers, text, and time.

After the field splitting configuration is completed, you can preview the first page of the splitting results. If the splitting results do not meet expectations, you can return to the previous page to modify the field splitting configuration. If they meet expectations, you can directly save it as a dataset.

By default, the API dataset does not have an import engine. If you need to perform computational analysis, you can set the import engine during dataset creation or modify the dataset settings to configure the update plan.