Http Api Test Runner

Build reusable HTTP API test artifacts from user-provided endpoints, authentication, request data, expected results, and validation rules. Use this skill when the user wants to create .http files, run HTTP/REST API checks, replay browser or curl requests, validate JSON fields or response markers, compare expected vs actual responses, or generate formatted PASS/FAIL API test reports.

Use this skill to turn ad hoc HTTP endpoint checks into repeatable .http cases and a runnable formatter script.

The user provides endpoints, authentication, request data, scenarios, and expected results. Produce artifacts that can be rerun and that print a readable PASS/FAIL report instead of raw response dumps.

When To Use

Use this skill when the user asks to:

• Write .http tests for an HTTP/REST API.
• Replay browser Network requests or cURL commands.
• Test endpoints with cookies, bearer tokens, custom headers, request bodies, or no authentication.
• Validate status codes, JSON paths, response markers, list membership, error codes, or login behavior.
• Run several API cases and summarize which cases passed or failed.
• Create a formatted script for API response inspection.

Collect Inputs

Ask only for missing information. Do not ask for values already provided.

Collect:

• Base URL / host: for example https://api.example.test.
• Endpoints: full URLs or path templates.
• HTTP method: default to GET when unspecified.
• Authentication: cookie, bearer token, custom headers, or unauthenticated.
• Request data: query params, path params, JSON body, form body, content type.
• Cases: positive, negative, auth failure, validation failure, boundary cases.
• Expected results: exact status code, JSON field, marker, list entry, or error behavior.
• Output needs: short summary, detailed per-case report, key fields, raw response save path.

For cookie-based tests, tell the user to copy the complete request cookie from a successful browser Network request:

Please copy the full `Cookie:` request header from the browser Network panel for a successful target request. Do not reconstruct cookies only from the Application/Storage panel, because HttpOnly, path, domain, and host-bound cookies may be missed.

Use the user's language for generated file comments and final instructions.

Artifact Pattern

Create two files by default, near the user's existing test or technical docs unless they specify another location:

<feature>.api-tests.http
<feature>.api-verify.sh

If the project already uses another naming style, follow it.

The .http file is the source of truth. It should contain:

• Variables such as @host, @resourceId, @cookie, @token.
• One request per case.
• Human-readable case titles using ###.
• Explicit expect.* comments for assertions.
• No real secrets by default.

The formatter script should:

• Read the .http file.
• Resolve variables such as {{host}}, {{resourceId}}, {{cookie}}.
• Accept secrets from environment variables.
• Execute requests with timeout handling.
• Parse JSON responses when possible.
• Print formatted PASS/FAIL/SKIP output.
• Exit non-zero if any non-skipped case fails.

Prefer bash + curl for portability. For JSON parsing, prefer python3 embedded snippets over mandatory jq, unless the repo already depends on jq.

Secret Handling

Default behavior:

• Do not write real cookies, service tokens, bearer tokens, session IDs, user IDs, or internal credentials into skill files or generated committed artifacts.
• Use placeholders in .http, such as @cookie = <set via COOKIE> or @token = <set via AUTH_TOKEN>.
• Read secrets from environment variables in scripts, such as COOKIE, AUTH_TOKEN, ADMIN_COOKIE.
• If secrets are missing, skip authenticated cases with a clear [SKIP] message instead of failing parsing.

If the user explicitly asks to embed cookies or tokens for local convenience:

• Do it only in a local test script, never in publishable docs.
• Add a warning near the top of the file:

This script temporarily embeds real credentials for local testing only. Do not commit, share, or publish it.

Before publishing or committing, check for secrets and internal identifiers with searches such as:

rg -n "password|secret|session_id|auth_token|access_token|refresh_token" <artifact-dir>
rg -n "Authorization: Bearer [A-Za-z0-9._-]+|C[o]okie: [A-Za-z0-9_%-]+=" <artifact-dir>

Assertion Contract

Prefer explicit assertions in .http comments. Title-based inference is allowed only as a fallback.

Supported assertion comments:

# expect.status = 200
# expect.errno = 0
# expect.errno_not = 0
# expect.contains = TARGET_MARKER
# expect.not_contains = ERROR_MARKER
# expect.json_path = data.status
# expect.equals = enabled
# expect.exists = data.items[0].id
# expect.absent = data.deprecatedField
# expect.list_contains = data.items[].code:TARGET_CODE
# expect.list_not_contains = data.items[].code:TARGET_CODE
# expect.save = tmp/case-1.response.json
# output.fields = code,message,data.id,data.status

Rules:

• expect.status validates HTTP status.
• expect.errno and expect.errno_not apply to APIs that expose an errno-style field.
• expect.contains and expect.not_contains search the response text.
• expect.json_path + expect.equals validates one JSON field.
• expect.exists and expect.absent validate JSON path presence.
• expect.list_contains validates a projected list path contains a value.
• expect.save optionally saves raw response for debugging.
• output.fields controls key fields printed in the formatted report.

If an API uses code, status, error, or another convention instead of errno, adapt the generated assertions and output names to the API.

.http Template

Use this structure for common GET and POST cases:

### <feature> API tests
#
# Usage:
# 1. Confirm the target environment has the code/config under test.
# 2. Export secrets when needed:
#    COOKIE='full Cookie header' AUTH_TOKEN='token value' bash './<feature>.api-verify.sh'
# 3. Run the formatter script to see PASS/FAIL output.

@host = https://api.example.test
@resourceId = sample-123
@cookie = <set via COOKIE>
@token = <set via AUTH_TOKEN>

### Case-1 GET positive: resource should be enabled
# auth = cookie
# expect.status = 200
# expect.json_path = data.status
# expect.equals = enabled
# output.fields = code,message,data.id,data.status
GET {{host}}/v1/resources/{{resourceId}}
Cookie: {{cookie}}

### Case-2 GET negative: missing resource should return error
# expect.status = 404
# expect.contains = not_found
GET {{host}}/v1/resources/missing-id

### Case-3 POST positive: create should return success
# auth = bearer
# expect.status = 200
# expect.json_path = code
# expect.equals = 0
POST {{host}}/v1/resources
Content-Type: application/json
Authorization: Bearer {{token}}

{
  "name": "sample"
}

Script Output

The formatter script should print enough information to diagnose failures without opening raw JSON:

[PASS] Case-1 GET positive: resource should be enabled
  URL: https://api.example.test/v1/resources/sample-123
  method: GET
  expectation: json_path data.status == enabled
  HTTP=200 cost=123ms
  key fields: code=0,message='',data.id='sample-123',data.status='enabled'

[FAIL] Case-2 GET negative: missing resource should return error
  URL: https://api.example.test/v1/resources/missing-id
  method: GET
  expectation: contains not_found
  HTTP=200 cost=98ms
  reason: expected HTTP status 404 but got 200

Summary: PASS=1 FAIL=1 SKIP=0

For non-JSON responses, print status, headers or content type when useful, response size, and text marker results.

Implementation Guidance

When generating the formatter script:

• Keep case definitions synchronized with the .http file.
• Support GET, POST, PUT, PATCH, and DELETE when present.
• Preserve request headers from the .http case.
• Preserve JSON or form bodies exactly enough for repeatable testing.
• Use curl --silent --show-error --location --max-time 20.
• Separate transport failures, HTTP status failures, JSON parse failures, and assertion failures.
• Print the case containing a matching marker or failed list element when validating list responses.
• Save raw responses only when requested or helpful for debugging.

Running Tests

After creating artifacts:

1. Syntax-check the script:

bash -n './<feature>.api-verify.sh'

2. Run without secrets to verify parsing and expected SKIP behavior:

bash './<feature>.api-verify.sh'

3. Run with user-provided secrets:

COOKIE='full Cookie header' AUTH_TOKEN='token value' bash './<feature>.api-verify.sh'

4. Summarize:

• Which cases passed, failed, or skipped.
• Whether failures look like auth, request shape, environment config, data setup, or code behavior.
• Which fields or raw response files are most relevant.

Debugging Rules

If authenticated cases fail or return login errors:

• Ask for the full Cookie: or Authorization: request header from a successful Network request.
• Check host, path, and environment mismatch.
• Check missing app-specific auth headers or CSRF headers.

If a positive case does not contain the expected marker:

• Print the exact assertion and the inspected JSON path or response section.
• Re-check feature switch/config, user/account conditions, region, state, type, and time-window requirements.

If list validation appears to pass but the displayed item looks unrelated:

• Display the exact list item that matched.
• Prefer explicit JSON path assertions over broad text search when the response structure is stable.

Completion Checklist

Before finishing:

• .http and script contain the same cases.
• Authenticated cases skip cleanly when secrets are missing.
• Script runs with provided secrets and produces formatted PASS/FAIL output when the network is reachable.
• Raw secrets are not embedded unless the user explicitly requested local-only embedding.
• Publishable skill files contain no internal hostnames, real identifiers, company-specific markers, or credentials.
• Final response tells the user how to run the script and how to interpret the output.