Skill Tracer
Skills › Scientific Agent Skills › usfiscaldata › response-format.md

response-format.md

184 lines · skills/usfiscaldata/references/response-format.md at 5f2bc03

Open on GitHub
1# Response Format — U.S. Treasury Fiscal Data API
2

Response Format — U.S. Treasury Fiscal Data API

Raw lines 1 to 2
3## Response Structure (JSON)
4
5Illustrative shape, not an observed debt value or a complete 100-row page.
6
7```json
8{
9 "data": [
10 {
11 "record_date": "2024-03-31",
12 "tot_pub_debt_out_amt": "34589629941.12"
13 }
14 ],
15 "meta": {
16 "count": 100,
17 "labels": {
18 "record_date": "Record Date",
19 "tot_pub_debt_out_amt": "Total Public Debt Outstanding"
20 },
21 "dataTypes": {
22 "record_date": "DATE",
23 "tot_pub_debt_out_amt": "CURRENCY"
24 },
25 "dataFormats": {
26 "record_date": "YYYY-MM-DD",
27 "tot_pub_debt_out_amt": "10.2"
28 },
29 "total-count": 3790,
30 "total-pages": 38
31 },
32 "links": {
33 "self": "&page%5Bnumber%5D=1&page%5Bsize%5D=100",
34 "first": "&page%5Bnumber%5D=1&page%5Bsize%5D=100",
35 "prev": null,
36 "next": "&page%5Bnumber%5D=2&page%5Bsize%5D=100",
37 "last": "&page%5Bnumber%5D=38&page%5Bsize%5D=100"
38 }
40```

Response Structure (JSON)

Illustrative shape, not an observed debt value or a complete 100-row page.

{
  "data": [
    {
      "record_date": "2024-03-31",
      "tot_pub_debt_out_amt": "34589629941.12"
    }
  ],
  "meta": {
    "count": 100,
    "labels": {
      "record_date": "Record Date",
      "tot_pub_debt_out_amt": "Total Public Debt Outstanding"
    },
    "dataTypes": {
      "record_date": "DATE",
      "tot_pub_debt_out_amt": "CURRENCY"
    },
    "dataFormats": {
      "record_date": "YYYY-MM-DD",
      "tot_pub_debt_out_amt": "10.2"
    },
    "total-count": 3790,
    "total-pages": 38
  },
  "links": {
    "self": "&page%5Bnumber%5D=1&page%5Bsize%5D=100",
    "first": "&page%5Bnumber%5D=1&page%5Bsize%5D=100",
    "prev": null,
    "next": "&page%5Bnumber%5D=2&page%5Bsize%5D=100",
    "last": "&page%5Bnumber%5D=38&page%5Bsize%5D=100"
  }
}
Raw lines 3 to 41
42## `meta` Object
44| Field | Description |
45|-------|-------------|
46| `count` | Number of records in this response page |
47| `total-count` | Total records matching the query (all pages) |
48| `total-pages` | Total pages available at current page size |
49| `labels` | Human-readable column labels |
50| `dataTypes` | Logical types include `STRING`, `NUMBER`, `DATE`, `INTEGER`, `PERCENTAGE`, `CURRENCY` and precision variants such as `CURRENCY0`; calendar parts also use `YEAR`, `QUARTER`, `MONTH`, `DAY` |
51| `dataFormats` | Format hints: `YYYY-MM-DD`, `10.2`, `$10.20`, `String` |

meta Object

FieldDescription
countNumber of records in this response page
total-countTotal records matching the query (all pages)
total-pagesTotal pages available at current page size
labelsHuman-readable column labels
dataTypesLogical types include STRING, NUMBER, DATE, INTEGER, PERCENTAGE, CURRENCY and precision variants such as CURRENCY0; calendar parts also use YEAR, QUARTER, MONTH, DAY
dataFormatsFormat hints: YYYY-MM-DD, 10.2, $10.20, String
Raw lines 42 to 52
53## `links` Object
55Use the `links` object to navigate pagination programmatically:
57| Field | Value |
58|-------|-------|
59| `self` | Current page query params |
60| `first` | First page |
61| `prev` | Previous page (null if on first page) |
62| `next` | Next page (null if on last page) |
63| `last` | Last page |
65The HTTP response also includes a **`Link` header** with RFC 5988 relations (`rel="first"`, `rel="prev"`, `rel="next"`, `rel="last"`). JSON link values can be `&page...` fragments. Preserve all original filters, fields and sort when updating pagination; a fragment alone is not a complete URL.

links Object

Use the links object to navigate pagination programmatically:

FieldValue
selfCurrent page query params
firstFirst page
prevPrevious page (null if on first page)
nextNext page (null if on last page)
lastLast page

The HTTP response also includes a **Link header** with RFC 5988 relations (rel="first", rel="prev", rel="next", rel="last"). JSON link values can be &page... fragments. Preserve all original filters, fields and sort when updating pagination; a fragment alone is not a complete URL.

Raw lines 53 to 66
67## `data` Object
69Array of row objects. All values are **strings**, regardless of logical type.

data Object

Array of row objects. All values are strings, regardless of logical type.

Raw lines 67 to 70
71## Response Codes
73| Code | Meaning |
74|------|---------|
75| 200 | OK — successful GET |
76| 400 | Bad Request — malformed URL or invalid parameter |
77| 403 | Access denied; public API requests do not require keys |
78| 404 | Not Found — endpoint does not exist |
79| 429 | Too Many Requests — rate limited |
80| 500 | Internal Server Error |

Response Codes

CodeMeaning
200OK — successful GET
400Bad Request — malformed URL or invalid parameter
403Access denied; public API requests do not require keys
404Not Found — endpoint does not exist
429Too Many Requests — rate limited
500Internal Server Error
Raw lines 71 to 81
82## Error Object
84An invalid field produced HTTP 400 with this JSON error in the live review:
86```json
88 "error": "Invalid Query Param",
89 "message": "Invalid query parameter: Field 'not_a_column' does not exist. For more information, please see the documentation."
91```
93Unknown parameter names can be silently ignored: live `sorts=-record_date` returned
94HTTP 200 with the default ascending order, despite the official guide using that
95typo in its error example. Validate parameter names locally and check returned
96ordering/filters. The helpers in this skill reject unknown parameter names.
98Treat errors as errors even if a proxy sends HTML instead of JSON. Check status
99before parsing a successful response; on failure, inspect a bounded response body
100and any JSON `error`/`message` fields. Do not turn an error into an empty DataFrame.

Error Object

An invalid field produced HTTP 400 with this JSON error in the live review:

{
  "error": "Invalid Query Param",
  "message": "Invalid query parameter: Field 'not_a_column' does not exist. For more information, please see the documentation."
}

Unknown parameter names can be silently ignored: live sorts=-record_date returned HTTP 200 with the default ascending order, despite the official guide using that typo in its error example. Validate parameter names locally and check returned ordering/filters. The helpers in this skill reject unknown parameter names.

Treat errors as errors even if a proxy sends HTML instead of JSON. Check status before parsing a successful response; on failure, inspect a bounded response body and any JSON error/message fields. Do not turn an error into an empty DataFrame.

Raw lines 82 to 101
102## Common Error Causes
104- Invalid field name in `fields=` parameter
105- Invalid filter operator (use `eq`, `gte`, `lte`, `gt`, `lt`, `in`)
106- Wrong date format (must be `YYYY-MM-DD`)
107- Accessing a v2 endpoint with `/v1/` in the URL
108- `sort` field not available in the endpoint

Common Error Causes

  • Invalid field name in fields= parameter
  • Invalid filter operator (use eq, gte, lte, gt, lt, in)
  • Wrong date format (must be YYYY-MM-DD)
  • Accessing a v2 endpoint with /v1/ in the URL
  • sort field not available in the endpoint
Raw lines 102 to 109
110## Parsing Responses
112```python
113import requests
114import pandas as pd
116def api_to_dataframe(endpoint, params=None):
117 """Fetch ONE PAGE and return a DataFrame plus pagination metadata."""
118 base = "https://api.fiscaldata.treasury.gov/services/api/fiscal_service"
119 resp = requests.get(f"{base}{endpoint}", params=params, timeout=30)
120 resp.raise_for_status()
121 result = resp.json()
123 df = pd.DataFrame(result["data"])
124 meta = result["meta"]
126 # Apply type conversions using metadata
127 for col, dtype in meta["dataTypes"].items():
128 if col not in df.columns:
129 continue
130 if dtype in ("NUMBER", "PERCENTAGE") or dtype.startswith("CURRENCY"):
131 df[col] = pd.to_numeric(df[col].replace("null", None), errors="coerce")
132 elif dtype == "DATE":
133 df[col] = pd.to_datetime(df[col].replace("null", None), errors="coerce")
134 elif dtype == "INTEGER":
135 df[col] = pd.to_numeric(df[col].replace("null", None), errors="coerce").astype("Int64")
137 return df, meta
139# Usage
140df, meta = api_to_dataframe(
141 "/v2/accounting/od/debt_to_penny",
142 params={"sort": "-record_date", "page[size]": 30}
144print(f"Total records available: {meta['total-count']}")
145print(df[["record_date", "tot_pub_debt_out_amt"]].head())
146```
148Numeric pandas conversion can lose cent precision for large debt amounts. Keep raw
149strings and use `decimal.Decimal` for accounting reconciliation. Calendar-part
150types may be retained as strings to preserve zero padding. Units must come from
151the field dictionary; a `CURRENCY` label alone does not distinguish USD from
152millions of USD.

Parsing Responses

import requests
import pandas as pd

def api_to_dataframe(endpoint, params=None):
    """Fetch ONE PAGE and return a DataFrame plus pagination metadata."""
    base = "https://api.fiscaldata.treasury.gov/services/api/fiscal_service"
    resp = requests.get(f"{base}{endpoint}", params=params, timeout=30)
    resp.raise_for_status()
    result = resp.json()
    
    df = pd.DataFrame(result["data"])
    meta = result["meta"]
    
    # Apply type conversions using metadata
    for col, dtype in meta["dataTypes"].items():
        if col not in df.columns:
            continue
        if dtype in ("NUMBER", "PERCENTAGE") or dtype.startswith("CURRENCY"):
            df[col] = pd.to_numeric(df[col].replace("null", None), errors="coerce")
        elif dtype == "DATE":
            df[col] = pd.to_datetime(df[col].replace("null", None), errors="coerce")
        elif dtype == "INTEGER":
            df[col] = pd.to_numeric(df[col].replace("null", None), errors="coerce").astype("Int64")
    
    return df, meta

# Usage
df, meta = api_to_dataframe(
    "/v2/accounting/od/debt_to_penny",
    params={"sort": "-record_date", "page[size]": 30}
)
print(f"Total records available: {meta['total-count']}")
print(df[["record_date", "tot_pub_debt_out_amt"]].head())

Numeric pandas conversion can lose cent precision for large debt amounts. Keep raw strings and use decimal.Decimal for accounting reconciliation. Calendar-part types may be retained as strings to preserve zero padding. Units must come from the field dictionary; a CURRENCY label alone does not distinguish USD from millions of USD.

Raw lines 110 to 153
154## CSV Format Response
156When `format=csv` is specified, the response body is plain CSV text (not JSON):
158```python
159import io
161resp = requests.get(
162 "https://api.fiscaldata.treasury.gov/services/api/fiscal_service/v2/accounting/od/debt_to_penny",
163 params={"format": "csv", "sort": "-record_date", "page[size]": 100},
164 timeout=30,
166resp.raise_for_status()
167df = pd.read_csv(io.StringIO(resp.text))
168```

CSV Format Response

When format=csv is specified, the response body is plain CSV text (not JSON):

import io

resp = requests.get(
    "https://api.fiscaldata.treasury.gov/services/api/fiscal_service/v2/accounting/od/debt_to_penny",
    params={"format": "csv", "sort": "-record_date", "page[size]": 100},
    timeout=30,
)
resp.raise_for_status()
df = pd.read_csv(io.StringIO(resp.text))
Raw lines 154 to 169
170## XML Format Response
172When `format=xml` is specified, the response body is XML:
174```python
175import xml.etree.ElementTree as ET
177resp = requests.get(
178 "https://api.fiscaldata.treasury.gov/services/api/fiscal_service/v2/accounting/od/debt_to_penny",
179 params={"format": "xml", "page[size]": 10},
180 timeout=30,
182resp.raise_for_status()
183root = ET.fromstring(resp.text)
184```

XML Format Response

When format=xml is specified, the response body is XML:

import xml.etree.ElementTree as ET

resp = requests.get(
    "https://api.fiscaldata.treasury.gov/services/api/fiscal_service/v2/accounting/od/debt_to_penny",
    params={"format": "xml", "page[size]": 10},
    timeout=30,
)
resp.raise_for_status()
root = ET.fromstring(resp.text)
Raw lines 170 to 184