Open Datasets & Machine-Readable Feeds
What datasets are available in the public index and download feeds?
Canonical open data downloads for financial researchers, econometricians, and algorithmic agents:
| Dataset Title | Data Grain | Coverage Window | Direct Download |
|---|---|---|---|
| US Congressional Stock Trading Disclosures | 1 row per transaction | 5 Years (2021–Present) | Download CSV |
| Congress Stock Trade Disclosure Latency | 1 row per disclosure + days-late | Rolling (most recent 5,000) | Download CSV |
| Federal Reserve & Treasury Net Liquidity Metrics | 1 row per settlement day | 2 Years (2024–Present) | Download CSV |
How do public federal intelligence datasets integrate into open data architectures?
Gemral Edge provides open, standardized, machine-readable data feeds synthesized from primary United States federal government records and statutory disclosures. Designed for quantitative researchers, financial data scientists, investigative journalists, and autonomous AI agents, our public datasets eliminate the operational overhead of scraping and cleaning complex government portals while maintaining strict primary-source provenance.
What are the four foundational federal datasets underlying the platform?
Our public repository currently publishes four core datasets updated on continuous rolling schedules: (1) Congressional Stock Trading Disclosures, capturing all equity, option, and asset transactions filed by sitting members of the U.S. House of Representatives and U.S. Senate under the Stop Trading on Congressional Knowledge Act of 2012 (STOCK Act); (2) Federal Defense and Technology Prime Contracts, tracking newly obligated procurement awards across the Department of Defense, DARPA, and federal agencies resolved to listed parent corporations; (3) Senate LDA Corporate Lobbying Disclosures, indexing corporate lobbying spend, registered lobbying firms, client entities, and targeted legislative issue codes; and (4) Macroeconomic Liquidity Metrics, calculating daily net liquidity flows derived from Federal Reserve total assets, the Treasury General Account (TGA), and the Overnight Reverse Repurchase Agreement facility (ON RRP).
How does the dual-licensing framework balance CC0 public domain and CC-BY attribution?
To maximize research freedom while preserving documentation integrity, Gemral Edge implements a transparent dual-licensing framework. All raw statutory records, transaction data points, and government filings originate from official public domain sources and are released under the Creative Commons CC0 1.0 Universal Public Domain Dedication. Users may copy, modify, distribute, and execute commercial or non-commercial analyses without requesting prior permission. All accompanying schemas, metadata descriptors, and analytical methodologies authored by Gemral Edge are licensed under Creative Commons Attribution 4.0 International (CC-BY-4.0). We request simple citation attribution: Data: Gemral Edge (https://www.gemral.com/edge/datasets).
How are programmatic CSV exports and agentic ingestion configured?
Every dataset is accessible via direct, permanent HTTP endpoints delivering RFC 4180 compliant UTF-8 CSV streams with full Cross-Origin Resource Sharing (CORS) headers. AI models, automated pipelines, and statistical environments such as Python Pandas or R can stream datasets directly without authentication barriers or rate-limiting friction.
What standardized column normalization standards ensure quantitative modeling compatibility?
Every dataset distributed across the Gemral Edge open repository adheres to strict schema normalization standards. All transaction records, procurement obligations, and lobbying filings feature ISO 8601 UTC timestamps, standardized ISO-3166 jurisdiction codes, verified Legal Entity Identifiers (LEI), and Commercial and Government Entity (CAGE) codes. Numerical values—including obligation amounts and transaction valuation brackets—are normalized into standard floating-point figures, allowing quantitative researchers to ingest clean tabular feeds directly into Pandas, NumPy, and Polars without custom data wrangling scripts.
How are cryptographic validation hashes and immutable changelogs maintained?
To maintain absolute scientific reproducibility, every released dataset is accompanied by a SHA-256 cryptographic checksum and a timestamped release manifest. When official government revisions occur, our data pipeline issues versioned snapshot archives, ensuring that backtesting researchers can accurately reproduce historical quantitative experiments without risk of look-ahead contamination or silent schema drift.
How do direct Parquet and DuckDB connectors optimize analytical query throughput?
For quantitative researchers analyzing millions of historical procurement rows and multi-decade legislative filings, Gemral Edge provides columnar Apache Parquet files optimized for high-speed analytical engines like DuckDB and ClickHouse. Columnar partitioning by fiscal year and industry sector reduces query I/O by up to 90%, enabling instantaneous aggregations and statistical regression analysis on commodity hardware.
What licensing terms and academic research exemptions govern open dataset redistribution?
All public datasets published across the Gemral Edge open repository are distributed under permissive open-data licensing frameworks that permit unrestricted commercial and academic utilization. University research departments, economic think tanks, and independent investigative journalists can freely download, redistribute, and integrate our historical datasets into published peer-reviewed scholarship with standard citation attribution.
How do automated daily validation scripts guarantee zero data loss in historical repositories?
Our data engineering infrastructure executes continuous checksum reconciliation scripts that compare historical database partitions against immutable raw government filing mirrors. By verifying record counts, sequence continuity, and field boundary constraints across multi-decade archives, the platform guarantees that historical datasets remain pristine and free from corrupt or missing records.
How do RESTful pagination and cursor-based streaming protocols optimize bulk downloads?
Our public dataset distribution architecture implements RFC 5988 cursor pagination and streaming ndjson protocols, allowing quantitative engineering teams to ingest millions of historical rows with deterministic state resumption and zero memory overflow risks on resource-constrained compute instances.
How do data schema versioning policies ensure backwards compatibility for automated quantitative pipelines?
All Gemral Edge public data endpoints adhere strictly to semantic data versioning protocols. Schema changes, field deprecations, and type expansions are published with multi-month migration runways and backwards-compatible JSON schema definitions, guaranteeing that institutional trading algorithms and production ETL pipelines operate with unbroken continuous ingestion.
Automated regression test suites continuously benchmark query latency across all historical partitions to maintain deterministic sub-second delivery SLAs.
What related alternative intelligence streams are available?
Cross-reference correlated public records across our alternative intelligence directories, statutory trackers, and quantitative research libraries:
- Federal contracts radar
- Congressional trading directory
- Government contractors directory
- Calculation methodology and standards
Access Real-Time Terminal Intelligence & Quantitative Signals
Unlock instant Telegram alerts, full congressional portfolio archives, and algorithmic catalyst radar.
Upgrade to Gemral Edge Pro ($39/mo)Frequently asked questions
What are Gemral Edge Public Datasets?
Gemral Edge Public Datasets are structured, standardized data archives published freely for academic researchers, quantitative developers, and open-source contributors. These datasets aggregate historical public disclosures—including congressional stock transactions, federal procurement contract obligations, executive guidance milestones, and sovereign central bank balance sheet time series—into clean, normalized CSV and JSON formats. By eliminating the technical friction of scraping and entity resolution, these public feeds enable researchers to backtest quantitative factors, train machine learning models, and analyze public capital flows with complete data provenance. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
What open licenses govern the public datasets?
Gemral Edge Public Datasets are released under open, transparent licensing frameworks designed to maximize academic and research utility while respecting public domain statutory data. Core public government records—such as federal contract data and STOCK Act filings—are published under Creative Commons Zero (CC0 1.0 Universal) dedication, ensuring unrestricted public domain access. Derived indices, standardized schemas, and normalized entity crosswalks are made available under Creative Commons Attribution (CC-BY 4.0), permitting open redistribution, adaptation, and commercial use with appropriate attribution to Gemral Edge. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
How can researchers and AI models access these datasets?
Researchers and autonomous AI models can access Gemral Edge datasets through direct public download links, GitHub repositories, Kaggle datasets, and native Model Context Protocol (MCP) server endpoints. Public CSV feeds are indexed with transparent column dictionaries and schema specifications, allowing seamless ingestion into Python pandas, R, and SQL databases. Autonomous AI assistants—such as Claude Code and ChatGPT—can query these datasets directly via declarative MCP tools, enabling programmatic synthesis of congressional trading data, contract velocity, and macro liquidity trends. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models.
How are proprietary metrics protected in public data exports?
Public dataset exports focus on factual statutory disclosures, normalized entity identifiers, and foundational time series, ensuring complete public availability of underlying government records. Real-time feeds and convergence alerts are part of the paid Pro and Premium plans, while programmatic access (the REST API v1 and HMAC-signed webhooks) is available only on the API plan. This dual model ensures open data access for researchers while sustaining institutional analytical infrastructure. All data signals on Gemral Edge are delivered with complete public provenance, maintaining strict non-partisan objectivity and transparent computational methodology without constituting investment advice. Every regulatory event retains its original statutory filing accession number, enabling compliance and investment teams to audit raw data directly. Historical time series are maintained with strict point-in-time veracity to prevent survivorship and look-ahead biases in quantitative research models. Platform data models follow strict entity resolution standards, linking obscure procurement subsidiaries and corporate divisions to primary trading tickers.
Risk Disclaimer
Trading and investing in digital assets, financial instruments, and predictive events involve substantial risk of loss and are not suitable for every investor. The predictive intelligence, probability distributions, historical precedents, and scenario modeling presented on this page are compiled for informational and research purposes only and do not constitute financial, investment, legal, or tax advice. Past performance and statistical precedents do not guarantee future outcomes. Always conduct independent due diligence before committing capital.