Jan 16
Loading video player...
Blockchain generates millions of transactions daily, making it a rich yet complex source of data for developers, analysts, and researchers. While Google BigQuery offers public access to Solana’s historical data, repeated querying at scale can become costly and slow, especially during iterative exploration and analysis. In this talk, I’ll demonstrate a practical workflow that combines the power of BigQuery for data extraction with the speed and flexibility of DuckDB for local, in-memory analytics. We’ll show how to efficiently query Solana data in BigQuery, export it to partitioned Parquet files, and use DuckDB to run fast, repeatable SQL queries without incurring additional cloud costs. You'll learn: Basic terms in blockchain data structure and how transactions are saved. How to navigate and query Solana’s public datasets on BigQuery. How to export filtered blockchain data to efficient Parquet files. How DuckDB can serve as a lightweight analytics engine for on-chain data. Tips for partitioning, enriching, and automating your Solana data pipeline. This demo would all run within Google collab to save time and also enable participant follow through the session. Whether you're working on blockchain analytics, wallet behavior analysis, or on-chain data engineering, this talk will equip you with a practical approach to blockchain data workflows using open tools.