Summary of CBOE Book Viewer Collector
The article discusses a software program designed to continuously collect and analyze market data from the CBOE (Chicago Board Options Exchange) for the SPY (S&P 500 ETF). The program operates 24/7, polling data from the CBOE's book viewer for the SPY symbol on the EDGX venue. It is structured to run continuously, with specific functionalities and configurations aimed at ensuring data integrity and timely updates.
Key Features
- Polling Frequency: The program polls the CBOE data once every second during market hours (04:00 to 20:00 New York time) and once every minute during closed sessions.
- Data Output: It writes CSV files every second during live sessions and JSON files every minute, along with immediate JSON outputs when there are changes in data quality or parsing state.
- Session Management: The program identifies market sessions (open, closed, early, pre-market, regular, after-hours) and adjusts its polling frequency accordingly.
- Error Handling: In case of browser or page failures, the program is designed to recover by recreating the browser session.
- Data Quality Checks: The program flags stale data and ensures that the market state and exchange timestamps are fresh before processing the data.
Technical Implementation
The program is implemented in C# and utilizes the Playwright library for browser automation. It includes various methods for managing browser sessions, capturing market data, and parsing the data into structured formats. The data is stored in both CSV and JSON formats for easy access and analysis.
Data Parsing and Storage
The program captures market data, including the best bid and ask prices, trade prints, and market depth. It organizes this data into structured records that can be easily analyzed. The captured data is stored in a specified output directory, with separate files for session summaries and heartbeats.
Logging and Monitoring
The program includes logging capabilities to track its operations, errors, and data quality issues. This ensures that any problems can be diagnosed and addressed promptly, maintaining the reliability of the data collection process.
Conclusion
The CBOE Book Viewer Collector is a robust tool for real-time market data collection, designed to operate continuously with high reliability. Its structured approach to data polling, error handling, and output management makes it a valuable asset for traders and analysts looking to monitor market conditions effectively.