Summary of CBOE Book Viewer Collector
The article discusses a software program designed to continuously collect and analyze market data from the CBOE (Chicago Board Options Exchange) for the SPY (S&P 500 ETF). The program operates 24/7, polling data from the CBOE's book viewer at specified intervals, and outputs the data in both CSV and JSON formats.
Key Features
- Symbol and Venue: The program focuses on the SPY symbol and operates on the EDGX venue.
- Polling Frequency: It polls data every second during market hours (04:00 to 20:00 New York time) and every minute during closed sessions.
- Data Output: The program writes a CSV file every second during live sessions and a full JSON snapshot every minute, along with immediate snapshots when data quality changes occur.
- Session Management: It handles different market sessions (closed, early, pre-market, regular, after-hours) and adjusts polling intervals accordingly.
- Error Handling: The program is designed to recover from browser or page failures by recreating the browser session as needed.
Data Collection Process
The program utilizes the Playwright library to automate browser interactions. It opens the CBOE book viewer page, dismisses cookie banners, and selects the appropriate venue for data collection. The data is parsed from the DOM (Document Object Model) and includes details such as best bid and ask prices, trade prints, and market depth.
Data Quality and Freshness
The program includes mechanisms to ensure data quality by checking the freshness of the exchange timestamp and the completeness of the parsed data. If the data is stale or incomplete, it flags the entries accordingly.
Logging and Monitoring
The program logs various events, including session transitions, errors, and heartbeat signals during closed sessions. This logging is crucial for monitoring the health of the data collection process.
Conclusion
The CBOE Book Viewer Collector is a robust tool for real-time market data collection, designed to operate continuously and handle various market conditions. Its structured output and error handling capabilities make it a valuable asset for traders and analysts looking to monitor market dynamics effectively.