robots.txt
Authorization
EventManager.close
PersistState
networkidle
block_requests
AdaptivePlaywrightCrawler
run()
PydanticAiSelectorExtractor
extract()
BrowserPool
context.request.headers