- Expose error log for loading process
- add support for proxy_credential
- fix bug of network error and add retry
- add support for executing Python based functions
- add support for returning script result
- change GERAPY_ENABLE_REQUEST_INTERCEPTION default to False
- fix pu_request.headers decode error
- add ua setting
- fix pyppeteer_meta name error
- add chrome pretend test
- update pretend script & fix local pretend invalid
- Fix bug about executablePath arg name
- Fix bug about executablePath arg name
- Fix bug about crawling with Pyppeteer of normal Request
- Add settings of
GERAPY_ENABLE_REQUEST_INTERCEPTION
#6
- Add
pretend
attribute forPyppeteerRequest
, which can overrideGERAPY_PYPPETEER_PRETEND
- Add support for
dict
format ofwait_for
attribute ofPyppeteerRequest
- Change the priority of
request.meta.get('proxy')
andpyppeteer_meta.get('proxy')
- Add support for screenshot
- Fix bug of name
GERAPY_PYPPETEER_IGNORE_RESOURCE_TYPES
- Add support for pretending as real Browser instead of WebDriver
- Fix bug of ValueError when
wait_until
is None - Fix error position of log message about
wait_for
- Add meta info from PyppeteerRequest attributes
- Skip validation of PyppeteerRequest
- Add support for
ignoreHTTPSErrors
,slowMo
,ignoreDefaultArgs
,handleSIGINT
,handleSIGTERM
,handleSIGHUP
,autoClose
args.
- Fix Bug of un-closing Pyppeteer when loaded failed
- Add support for
GERAPY_IGNORE_RESOURCE_TYPES
- Add support for retrying