Updated Fetching pages with WebDriver (markdown)

master
dgtlmoon 2022-10-19 10:00:35 +02:00
rodzic cf671d7423
commit a51f85aed4
1 zmienionych plików z 1 dodań i 10 usunięć

@ -1,4 +1,4 @@
Many modern web pages use javascript to fill-in the content, they are more dynamic and sometimes need a real chrome browser for fetching the content, although many may work with our built in 'fetcher'
Many modern web pages use JavaScript to fill-in the content, they are more dynamic and sometimes need a real chrome browser for fetching the content, although many may work with our built in 'fetcher'
The backend can be configured to fetch pages via Chrome (ChromeDriver) using the built in [WebDriver](https://www.selenium.dev/documentation/webdriver/) network interface, this is mainly used where the pages you are watching are using JavaScript to render the page content (the basic fetcher does not execute any JS!), The easiest way is to enable it is to uncomment the following in your local [docker-compose.yml](https://github.com/dgtlmoon/changedetection.io/blob/master/docker-compose.yml) and restart your docker-compose.
@ -67,12 +67,3 @@ Tested with Chrome version 99.0
![image](https://user-images.githubusercontent.com/275001/160232926-70fee16f-634b-42cd-86c3-0ffaa5a0aa22.png)
# There's a few things left todo
* Make tests use the full docker stack with the chromedriver + relevant settings
* Lower the number of workers (make it configurable per backend?) 10 is too many (is it?), but this is totally fine for requests/plaintext
* Some kind of dom-wait-until-fully loaded or other
* Handle failures better (better feedback so we know if its from chromedriver, or from the website)
* Save last screenshot? _(This extends to saving an entire page screenshot, abstracting out the handler for diff-management, then supplying an image-diff handler)_