Consolidate testcontainer used by WebDriver test (#112)

* Consolidate testcontainer used by WebDriver test

Previously, we hard coded the version of the docker image for
provisioning Chrome browser and WebDriver server because the
version of the Chrome browser has to match the version of
the webdriver client, otherwise the screenshot test will fail.

Changing to use BrowserWebDriverContainer can delegate the match
to the library itself because it chooses the correct docker image
version based on the WebDriver version on our classpath.

* Increase maxColorDiff to 20

This is to supress the test flakness after switching to use
BrowserWebDriverContainer to provision browser and webdriver
service.
This commit is contained in:
Shicong Huang 2019-06-18 11:51:55 -04:00 committed by GitHub
parent c7ca9e7d09
commit 6ca5cab66f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 9 additions and 5 deletions

View file

@ -26,6 +26,7 @@ import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.BrowserWebDriverContainer;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.wait.strategy.Wait;
@ -39,8 +40,11 @@ class DockerWebDriverRule extends ExternalResource {
private WebDriver webDriver;
private static URL getWebDriverUrl() {
// BrowserWebDriverContainer will try and match the version of the Dockerized
// browser to whichever version of Selenium is found on the classpath
GenericContainer container =
new GenericContainer("selenium/standalone-chrome:3.141.59-mercury")
new BrowserWebDriverContainer()
.withFileSystemBind("/dev/shm", "/dev/shm", BindMode.READ_WRITE)
.withExposedPorts(CHROME_DRIVER_SERVICE_PORT)
.waitingFor(Wait.forHttp("/").withStartupTimeout(Duration.of(20, ChronoUnit.SECONDS)));

View file

@ -1,7 +1,7 @@
# Webdriver Tests
This is the home of the Nomulus webdriver tests. These tests run our user-facing
HTML/JavaScript code in the context of a set of common browsers. See the[Webdriver documentation](https://www.seleniumhq.org/docs/03_webdriver.jsp)
HTML/JavaScript code in the context of a set of common browsers. See the [Webdriver documentation](https://www.seleniumhq.org/docs/03_webdriver.jsp)
for more information on using webdriver.
## My Screenshot Tests are Failing!
@ -12,7 +12,7 @@ for more information on using webdriver.
2. Missing golden images
* If you added a new test using screenshot comparison, you have to generate
the golden image for that test in advance and copy it to
[goldens/](https://github.com/google/nomulus/tree/master/javatests/google/registry/webdriver/goldens)
[goldens/](https://github.com/google/nomulus/tree/master/core/src/test/java/google/registry/webdriver/goldens)
folder. There
is an auxiliary Gradle build task to help with this, and here are some examples:
```shell
@ -23,7 +23,7 @@ for more information on using webdriver.
# Generate the golden image for a certain test
$ ./gradlew :core:generateGoldenImages \
--tests "google.registry.webdriver.OteSetupConsoleScreenshotTest.get_owner_fails"
```-webkit-transition
```
3. Screenshot differs by X pixels
* If you made any change to the existing test and expected to affect the web page,
the screenshot should be different from the previously generated golden image.

View file

@ -58,7 +58,7 @@ public final class WebDriverPlusScreenDifferRule extends ExternalResource
// the sum of the absolute difference between the values of the RGB channels. So a 120,30,200
// reference pixel and a 122,31,193 result pixel have a difference of 10, and would be considered
// identical if MAX_COLOR_DIFF=10
private static final int MAX_COLOR_DIFF = 10;
private static final int MAX_COLOR_DIFF = 20;
// Percent of pixels that are allowed to be different (more than the MAX_COLOR_DIFF) between the
// images while still considering the test to have passed. Useful if there are a very small