Using Selenium to Automate Interaction with External Resources

Using Selenium for automation of external resource interaction may seem like an unusual way to apply such a handy testing tool. In this article, I’ll try to show that it really suits the task and does a great job saving time and efforts.

Let me start with a brief overview of Selenium and what it is typically used for. First of all, yes, it’s a software testing suite for web applications. It provides a platform for launching web applications in different browsers and several element lookup mechanisms such as XPath or CSS selectors.

Selenium tools

Selenium consists of several parts.

• Selenium 2 (Selenium WebDriver)

The latest milestone in the Selenium project integrates the WebDriver API into Selenium 1. This combination provides the most effective and stable automating tool.

• Selenium 1 (Selenium Remote Control)

Selenium RC was the main development focus of the Selenium project before Selenium 2 emerged. It is still widely used and supported.

• Selenium IDE

The development environment for prototyping test scenarios shipped as a Firefox plugin. Selenium IDE has a simple and convenient GUI for automated test creation.

• Selenium Grid

Selenium Grid is a version of Selenium for full-scale automation of large test suites and running multiple tests in parallel across multiple environments. With Selenium Grid different tests can be launched at the same time on different remote machines.

Using Selenium for not-testing tasks in a real project

For example, let’s take a look at an existing project of building large-scale web application that interacted with many external resources. We know nothing about the internal structure of the application and the organization of the external resources, but we can create new accounts on the known resources as well as use the existing accounts through the provided web interface.

Essentially the application was an aggregator of accounts at some third party resources and it performed some operations under them. The operations were almost the same for every resource and they were performed using the resource API. However, the developers encountered a lack of API methods for some of the external resources. Several resources didn’t provide an API and they were scheduled for inclusion into the application as well.

In a nutshell, some of the resources provided a convenient API with a complete set of required methods and some didn’t provide the required methods or didn’t have an API at all. The APIs, if present, also were slightly different and each had their own requirements, that greatly increased the project size, so it would be pointless to create a uniform API-based approach. The only way to standardize the method calls would be to create a set of adapter for every resource.

At some point we decided to try Selenium for simulating user actions on a third party resource: launching a browser, filling form data, browsing and logging out.

You might say that setting up a Selenium server would be irrational and it’s easier to analyse the request/response pair for every request in a chain of page views and implement a server-side CURL-based library based on the results. But in our experience analysing the requests takes time due to individual complexities in third party resources, such as:

• Hash parameters in the request for verifying integrity;
• JS-based cookies;
• API-specific request sequences;
• Heavy AJAX traffic;
• Header-specific data verification.

The Selenium approach also benefits from flexible and fast adaptation to changes in the external resource.

To sum up, the reasons to use the Selenium approach were:

• Limited or missing third party APIs;
• Significant API differences between resources, external dependencies;
• Requests and responses that require time-consuming analysis;
• Data protection mechanisms in the external resources and their specific features;
• Increased productivity during the initial development and later maintenance.

This approach has two disadvantages: execution time and stability. Both points should be considered before using Selenium in speed-critical tasks. Of course, the execution time for user operations can be optimized, as we did in our project.

As for stability, it can be raised to an acceptable level by adding components for more stable interaction with Selenium, in particular monitoring components for the Selenium server. The server itself may be configured to run faster. Various aspects of configuration can be found in many online tutorials.

Installing and configuring the Selenium server

To install the Selenium server and make it usable from PHP applications follow the instructions below. These instructions apply to CentOS-based server environments.

Install Java:

yum install java;
Install virtual frame buffer for X emulation:
yum install Xvfb;

Create a script to run xvfb as a daemon:

/etc/init.d/xvfb

Set the appropriate permission and run the script:

chmod +x /etc/init.d/xvfb 
chkconfig xvfb on 
service xvfb start

Get Selenium (the latest version may be different):

cd /usr/local/lib/selenium
wget http://selenium.googlecode.com/files/selenium-server-standalone-2.20.0.jar
mkdir -p /var/log/selenium/
chmod a+w /var/log/selenium/

Create a startup script:

/etc/init.d/selenium

Don’t forget to add the following parameters for full-scale usage:

PARAMS="-disable-web-security -ignore-certificate-errors -interactive -multiWindow -trustAllSSLCertificates"

How to run:

chmod +x /etc/init.d/selenium
chkconfig selenium on
service selenium start

Install the browser.

Tip: use Google Chrome and don’t forget to add the following repository /etc/yum.repos.d/google.repo: yum install google-chrome-stable

Google Chrome may require WebDriver:

PARAMS="-disable-web-security -ignore-certificate-errors -interactive -multiWindow -trustAllSSLCertificates -
Dwebdriver.chrome.driver=/var/….. /chromedriver"

Install PHPUnit:

yum install php-pear
pear install phpunit/PHPUnit_Selenium

Add the following PEAR channels:

pear channel-discover pear.phpunit.de
pear channel-discover pear.symfony-project.com
pear channel-discover components.ez.no
Install the Selenium server interaction library for PHP:
pear install -a Testing_Selenium-0.4.4

Summing up

To summarize, I recommend using Selenium server in non-testing purposes in projects that are not sensitive to execution time. This approach has it’s clear benefits: the simplicity of making changes, since you don’t need to analyze each query separately, and the simplicity of the development process itself. All these make Selenium a great “extension” of a given API to the required level.