Обработка ошибок python selenium

In the world of automation testing, Selenium is a free and open-source framework used to perform web application testing in web browsers like Chrome, Safari, Firefox, Opera, and Edge. You can write Selenium IDE tests using various programming languages such as Python, Java, JavaScript(Node.js), Kotlin, Ruby, PHP, and C#.

While the advantages of automated testing using Selenium are endless, there are incidences of errors and exceptions that happen regularly in the testing tool (or framework). When a program’s execution is disrupted during run-time, an exception is raised, preventing further instructions from being executed. Runtime errors are raised when you perform an unanticipated operation during runtime.

Exception and error handling is a programming mechanism by which the occurrence of errors and exceptions that halt the execution of a program are responded to by the program to ensure the normal execution of code.

In this blog on handling exceptions in Selenium Python, we will look at the variety of exceptions and errors that can happen when a Selenium test is running. By the end of this blog, you will be able to implement error and exception handling for Selenium automation tests.

So, let’s get started!

Introduction to Errors and Exceptions in Selenium Python

For a program or application to run, a developer must write code that tests every aspect of its functionality. However, the instructions outlined in the code sometimes do not execute as intended.

For QA engineers, the same rule holds when writing code used to test the features and functionality of the applications that have been created to ensure that the application is ready for production and then shipped to the users.

Here is an analogy to help you better understand errors and exceptions in Selenium Python.

Imagine you have a work meeting on Zoom or any online video conference application while at home. During the meeting, the electricity was cut off, which is reported to have affected the whole city. You have no choice but to call off the meeting and wait for the electricity to be reinstated. Akin to this, errors are raised during runtime, and rectifying them is nearly impossible.

In contrast, exceptions are events that raise interruptions while an application runs and can be rectified using exception handling.

Exceptions in Selenium Python are generally classified into two types.

• Checked Exceptions
• Unchecked Exceptions

Checked Exception handling in Selenium is conducted while writing the code. On the other hand, Unchecked Exceptions are raised during runtime and can generate fatal errors compared to the former type of exception.

The class hierarchy of error and exceptions in Selenium Python

You can quickly sharpen your knowledge of exceptions in Selenium Python by reading through this detailed blog on different exceptions in Selenium.

Performing Selenium automated browser testing differs across the ecosystem in which the tests are being carried out. The test scripts may differ depending on the operating systems, browsers, or programming languages they are written in. Therefore, the raised errors andexceptions in Selenium Python may be different. For example, if the browsers don’t have the specified element, Selenium testing of an element property in web browsers may raise exceptions like NoSuchElementException.

To ensure that the applications function correctly and obstruct users’ interactions with the finished product, it is advised that Selenium error and exception handling be used as a standard practice.

Common Exceptions in Selenium Python

According to a recent Stack Overflow developer survey, Python is the fourth most popular programming language among developers.

It offers a robust support system for test automation frameworks and is the most user-friendly and straightforward programming language available.

Errors and exceptions in Selenium Python are common and can occur when writing tests. The exception instances must be derived from a class that inherits from BaseException. If two exception instances do not have a subclass relationship, they are not considered equal despite having the same properties, such as names.

While writing code for the test scripts, the code can run into built-in exceptions, or you can raise exceptions in the code. The interpreter or built-in functions generate the built-in exceptions.

The selenium.common.exceptions package contains the exception classes, which can be imported in the following way.

Syntax: from selenium.common.exceptions import [Exception Name]

An example of an exception being imported is below:

This exception indicates that the website does not contain the attribute of the element being looked for.

Other exceptions in Selenium Python include:

• TimeOutException

When a command does not complete in the time duration set (seconds), this exception is raised or thrown. The wait time can either be implicitly or explicitly defined.

• NoSuchElementException

When an element cannot be found, this exception is raised or thrown. Nonetheless, different factors can lead to this exception being generated:

• The element is not yet in the DOM when the find operation is performed because the page loading is in progress.
• Incorrect element locator is used in the source code.

• ElementNotInteractableException

Such exceptions in Selenium Python is raised whenever an element in the DOM is present but cannot be interacted with.

• ElementNotVisibleException

This exception typically arises when the interaction involves clicking or reading text from an element that is not visible in the DOM.

• StaleElementReferenceException

An outdated or unavailable element in the DOM is referred to as a stale element. A reference to an element that is currently “stale” will raise or throw this exception.

For example, the following reasons could lead to the StaleElementReferenceException exception:

• After the element was located, the page either changed or was refreshed.
• A refreshed context may have contained the element.
• Complete deletion of the referenced web element.

• ElementClickInterceptedException

This exception arises when the element receiving the interaction impedes the element intended to be clicked, thereby preventing the Element Click operation from being performed.

• NoSuchCookieException

This exception will be raised if a cookie with the specified path name is not contained amongst the corresponding cookies of an active document in the current browsing session. To learn more about cookies in Selenium, check out this detailed tutorial on handling cookies in the Selenium WebDriver.

Watch this video to understand how you can handle Cookies and perform different operations like deleting, getting the parameter values, and adding them to Selenium WebDriver using Java.

• InvalidArgumentException

A command with improper or incorrect argument inputs will raise an InvalidArgumentException.

• InvalidElementStateException

This exception is raised when we try to perform an operation that does not apply to an element. For example, if we perform a click operation on a radio button that is disabled, the exception will be raised, and hence the command given cannot be executed.

• NoSuchWindowException

This exception is thrown when the intended window target cannot be found.

If you are a Python programmer looking to make a mark, you can take your career to the next level with the Selenium Python 101 certification from LambdaTest.

Here’s a short glimpse of the Selenium Python 101 certification from LambdaTest:

Handling Errors and Exceptions in Selenium Python

Handling errors and exceptions in Selenium Python are vital when writing production-ready code, since exceptions and errors can arise for different reasons, as discussed earlier.

A prerequisite to writing tests using Selenium Python is to install Python and Selenium on your machine. When using Python for Selenium testing, the default framework used is PyUnit, also known as unittest, which is influenced by JUnit. It functions similarly to other unit testing frameworks.

However, in this blog on handling errors and exceptions in Selenium Python, we will use Pytest, another Python testing framework that supports functions, APIs, and unit testing. It is compatible with the current versions of Python and is easy to configure using the Python Package Manager (pip) command in the terminal.

If you are new to testing using Selenium Python, here is a detailed step-by-step tutorial on how to get started with Selenium Python. You will learn how to configure Python, Selenium, and Pytest on your machine to start automation testing with Selenium. You can learn more about Pytest through this Selenium Pytest tutorial.

Project Setup

For the test project setup, we are using the Visual Studio (VS) Code IDE to write the test scripts. Alternatively, there are other IDEs available, such as Pycharm, Atom, and Sublime Text, that you can use as well.

For the package manager, we will use Poetry, a Python tool that manages and packages the necessary dependencies in a specific virtual environment for your project. You can declare the libraries your project depends on, and it will take care of managing (installing and updating) them.

1. To install Poetry on your local machine’s terminal, please use the following command:
• For Windows

pip install poetry

• For Mac

pip3 install poetry (depends on the Python version you are using).

2. Move to the directory where your project will be contained.



3. To initialize Poetry in the terminal, we use the command poetry init. This command will generate a pyproject.toml configuration file containing the build system requirements for Python projects.



4. We need the following libraries in our project to run Selenium Python tests:
• selenium (v 4.3.0)
• flake8 (v 4.0.1)
• pytest (v 7.1.2)
• pytest-xdist (v 2.5.0)
• autopep8 (v 1.6.0)

The pytest-xdist plugin adds new test execution modes to Pytest, allowing us to run parallel Selenium Python tests. Autopep8 formats the Python code to adhere to the PEP 8 style manual. Flake8 is a great tool that tests your code for errors against the PEP 8 style guide.

5. Adding Selenium:



6. Adding pytest:



7. Adding pytest-xdist.



8. Adding autopep8.



9. Adding flake8.



10. After installing the required dependencies, continue to press Enter, then confirm the generation of the file.
11. The pyproject.toml file contains all the packages that must be installed in the virtual environment.

To install the dependencies mentioned, run the following command to create the pyproject.toml and poetry.lock files in the directory.

A virtual environment (venv) containing the dependencies installed will be automatically created by Poetry. This is important since we do not want to mix Python’s global dependencies with the specific dependencies we require to run the tests in the project.

In our project, we create a folder that will hold the test functions. Let’s call it “seltests”.

Handling Errors and Exceptions on the Cloud Selenium Grid

Now that we have covered the essentials to get started with Python web automation in Selenium, we will look at handling errors and exceptions in Selenium Python on a cloud grid. In this instance, we are using LambdaTest.

LambdaTest is a cloud-based cross browser testing platform that allows you to perform Python automation testing at scale over a cloud Selenium Grid. The unique and fully featured platform offers an online browser farm of 3000+ real browsers across multiple operating systems; thus helping businesses deliver more quality software at a more rapid speed.

You can also Subscribe to the LambdaTest YouTube Channel and stay updated with the latest tutorials around automated browser testing, Selenium testing, Cypress E2E testing, CI/CD, and more.

To get started with LambdaTest, please follow the below-mentioned steps:

1. Register and log in to your LambdaTest account.
2. Move to the LambdaTest Profile Section and copy your username and access key, which will be used to access the Remote Selenium Grid of LambdaTest.
3. You will also need the desired capabilities generated using the LambdaTest Capabilities Generator. You can generate the capabilities needed for the specific test you are carrying out on that page.

We have listed the different types of exceptions in Selenium Python in the blog. Now, we will dig deeper into two common exceptions in Selenium Python when performing automated tests.

NoSuchElementException

A NoSuchElementException is thrown if we attempt to locate any element unavailable on the page (or the DOM).

Scenario 1 (Chrome Browser)

1. Go to the URL link https://www.lambdatest.com/selenium-playground/.
2. Locate an element called “p-10” by its name.
3. Click on the element.

Scenario 2 (Firefox Browser)

1. Go to the URL link https://www.lambdatest.com/selenium-playground/.
2. Locate an element called “level-up” by its name.
3. Click on the element.

Implementation (Scenario 1 on the Chrome Browser):

FileName – test_nosuchelement_google.py

Implementation (Scenario 2 on the Firefox Browser):

FileName – test_nosuchelement_firefox.py

GitHub

Code Execution:

We run the two test scripts using parallel testing, which allows the execution of the same tests simultaneously. To run the test, type the following command in the terminal.

The terminal shows that the test execution failed.

In the LambdaTest Builds dashboard, the failed test executions are displayed.

Code Walkthrough:

We are conducting a joint code walkthrough because the imported Selenium Python libraries used in both test scenarios are identical.

Step 1: Import the necessary Selenium Python classes – pytest, webdriver, sys, By, NoSuchElementException.

We imported pytest, primarily used for unit testing in Selenium Python. Selenium WebDriver is a web framework that enables us to perform cross-browser testing, and in this case, we are using Python to write the test scripts. The By query is based on the find_element() method in Selenium that targets a specific locator.

Step 2: We specify the browser capabilities based on the browser we use to conduct the automated tests in LambdaTest. They are generated using the LambdaTest Capabilities Generator.

Chrome

Firefox

Step 3: We define a function that starts with “test” in which we assign the test methods. You can learn more about it through this blog on Pytest testing using Selenium WebDriver.

Step 4: We then set our LambdaTest username and accesskey to variables we appended to remote_url. This remote URL connects us to the Remote Selenium Grid (@hub.lambdatest.com/wd/hub).

Step 5: We then use the remote_url and browser capabilities to instantiate the corresponding browser (i.e. Chrome or Firefox)

Chrome

Firefox

Step 6: We then get the URL of the website on which we want to locate a specific element. In our test, we try to locate an element by its name and click on it. Once the element had been located and clicked, we quit the test.

Chrome

Firefox

Watch this video to learn exception handling in Selenium, which covers different types of exceptions and how to handle them using various methods.

NoSuchElementException Handling in Action

According to the exception mentioned above, we must catch “NoSuchElementException” and print the proper log messages to guarantee that our test fails with the proper error message.

We use the try and except method to catch the exception when it is raised in the test script. The “try” block lets you check the code for errors, while the “except” block lets you handle the errors or exceptions raised.

Chrome

Firefox

Execution:

Use the terminal to enter the following command to perform Selenium Python testing in parallel on the LambdaTest Grid.

The tests will be executed successfully.

The passed tests will be shown as follows in the LambdaTest Dashboard.

The passed tests will be shown as follows in the LambdaTest Dashboard.

Code Walkthrough:

The “try” block will run the find_element() argument and take in the By query to locate an element using its name in the URL provided.

Step 1: In the browser’s inspect tool, click on Console and type the following command to find the element with the name “p-10”.

The test does not fail since the element with the name “p-10” does not exist. The “except” block will run, catching the exception.

Step 2: The try and except code block is the same in Chrome and Firefox text scripts.

Chrome

Firefox

StaleElementReferenceException

This exception is raised whenever an element is not present in the DOM or is deleted.

Scenario 1 (Chrome Browser)

1. Go to the URL link https://ecommerce-playground.lambdatest.io/index.php?route=account/login.
2. Find the reference to the email and password text fields.
3. Click on a link that reloads the entire web page.
4. Enter the password into the password field with the reference we located before the web page reloaded.

Scenario 2 (Firefox Browser)

1. Go to the URL link https://ecommerce-playground.lambdatest.io/index.php?route=account/login.
2. Find the reference to the email and password text fields.
3. Click on a link that reloads the entire web page.
4. Enter the password into the password field with the reference we located before the web page reloaded.

Implementation (Scenario 1 on the Chrome Browser):

FileName – test_staleelement_chrome.py

Implementation (Scenario 2 on the Firefox Browser):

FileName – test_staleelement_firefox.py

Execution:

Run the following command in the terminal to execute the test scripts in parallel testing.

poetry run pytest seltests/test_staleelement_google.py  seltests/test_staleelement_firefox.py

We receive a StaleElementExceptionError message in the terminal.

We can view the raised exceptions in the LambdaTest Dashboard.

Run Python test scripts across 3000+ devices & browsers. Try LambdaTest Now!

 
Code Walkthrough:

Step 1: Import the necessary Selenium Python classes – pytest, webdriver, sys, By, and StaleElementReferenceException.

import pytest from selenium import webdriver import sys from selenium.webdriver.common.by import By from selenium.common.exceptions import StaleElementReferenceException

Step 2: We generate capabilities in the LambdaTest Capabilities based on the browser and OS.

Chrome

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

ch_capabilities = {     ‘LT:Options’: {         «user»: «<username>»,         «accessKey»: «<accesskey>»,         «build»: «StaleElementReferenceException Test on Chrome»,         «name»: «StaleElementReferenceException Test on Chrome»,         «platformName»: «Windows 10»     },     «browserName»: «Chrome»,     «browserVersion»: «102.0», } <pre> <center><strong>Firefox</strong></center> <pre>ff_capabilities = {         ‘LT:Options’ : {             «user» : «<username>»,             «accessKey» : «<accesskey>»,             «build» : «StaleElementReferenceException Test for LambdaTest (Firefox)»,             «name» : «StaleElementReferenceException Test for LambdaTest (Firefox)»,             «platformName» : «Windows 10»         },         «browserName» : «Firefox»,         «browserVersion» : «101.0»,     }

Step 3: We create a function that will contain our test methods. In the function, we define the username and password for a remote_url that will connect to the Remote Selenium Grid.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

def test_ecommerceplayground_staleelement():     # LambdaTest Profile username     user_name = «<username>»     # LambdaTest Profile access_key     app_key = «<accesskey>»     # Remote Url to connect to our instance of LambdaTest     remote_url = «https://» + user_name + «:» +         app_key + «@hub.lambdatest.com/wd/hub»     # creating an instance of Chrome based on the remote url and the desired capabilities     ch_driver = webdriver.Remote(         command_executor=remote_url, desired_capabilities=ch_capabilities)     ch_driver.get(      ‘https://ecommerce-playground.lambdatest.io/index.php?route=account/login’)     emailElement = ch_driver.find_element(By.ID, «input-email»)     passwordElement = ch_driver.find_element(By.ID, «input-password»)     emailElement.send_keys(«email@gmail.com»)     ch_driver.find_element(By.XPATH, «//input[@type=’submit’]»).click()     passwordElement.send_keys(«password»)     ch_driver.quit()

The code for the browser tests is the same. We set the Chrome webdriver to ch_driver while the Firefox webdriver is ff_driver. Based on their ids, the emailElement variable will locate the email input element, while the passwordElement variable will locate the input element.

We then enter an email into the email input using the emailElement.send_keys() function and locate and click the login button, which is the type “submit”. Then enter the password and finally quit the test with the .quit() function.

When the code is run, a StaleElementReferenceException will be raised for the following reasons:

• In Chrome, the element is not attached to the page document.
• In Firefox, the element reference password is no longer attached to the DOM.

Code Walkthrough:

We apply the try and except code blocks to handle the “StaleElementReferenceException” when it is thrown during our test’s execution.

The try block will run the code to add the password in which the exception will be raised. The except block will handle the exception by locating the password input element and sending the password value.

When the code is run, it will execute successfully in the terminal.

poetry run pytest seltests/test_handle_staleelement_google.py seltests/test_handle_staleelement_firefox.py

We can see successful test executions in the LambdaTest Build dashboard.

For a complete view of all test results and trends, check out your LambdaTest Analytics Dashboard. It lets you see how the tests are moving along, so you can make sure that everything is executed accordingly.

Conclusion

In this blog, we have looked at some common exceptions in Selenium Python. We have also deep-dived into various exceptions and looked at why NoSuchElementException and StaleElementReference are generated and how to handle them. Selenium Python provides methods using which you can handle various exceptions, and in this case, we have focused on the try and except method.

Frequently Asked Questions (FAQs)

What are the exceptions in Selenium Python?

An exception is an event, which occurs while our program is running, that prevents the execution of statements as normal. When an exception occurs, the execution of statements stops, and Python starts searching for a place to “jump out” to.

What are the five exceptions in Selenium?

• NoSuchWindowException.
• NoSuchFrameException.
• NoSuchElementException.
• NoAlertPresentException.
• InvalidSelectorException.

What are the Exception types in Selenium?

Checked Exceptions are handled during the process of writing the codes. At that time, these exceptions are handled before the compiling process. If it happens, such an exception is examined at the compile time. Checked Exceptions are the ones that are handled at the time when the code is compiled. These exceptions have to be handled at compile time, and hence, these exceptions get examined before compiling the code.

Eugene Kwaka

I am a Software Developer with a background in Python(Django) and a tech enthusiast who loves writing and researching different topics on emerging trends in technology. I am keen on Software Testing, Backend Software Development, and best practices. I love tooling around and getting my hands dirty by building and learning new projects. If I’m not coding, you’ll find me traveling, listening to music, or trying different foods.

Exceptions that may happen in all the webdriver code.

Exceptions

ElementClickInterceptedException(msg, …)	The Element Click command could not be completed because the element receiving the events is obscuring the element that was requested to be clicked.
ElementNotInteractableException(msg, screen, …)	Thrown when an element is present in the DOM but interactions with that element will hit another element due to paint order.
ElementNotSelectableException(msg, screen, …)	Thrown when trying to select an unselectable element.
ElementNotVisibleException(msg, screen, …)	Thrown when an element is present on the DOM, but it is not visible, and so is not able to be interacted with.
ImeActivationFailedException(msg, screen, …)	Thrown when activating an IME engine has failed.
ImeNotAvailableException(msg, screen, stacktrace)	Thrown when IME support is not available.
InsecureCertificateException(msg, screen, …)	Navigation caused the user agent to hit a certificate warning, which is usually the result of an expired or invalid TLS certificate.
InvalidArgumentException(msg, screen, stacktrace)	The arguments passed to a command are either invalid or malformed.
InvalidCookieDomainException(msg, screen, …)	Thrown when attempting to add a cookie under a different domain than the current URL.
InvalidCoordinatesException(msg, screen, …)	The coordinates provided to an interaction’s operation are invalid.
InvalidElementStateException(msg, screen, …)	Thrown when a command could not be completed because the element is in an invalid state.
InvalidSelectorException(msg, screen, stacktrace)	Thrown when the selector which is used to find an element does not return a WebElement.
InvalidSessionIdException(msg, screen, …)	Occurs if the given session id is not in the list of active sessions, meaning the session either does not exist or that it’s not active.
InvalidSwitchToTargetException(msg, screen, …)	Thrown when frame or window target to be switched doesn’t exist.
JavascriptException(msg, screen, stacktrace)	An error occurred while executing JavaScript supplied by the user.
MoveTargetOutOfBoundsException(msg, screen, …)	Thrown when the target provided to the ActionsChains move() method is invalid, i.e.
NoAlertPresentException(msg, screen, stacktrace)	Thrown when switching to no presented alert.
NoSuchAttributeException(msg, screen, stacktrace)	Thrown when the attribute of element could not be found.
NoSuchCookieException(msg, screen, stacktrace)	No cookie matching the given path name was found amongst the associated cookies of the current browsing context’s active document.
NoSuchElementException(msg, screen, stacktrace)	Thrown when element could not be found.
NoSuchFrameException(msg, screen, stacktrace)	Thrown when frame target to be switched doesn’t exist.
NoSuchShadowRootException(msg, screen, …)	Thrown when trying to access the shadow root of an element when it does not have a shadow root attached.
NoSuchWindowException(msg, screen, stacktrace)	Thrown when window target to be switched doesn’t exist.
ScreenshotException(msg, screen, stacktrace)	A screen capture was made impossible.
SessionNotCreatedException(msg, screen, …)	A new session could not be created.
StaleElementReferenceException(msg, screen, …)	Thrown when a reference to an element is now “stale”.
TimeoutException(msg, screen, stacktrace)	Thrown when a command does not complete in enough time.
UnableToSetCookieException(msg, screen, …)	Thrown when a driver fails to set a cookie.
UnexpectedAlertPresentException(msg, screen, …)	Thrown when an unexpected alert has appeared.
UnexpectedTagNameException(msg, screen, …)	Thrown when a support class did not get an expected web element.
UnknownMethodException(msg, screen, stacktrace)	The requested command matched a known URL but did not match any methods for that URL.
WebDriverException(msg, screen, stacktrace)	Base webdriver exception.

exception selenium.common.exceptions.ElementClickInterceptedException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

The Element Click command could not be completed because the element
receiving the events is obscuring the element that was requested to be
clicked.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ElementNotInteractableException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when an element is present in the DOM but interactions with that
element will hit another element due to paint order.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ElementNotSelectableException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when trying to select an unselectable element.

For example, selecting a ‘script’ element.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ElementNotVisibleException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when an element is present on the DOM, but it is not visible, and
so is not able to be interacted with.

Most commonly encountered when trying to click or read text of an
element that is hidden from view.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ImeActivationFailedException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when activating an IME engine has failed.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ImeNotAvailableException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when IME support is not available.

This exception is thrown for every IME-related method call if IME
support is not available on the machine.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InsecureCertificateException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Navigation caused the user agent to hit a certificate warning, which is
usually the result of an expired or invalid TLS certificate.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidArgumentException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

The arguments passed to a command are either invalid or malformed.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidCookieDomainException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when attempting to add a cookie under a different domain than the
current URL.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidCoordinatesException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

The coordinates provided to an interaction’s operation are invalid.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidElementStateException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when a command could not be completed because the element is in
an invalid state.

This can be caused by attempting to clear an element that isn’t both
editable and resettable.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidSelectorException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when the selector which is used to find an element does not
return a WebElement.

Currently this only happens when the selector is an xpath expression
and it is either syntactically invalid (i.e. it is not a xpath
expression) or the expression does not select WebElements (e.g.
“count(//input)”).

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidSessionIdException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Occurs if the given session id is not in the list of active sessions,
meaning the session either does not exist or that it’s not active.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.InvalidSwitchToTargetException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when frame or window target to be switched doesn’t exist.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.JavascriptException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

An error occurred while executing JavaScript supplied by the user.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.MoveTargetOutOfBoundsException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when the target provided to the ActionsChains move() method is
invalid, i.e. out of document.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoAlertPresentException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when switching to no presented alert.

This can be caused by calling an operation on the Alert() class when
an alert is not yet on the screen.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchAttributeException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when the attribute of element could not be found.

You may want to check if the attribute exists in the particular
browser you are testing against. Some browsers may have different
property names for the same property. (IE8’s .innerText vs. Firefox
.textContent)

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchCookieException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

No cookie matching the given path name was found amongst the associated
cookies of the current browsing context’s active document.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchElementException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when element could not be found.

If you encounter this exception, you may want to check the following:

• Check your selector used in your find_by…
• Element may not yet be on the screen at the time of the find operation,
  (webpage is still loading) see selenium.webdriver.support.wait.WebDriverWait()
  for how to write a wait wrapper to wait for an element to appear.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchFrameException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when frame target to be switched doesn’t exist.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchShadowRootException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when trying to access the shadow root of an element when it does
not have a shadow root attached.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.NoSuchWindowException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when window target to be switched doesn’t exist.

To find the current set of active window handles, you can get a list
of the active window handles in the following way:

print driver.window_handles

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.ScreenshotException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

A screen capture was made impossible.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.SeleniumManagerException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Raised when an issue interacting with selenium manager occurs.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.SessionNotCreatedException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

A new session could not be created.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.StaleElementReferenceException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when a reference to an element is now “stale”.

Stale means the element no longer appears on the DOM of the page.

Possible causes of StaleElementReferenceException include, but not limited to:

• You are no longer on the same page, or the page may have refreshed since the element
  was located.
• The element may have been removed and re-added to the screen, since it was located.
  Such as an element being relocated.
  This can happen typically with a javascript framework when values are updated and the
  node is rebuilt.
• Element may have been inside an iframe or another context which was refreshed.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.TimeoutException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when a command does not complete in enough time.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.UnableToSetCookieException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when a driver fails to set a cookie.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.UnexpectedAlertPresentException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None, alert_text: Optional[str] = None)[source]¶

Thrown when an unexpected alert has appeared.

Usually raised when an unexpected modal is blocking the webdriver
from executing commands.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.UnexpectedTagNameException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Thrown when a support class did not get an expected web element.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.UnknownMethodException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

The requested command matched a known URL but did not match any methods
for that URL.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

exception selenium.common.exceptions.WebDriverException(msg: Optional[str] = None, screen: Optional[str] = None, stacktrace: Optional[Sequence[str]] = None)[source]¶

Base webdriver exception.

args¶

with_traceback()¶

Exception.with_traceback(tb) –
set self.__traceback__ to tb and return self.

14 Oct 2018

Selenium is a very versatile web scraping tool that is accessible via multiple programming languages.

It’s distinguished from text-parsing scrapers like BeautifulSoup as it actually simulates a web navigation experience, enabling you to scrape website running on a lot of Javascript and iframes.

That makes Selenium especially powerful when you are in need of scraping large websites, like e-commerce sites.

However, as with large websites, the pages you scrape won’t be totally identical with one another.

Hence, error or exception handling is very, very important.

Without proper exception handling, you may face errors after errors and waste time, as any error will simply halt your scraping work.

This is especially bad when you have set up your scraping task to take place over lunch or overnight.

Missing Element

Sometimes, certain pages do not have a certain element, which is very common.

For example, you might be scraping Amazon for products’ reviews. Some products that do not have any reviews simply do not have any review element to show you.

This can be easily solved by appending a «» or None to the list that you’re populating your scrape result.

from selenium.common.exceptions import NoSuchElementException

reviews = []

for page in pages_to_scrape:
  try:
    # Insert your scraping action here
    reviews.append(driver.find_element_by_css_selector('div.review').text)
  except NoSuchElementException:
    # Just append a None or ""
    reviews.append(None)
    

Timeout Exception

Some websites are simply slow or too small that your scraping has caused their server to overload.

The latter is always bad and you shouldn’t crash other people’s websites when scraping.

In any case, timeouts (i.e. page failed to load) are common on large websites.

However, I won’t recommend catching this error unless you know from experience that the website is prone to timeouts. It can be a huge waste of resource if the website has a 0.01% chance of timeout.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

driver.get("your url")
# Remove the while loop and break if you don't want to try again when it took too long
while True:
  try:
    # Define an element that you can start scraping when it appears
    # If the element appears after 5 seconds, break the loop and continue
    WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR, "your selector")))
    break
  except TimeoutException:
    # If the loading took too long, print message and try again
    print("Loading took too much time!")

Element Click Intercepted Exception

This error happens when the element you try to click (e.g. the «next page» button) gets blocked by another element and becomes unclickable.

The common cause of this is a pop-up being triggered, or there is a chat box that appears at the bottom-right of the page.

There are a few ways to solve this. The first way is to close the pop-up.

from selenium.common.exceptions import ElementClickInterceptedException

try:
  # Tries to click an element
  driver.find_element_by_css_selector("button selector").click()
except ElementClickInterceptedException:
  # If pop-up overlay appears, click the X button to close
  time.sleep(2) # Sometimes the pop-up takes time to load
  driver.find_element_by_css_selector("close button selector").click()

You may also use Javascript to hide that element (credit to Louis’ StackOverflow answer):

from selenium.common.exceptions import ElementClickInterceptedException

try:
  # Tries to click an element
  driver.find_element_by_css_selector("button selector").click()
except ElementClickInterceptedException:
  element = driver.find_element_by_class_name("blocking element's class")
  driver.execute_script("""
            var element = arguments[0];
            element.parentNode.removeChild(element);
            """, element)

If it’s not a pop-up, the problem could be solved by scrolling away, hoping that the blocking element moves with you and away from the button/link to be clicked.

from selenium.common.exceptions import ElementClickInterceptedException

try:
  # Tries to click an element
  driver.find_element_by_css_selector("button selector").click()
except ElementClickInterceptedException:
  # Use Javascript to scroll down to bottom of page
  driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Stale Element

Stale element happens when the element is was deleted or no longer attached to the DOM. 

Though a not very common error, some websites are prone to having this error.

When encountered with this error, you can just try again for a number of times.

from selenium.common.exceptions import StaleElementReferenceException

while True:    
  try:
    item_list.append(driver.find_element_by_id("item id").text)
  except StaleElementReferenceException:
    continue # If StaleElement appears, try again
  break # once try is successful, stop while loop

Best Practice — Error Handling & Exceptions

Only use necessary error exceptions

Catching errors and using these exceptions slows your code down significantly. It can cause a regular 1-hour scraping task to double in time especially if you wrap all your scraping actions with error exceptions.

Sometimes it’s totally inescapable, and that’s when a full coverage of error exceptions is necessary so that you can run it overnight without any worries.

However, I recommend to do a trial and error on your code and implement error exceptions only when you encounter them.

Start with a small, representative sample of your web pages to find out the type of errors that can happen.

Use placeholder element before appending to a list

If you’re scraping a few elements in a row, it’s best to assign the result of the scrape to a temporary, placeholder variable before appending to a list.

This is because if you’re appending all to their respective lists, an error in the later stages will cause your prior appended lists having more elements.

# ====================
# Use this way
# ====================
try:
  item_one_temp = driver.find_element_by_id("item one id").text
  item_two_temp = driver.find_element_by_id("item two id").text
  item_one.append(item_one_temp)
  item_two.append(item_two_temp)
except NoSuchElementException:
  item_one.append(None)
  item_two.append(None)

# ====================
# Instead of this
# ====================
try:
  item_one.append(driver.find_element_by_id("item one id").text)
  # if the next element does not exist, item_one list is already appended
  # i.e. one of your list is longer than another
  item_two.append(driver.find_element_by_id("item two id").text)
except NoSuchElementException:
  item_one.append(None)
  item_two.append(None)

Specify specific errors, unless your action plan for any error is the same

It’s useful to specify specific errors that you expect as different errors require different treatments.

Only use a broad «except» statements if you’re planning to do the same if any error arises.

I wish that I know all this before I started scraping as I had to waste a lot of time on StackOverflow. Hope this guide has been useful