Cypress is superior to React Testing Library — here’s why
Automated testing is a crucial step in engineering robust software, but knowing which technologies to utilize can be a laborious endeavor. This article aims to demystify the differences between two of the most popular React testing solutions: Cypress and React Testing Library.
What is Cypress?
Cypress is an automated testing tool used for end-to-end, integration, and component testing.
Cypress launches a browser (either headful or headless depending on your config), which provides benefits such as:
- Cross browser testing support
- Browser dev tools for debugging
- More confidence tests are running against what the user will see
Cypress ultimately controls the entire automation process from top to bottom, which puts it in the unique position of being able to understand everything happening in and outside of the browser. This means Cypress is capable of delivering more consistent results than any other testing tool. — cypress.io
What is React Testing Library?
RTL is a set of testing utilities built for testing React components. RTL is often used in conjunction with a testing framework to get a more complete set of testing functionality. In this article, we will be using RTL with Jest + JSDOM, as that is the most widely used toolchain.
Using this setup, the components are rendered in a virtual DOM that is meant to replicate the real DOM. The main benefit this provides over rendering in a real browser is faster execution speeds (generally).
The React Testing Library is a very light-weight solution for testing React components. It provides light utility functions on top of react-dom and react-dom/test-utils, in a way that encourages better testing practices. — testing-library.com
Integration Testing
We’ll be testing a small application that uses React + GraphQL. The application is just a CRUD application for managing monthly reminders. Both the Cypress and RTL implementations run through the same queries and assertions.
Cypress Implementation
This is the resulting test suite for the Cypress implementation. The only external plugin used for this test was the cypress-testing-library plugin.
Cypress Results
React Testing Library Implementation
This is the resulting tests for the RTL implementation. The only external library used for this test was MSW for mocking service responses. While there are other ways of mocking network requests, MSW is nice as it allows you to get further from the implementation details of your code base.
React Testing Library Results
Integration Test Comparison
Performance
Winner: Cypress*
For this test, the Cypress implementation managed to execute twice as fast when compared to the RTL implementation. However, this result will vary depending on the specific dependencies used, and how tests are parallelized.
I was curious if additional test cases would change the outcome, so I looped over the tests and ran 100 integration test cases. This many integration test cases in a single file isn’t a common real life scenario, but it can provide some context around performance. The result was the execution time of RTL narrowing to 120% from around 200% (twice the execution time) when compared to Cypress.
Test Setup
Winner: Cypress
With RTL you’ll likely end up mocking dependencies while writing tests (such as ‘next/router’ in the example above). This can be time-consuming, excruciating, and will require maintenance as dependencies are updated. Cypress on the other hand runs against the running application, which allows it to be further away from the implementation details of the code base.
Debuggability
Winner: Cypress
Cypress’s time travel debugging along with the ability to use Chrome/Firefox developer tools is a sanity saver when debugging. As a result, you can expect to spend far less time debugging issues compared to RTL. RTL does include debug and logTestingPlaygroundURL, but the difference in debuggability is still stark.
Dev Experience
Winner: Cypress (personal opinion)
Both implementations are very close in this category. What pushes Cypress ahead for me is the way it implements its queries and assertions with chaining, how much simpler custom utilities are to implement compared to RTL. This, along with cypress-testing-library bringing in the best part of the RTL implementation, makes Cypress the clear winner in my opinion.
Unit Testing
Unit tests focus on small individual pieces of functionality, such as a button component. We’ll be adding unit tests to an icon button component for both Cypress and RTL, and then we’ll compare the results between the two implementations. To give us better benchmarks, we’ll then loop over the test 100 times.
Cypress Implementation
This is the resulting tests for the Cypress implementation. The only external plugin used for this test was cypress-testing-library.
Cypress Results
React Testing Library Implementation
This is the resulting tests for the RTL implementation.
React Testing Library Results
Unit Test Comparison
Speed
Winner: React Testing Library
When running the icon button tests 100 times, RTL won handily in terms of execution time — running in just one third of the time.
Test Setup
Winner: tie
Unlike with integration testing, unit testing with Cypress includes the same pitfalls as RTL in regards to dependency mocking and test setup. This is because you are now rendering and testing components, instead of the actual application.
Debuggability
Winner: Cypress
Debugging unit tests is the same as integration testing — time travel debugging and being able to use browser dev tooling give Cypress the edge.
Dev Experience
Winner: Cypress (personal opinion)
The dev experience of implementing unit tests is the same as integration testing — the implementations of queries, assertions, and custom utilities are better in Cypress.
Ecosystem
The ecosystems for both Cypress and RTL + Jest are both great. The community support for both ecosystems generally ensures that there are plugins or information available for most functionality that you would want (such as plugins for functionality such as visual regression testing or accessibility testing). That said, Cypress’s plugin system for ease of use and discoverability (cypress plugins) does give it the slight edge in my opinion.
Conclusion
I came into this project expecting to recommend Cypress for certain use cases and RTL for other use cases. In the end, I struggled to find a good reason to use RTL when compared to Cypress. At this point in time, I really can’t see myself choosing RTL with Jest + JSDOM over Cypress for any type of testing. That said, the issues are more with the overall toolchain, and not RTL itself. The principles from Testing Library (the package RTL is based on) are outstanding to the point that my most recommended plugin for Cypress would ironically be cypress-testing-library. Even though the performance is slightly better or comparable for RTL in most scenarios, Cypress was the clear winner in test setup, debuggability, and dev experience.
