Wouldn’t it be awesome if you could automatically and accurately enter your timesheet everyday? I know it would save me at least a minute or two, and since I always forget, it would spare me a reprimanding email from my supervisor. If you are wondering, yes, I always forget to enter my timesheet, even though Binja, our timesheet software, automatically reminds me twice a day. So, how can we accomplish automating our timesheet entries?
This question doesn’t stray far from how we can approach automatically testing an application. Our goal is to write automated UI tests that evaluate the functionality of our app in an end-to-end fashion, the same way a user would use it. If we do this correctly, we should be able to find any issues and identify any bugs before deployments to production. It also increases our confidence that our software is working 100%. Now, let’s take a closer look into how we can do this with Cypress, a rival to Selenium, the most mature UI automation tool out there.
What is Cypress?
Cypress is a tool that helps developers and QA folks test their application. With a very easy set up, just run
npm install cypress --save-dev, and Cypress is installed and ready to use. If you’re not a developer, you might be asking, “What the heck did he just write?”
How does Cypress differ from Selenium?
Selenium is a powerful tool and in my opinion, much more mature when it comes to UI testing. Cypress is still in its infancy, although relatively mature at the current time of this writing. Cypress runs as part of your application directly in the browser, whereas selenium works over a network. This is a completely different approach, which provides the following advantages that I find to outweigh the disadvantages of this tool.
- Unit testing
- Integration testing
These three features are (to me) the the reason why Cypress is the tool that does complete end-to-end testing. What truly sets Cypress apart from Selenium is that it allows us to monitor the application for any API calls that are happening. For instance, I just submitted my entry for today’s timesheet. When I did that, there was an API call behind the scenes that posted my entry to the server and validated that the entry saved. With Cypress, we can test adding an item to our timesheet for the day, but we can also test to see that the API call to save our timesheet is successful or not based on our expectations. We expect that when we submit our timesheet, we will see a POST request. We also expect a specific response. To test this fully end-to-end, we use Cypress. If we are developers and don’t want to test against real data, we can use fixtures to set our expected responses. We can’t do all of this with Selenium.
But wait. You don’t trust the tool actually passes the tests that it says it passes? Cypress has an awesome UI that you can use to debug tests, see snapshots of each action that you perform in your test, see valuable, readable error messages, and also save videos of your test runs. Yes, you can watch your tests run through. You can even save them to the cloud. Selenium can handle screenshots but that’s about it. Debugging tests is a pain, you can’t record videos, and you usually have no idea what the errors mean (unless you are a seasoned developer).
Cypress automatically waits for the page to load, and never has stale elements, which two of the biggest issues QA engineer and developers experience when using Selenium. I found this statement on their website hard to believe, but after testing the web interface, it really does hold up to its claims. Not once have I had an issue waiting for a page to load or dealt with stale elements in the DOM within our application. During the following Binja example, I ran into a snag with waiting, but since this is not the application that we are focused on, I will give Cypress a pass.
***The following video was created by running Cypress via command line. It ran a headless Chrome browser and saved the test run.***
Some challenges I have come across while testing the web interface are mainly around downloading and uploading files to our application. Currently, I am working on a way to fully test some of these features, but must say it is not very intuitive. I plan to create some external scripts that will be run during the Cypress test to check for files on the system after the download happened. This actually is another advantage of Cypress. I can just run cy.exec(‘npm run checkForDownloadedFile’). This runs a script called checkForDownloadedFile that I declare in my package.json. I plan to use similar functions for cleaning up data in the database.
We eventually chose to use Cypress for all the reasons I list above but how we came to using it was almost happenstance. I joined SemanticBits about three months ago, knowing that I needed to standup test automation for my team. Naturally, I thought to myself, “Ok, I am going to use Selenium. Let me just see what languages the development team uses for the application, learn that language, and start building the framework.” I started by doing some API automation as the UI was secondary. However, our QA team had a revelation that UI testing was deemed more important than API testing.
The only negative for me was that Cypress didn’t test all browsers. This was something that was brought up as a major concern. Currently 50% of our users are using chrome, and 25% are using some versions of Internet Explorer. The other 25% are a combination of Firefox, Safari, and Opera. Our thinking was that if our tool can cover half of our users, then that is far better than anything we had before in terms of automation, because there was no automation. Right now, we are all aboard on Cypress.