Cypress, many experts believe, takes much of the pain out of testing. Gleb Bahmutov, a JavaScript ninja who helped develop Cypress in his previous job, gave a webinar on end-to-end testing of a real-time chat web application with this front-end JavaScript testing tool. Here follows a summary.
Web application based on Socket.io
How can you perform realistic end-to-end Cypress testing of a real-time chat application? That question is the focus of Gleb Bahmutov's online lecture. This 'JavaScript ninja' is currently senior director of engineering at Mercari US, was previously VP of engineering at Cypress and has written over 500 software development blog posts in his spare time.
In the webinar, of which code-heavy slides can be viewed here, he covers four separate points that all give a different perspective on your code: mock app code, mock socket connection, open second socket connection and running two test runners. The application Bahmutov discusses is a web application for chat based on Socket.io. He has placed the application's code on GitHub.
Does the application work?
First of all, Bahmutov raises the question: does the application work? "You can open it, but things get broken easily. Therefore, in reality, you have to test anyway and it is better to do that in an automated way. That's why we're going to write a Cypress test today." The JS guru explains that a typical end-to-end test normally follows the following pattern: command - assertion(s) - command - assertion(s).
On to the first test. "In Cypress you can really see everything, for example how the application looks in the real browser, the user name, et cetera. To be able to answer the question 'does the app really work?', we first have to ask ourselves whether a second user participating in the chat can see my messages and vice versa. After all, the application might mirror the messages, but not send them."
Test assistants
Bahmutov makes a small side-step: he explains that the 'problem' with this kind of test assistant is that they have to draw a line somewhere: which part do we test and which do we not? He continues: "From Cypress we can connect to a server of Socket.io and act as a second user. A normal test is performed on the page, another part of the code goes to the socket server and behaves like a second user. Another thing that can be done: open another Cypress test runner and let two browsers talk to each other."
Stub application code
"We start with something simple: stub application code. You can create a class or object in the application code, you can expose that application object to the test. From the test, you can then spy on the application object and test incoming messages." Bahmutov shows that the app displays UI messages that come in to the application code, the page then displays the messages.
Mock web socket
It has now been confirmed that the application code works up to the socket, only the socket commands could be wrong. "To test this, you can use a mock web socket. For that, you have to replace the prod web socket for the mock socket." Bahmutov points out that the intercept functionality of Cypress comes in handy here. Next, it's a matter of injecting the mock socket into the application's iFrame. "We have now verified that the socket API works within the application. That is, the UI code, the intermediary code and the web socket."
Testing whether the server works
While the socket API within the application may work, it could be that the server is broken. "During the test we want to check the page, go through the UI and connect and simulate the second user talking back and 'listening' to the messages by connecting to the same server." Bahmutov explains how this works and comes up with a tip: for the second user, use a connection from outside the browser page, as that is a cleaner way to make a second separate connection to the server.
Running two Cypress instances simultaneously
How do you run two Cypress instances side by side? For this you need two separate specs, both of which operate strictly through the page UI. Probably, as the test expert explains, you need to create separate Cypress configuration files since you are running two instances at the same time. Also, the NPM module must be installed concurrently.
According to Bahmutov, this method has a number of shortcomings at first glance. For example, Cypress test instances are not really waiting for one another, they are more or less blind to each other. So, you want to be able to control how the two test runners run the test. To synchronise the test runners, you can create a separate Socket.io. In doing so, you only need to implement two commands: checkpoint and wait for checkpoint. Then you need to synchronise the Socket.io server and implement the Cypress plugin file.
The first and second test runner
Bahmutov explains that the first test runner visits the page, sets the name and checks the server, then the first user joins the chat. Now the second user must be added to the chat. It then waits for the checkpoint that verifies that the second user has been added, after which the first test runner checks that the second user is actually there. He then has to post a message, so you can see if the message is visible.
The second spec file states that the first user must first reach the checkpoint. Then it is verified that the message comes from the first user, in the meantime the first test runner waits for this checkpoint. This is what, in a nutshell, the first test run looks like. Bahmutov: "In other words, first it starts, then it waits, then the messages from the second test round arrive, it connects, it waits and then the message is quickly communicated. On the second test run it is much quicker: it is connected, checkpoint, message seen and done. The two test runners always communicate with and wait for each other to make sure everything goes in the right order."
Conclusion: what's better?
"So, which is better?", Bahmutov concludes his webinar. "I showed four different ways to test code: stub app code, web socket mocking so you never have to run a server from Socket.io, acting as a second user through a connection to Socket.io that is isolated from the page and finally running two test runners side by side, also acting as a second user. In my opinion, acting as a second user, without starting the full test runner, is much better. After all, that is a full end-to-end test."
"The second browser communicates exactly as we would communicate through a connection to Socket.io. We only communicate via a public API and did not do anything confidential, as this was also not necessary." Moreover, the test is still very fast, assesses Bahmutov. "Less than two seconds to connect on both sides, load the page, exchange a few messages and finally confirm that the socket server is working, the page is shown and the page is sending things. Hence, this is my favourite way to test real-time chat applications."
Want to read more?
Want to dive deeper into the subject matter? Bahmutov has written three blog posts that go into this material in more detail: Test a socket.io chat app using Cypress, Run two Cypress test runners at the same time and Sync two Cypress runners via checkpoints.
