How to debug your app
-
# Pay close attention to the bug itself and try think why it is happening.
E.g. I had this weird looking issue with my code. I was configuring my RabbitMQ client and server using a helper function to generate
queueOptions
thus having same options. You know that your client app and server should use same queue options in order to be able to see each other and send messages to each other.Thus far everything was OK, except that I was using TS’s “Indexed Access Types”:
RmqOptions['options']['queueOptions']
but to my surprise it won’t throw an error if you use invalid object properties. As such I was passing some wrong values and myqueueOptions
was nested inside a property that was not correct.So my app was not bootstrapping the queue with the right configurations. Therefore when my
ClientProxy
was trying to publish a message in that queue, my app was crashing. As you can see most of the times changes that we make can be the root of an issue.
How to debug flaky tests effectively
First watch this YouTube video about flaky test which says that there are not flaky tests. Now let’s talk about different types of flakiness:
- Randomly passing and failing. Might be happening because of:
- Test order: test 1 has some side affects on test 2. They are not properly isolated.
- Some non-deterministic codes1.
- External dependencies.
- Concurrent processes.
- Asynchronous code.
- Environmental. Can be mitigated by a lot through using Docker. But nonetheless it could happen due to these reasons:
- Networking issues.
- Hardware differences.
- Subtle runtime differences.
- Dependency version difference.
[!TIP]
In Gitlab we have this post, in which we can learn about different scenarios we might end up dealing with flaky tests. It is a very good one, read it for sure.
How to identify them?
- Your VCS has some sort of tool set that shows you flaky tests. E.g. in GH we can use this Github Actions to do it.
Steps you can consider as your next action
-
RTFM: Whatever that’s causing the test to fail, a tool, a lib, or software.
No offense but I guess it is crystal clear to all of us that reading doc for 1 hour can save 6 hour of debugging. Numbers are fictional but I guess you got the message.
- Read logs.
- See what you’ve changed since last successful executed tests.
- Break it into smaller parts.
- Isolate the issue in a new, and isolated environment. Then try to reproduce it.
- Use
eslint
to find out about potential bugs and errors. - If you’re mocking something make sure that you’ve done it properly.
Some YouTube videos about flaky tests
- Flaky Tests.
- Eliminating Flaky Tests.
- Flaky tests by Andrei Solntsev.
- GTAC 2015: Your Tests Aren’t Flaky.
- [VDZ22] Removing complexity from integration tests using Testcontainers! by Kevin Wittek.
Footnotes
-
Non-deterministic code is code which can produce different outputs even when it is given the same inputs. ↩