Mocha - I built an opinionated TestRail reporter

Just a quick post to discuss my first foray into both functional and asynchronous programming...

As part of a project I am currently working on, a requirement was placed on each dev team, by the head of QA; all BDD tests must be captured in a central repository and their results for test runs recorded. As a business we already have tests being captured in TestRail

As an aside, I loathe test management tools. My opinion is that tests document the code and should therefore only ever live alongside the codebase or if documenting abstracted user journeys then a separate central automation framework, not in an isolated second source of truth. But I also know when to pick my battles.

The problem

We are a good dev team, building microservices that are well tested. We have good API driven tests that document the behaviour of our microservices, and we test against mocked providers. We also have a common set of libraries for our microservices including our choice of test framework: Mocha.

"How do we publish our tests and results to TestRail?"

The Solution

Mocha has an extensible reporter, and as such a vast array of third party reporters exist. There even already exists a TestRail reporter for Mocha. The issue with this reporter is that the fundamental assumption is TestRail is the source of truth, presuming that a test is written in TestRail and then linked via testId to a Mocha test. This is the opposite of what I want.

Thus, the very humbly named mocha-testrail-advanced-reporter, was born!

How does it work?

This is best answered by illustrating the resulting map of tests in Mocha to TestRail:

Mocha

describe('Array', () => {  
  describe('#indexOf()', () => {
    it('should return -1 when the value is not present', () => {
      assert.equal([1,2,3].indexOf(4), -1);
    });
  });
});

TestRail

Here the nested test suites in mocha have been squashed to form the section heading and the test case title has been added to the relevant section. Not only that, a test run with results is created:

Complexity

I have created a tool which will only create a suite, section or case if a matching one does not already exist in test rail. Otherwise, the existing assetId is mapped to the new test run.

This creates nested conditionals which I have attempted to resolve using Promises and PromiseAll. As mentioned in the beginning of this blog post, This is my first foray into functional and asynchronous programming, I am sure that I will review this code in 6 months and be utterly ashamed of myself... Not to mention I published this package without any tests!