GSoC @ Stingray: Testing Testing Testing …

After getting the code for bexvar in working condition, the next task in front of me was to test the code. As I mentioned in the last blog I was completely new to software testing. This meant a lot of new things to learn.

In this blog, I am going to write about my experience with testing. This was one of the best learning experiences of my GSoC journey so far. With the help of my mentors, I arrived at the conclusion that primarily there are three kinds of tests that would be needed for bexvar.

(1) Unit tests to check if various components of my code provide the correct outputs individually.

(2) Some functional tests to check if the entire bexvar() function works properly and gives the desired output.

(3) Some other tests to check if proper warnings and errors are raised in the case of bad inputs.

I started with the unit tests. Creating the unit tests for internal functions was comparatively easy. Two of the three internal functions provided deterministic outputs. All that was needed for testing them was to calculate the desired output and compare it with the output from the code.

One of the major problems arose while designing functional tests. The bexvar() function returns an array that contains the samples of the log of the bexvar (i.e. log(\sigma_{count rate})). These samples are proportional to the posterior distribution of log(\sigma_\{count rate}). These samples are generated by an MCMC generator and therefore the elements and the number of elements of the output array are different after each run for the same data. This makes testing the result more difficult as the elements of the array could not be compared with a predetermined result.

As the bexvar() aims to provide the posterior distribution of log(Sigma_\{count rate}) the elements as individuals do not matter but the shape of their histogram provides the posterior distribution. Therefore we decided to test the results with some statistical parameters derived from the generated samples which quantify the distribution.

The problems that I faced while designing the third type of test, helped me find and resolve multiple unseen issues with the code. It took me more time than I expected to complete the testing phase. However, I am glad to announce that the all tests are now passing successfully. Additionally, I also passed the mid-term evaluations! Phew!

At end of the testing phase, The PR was ready to be merged, after a few minor updates in the docstrings it was merged! Yaay! The work is not yet done though. Next, I am working to add a method to Stingray’s lightcurve class. In addition to this, I will be creating a tutorial page for using the bexvar and will also make some more improvements in the code if time permits.