GSOC 2020: Metadata searches using Fido

Missed Comet NEOWISE due to annoying cloud cover in Vadodara straight for 2 weeks :(

And that’s the central theme of the project :)

It would now be possible to query clients that return metadata tables using Fido. So SunPy’s Fido is a unified interface that allows searching and downloading solar physics data. In other words, it is a consistent and easy way to query most forms of solar physics data. It searches various archives and web services based on search attributes specified in the query.

SunPy currently supports metadata facilities viz., JSOC Client, HEK Client, and Helio Client.

SunPy’s hek module is used to access the information in the Heliophysics Event Knowledgebase (HEK). HEK helps solar and heliospheric researchers locate features and events of interest.

Joint Science Operations Center (JSOC) supports data products from various observatories and solar physics instruments.

But they were not Fido compatible for metadata searches. PR #4358 addresses it.

Generic Class for metadata clients

Since a lot of methods were similar in these clients, so I made a new superclass for them. JSOC, Helio, and HEK responses now inherit BaseQueryResponseTable to ease inspecting data retrieved through their clients. The idea was to retain the old look of response tables and also support a method to show all columns if required.

Deprecations

~sunpy.net.hek.attrs.Time is deprecated since we can now use ~sunpy.net.attrs.Time for HEK queries, making it redundant.

I also deprecated ~sunpy.net.jsoc.attrs.Keys because now the response table contains all keys by default. Users can specify the column names as *args in :meth:~sunpy.net.hek.HEKResponse.show for getting an ~astropy.table.Table instance containing only those columns.

Finally, HEKTable was renamed to HEKResponse for consistency in naming.

Devguide for writing Fido Clients

I have explained how to write a simple Fido client in PR 4387. This was the first documentation pull request that I made in SunPy. Work is in progress for adding details of writing an “AttrWalker” and registring an “Attr” for Fido.

Other Stuff

I reviewed PR #4394, which allows XRSClient to download reprocessed data for GOES Satellites. I also need to add support for this new pattern in my dataretriever refactoring pull request, #4321.

I also go through the discussions present in the net issues so I can fit their fixes in my project. I also suggest updates to the description of outdated issues and check if they still persist, like in Issue #2401.

Issue #2032 was fixed, so now helio wsdl_retriever returns the first valid taverna link.

We are going to now enter the last phase of the work period!

CARPE NOCTEM!