Midterm Evaluations Are Close!

So, in the past two weeks, upon discussion with my mentors, I started working on the stretch goal, which involves using Astroquery’s TAP Plus for making the URL calls. Additionally, I had to clean up some code and remove the instruments SPICE and PHI from returning wavelength data due to their individual issues.

For SPICE, the `wavemin’ and `wavemax` values are only available for the first spectral window in SOAR, so the data is not entirely representative of what a user might be searching for. Similarly, for PHI, the issue is that two different units are used to represent wavelength, which results in very
different `wavemin’ and `wavemax` data due to the data being in both angstrom and nanometer units. Unfortunately, this is not specified anywhere in SOAR, so to ensure that `sunpy-soar` does not
return any misleading data to the user, we’ve decided to exclude the wavelength data for these instruments.

Coming to the change in table fetching methods. TAP Plus basically uses SQL to fetch tables, which is different from the normal TAP that uses ADQL to fetch tables. This avoids the entire process of manually constructing Astropy tables after calling the SOAR API with TAP.

For comparison, this is how the method looks with TAP:

@staticmethod
    def _do_search(query):
        """
        Query the SOAR server with a single query.

        Parameters
        ----------
        query : list[str]
            List of query items.

        Returns
        -------
        astropy.table.QTable
            Query results.
        """
        tap_endpoint = "http://soar.esac.esa.int/soar-sl-tap/tap"
        payload = SOARClient._construct_payload(query)
        # Need to force requests to not form-encode the parameters
        payload = "&".join([f"{key}={val}" for key, val in payload.items()])
        # Get request info
        r = requests.get(f"{tap_endpoint}/sync", params=payload)
        log.debug(f"Sent query: {r.url}")
        r.raise_for_status()

        # Do some list/dict wrangling
        names = [m["name"] for m in r.json()["metadata"]]
        info = {name: [] for name in names}

        for entry in r.json()["data"]:
            for i, name in enumerate(names):
                info[name].append(entry[i])

        if len(info["begin_time"]):
            info["begin_time"] = parse_time(info["begin_time"]).iso
            info["end_time"] = parse_time(info["end_time"]).iso

        result_table = astropy.table.QTable(
            {
                "Instrument": info["instrument"],
                "Data product": info["descriptor"],
                "Level": info["level"],
                "Start time": info["begin_time"],
                "End time": info["end_time"],
                "Data item ID": info["data_item_id"],
                "Filename": info["filename"],
                "Filesize": info["filesize"],
                "SOOP Name": info["soop_name"],
            },
        )
        if "detector" in info:
            result_table["Detector"] = info["detector"]
        if "wavelength" in info:
            result_table["Wavelength"] = info["wavelength"]
        result_table.sort("Start time")
        return result_table

This is how the method looks with Astroquery TAP plus.

`def _do_search(query):
        """
        Query the SOAR server with a single query.

        Parameters
        ----------
        query : list[str]
            List of query items.

        Returns
        -------
        astropy.table.Table
            Query results.
        """
        tap_endpoint = "http://soar.esac.esa.int/soar-sl-tap/tap"
        sql_query = SOARClient._construct_payload(query)
        soar = TapPlus(url=tap_endpoint)
        job = soar.launch_job_async(sql_query)
        results = job.results
        new_colnames = {
            "instrument": "Instrument",
            "descriptor": "Data product",
            "level": "Level",
            "begin_time": "Start time",
            "end_time": "End time",
            "filename": "Filename",
            "filesize": "Filesize",
            "soop_name": "SOOP Name",
        }
        new_colnames.update({k: k.capitalize() for k in ["wavelength", "detector"] if k in results.colnames})
        for old_name, new_name in new_colnames.items():
            results.rename_column(old_name, new_name)
        results.sort("Start time")
        return results

Even the query construction methods, they looks a lot more cleaner. As the final query generated is an SQL query not an ADQL query.