Universe OpenAstronomy (Posts by Deus1704)

Blog 4: Testing the New Coalignment with a Catch

Deus1704 — Mon, 22 Jul 2024 17:59:31 GMT

Now that we’ve agreed on the structure of the co-alignment API and also laid down its foundation, all that’s left is to validate it through actual tests and generate some gallery examples.

First try with coaligning EIS raster with AIA map

Let’s take a closer look at the EIS raster first. I found an AIA image close to the date_average of the raster.


EIS raster	AIA Full-disc image


Updated EIS

At first glance, the results seemed promising, but a more detailed analysis revealed some discrepancies.

The updated WCS metadata looked fine at first, but on closer inspection, it wasn’t entirely correct. We’ll dive into the specifics later in this blog. For now, let’s discuss why it might look correct at a glance. How does a common user check if the maps are aligned? By overlaying them and checking the overlaps!


EIS overlaid on AIA	Zoomed

This visual inspection might convince some that the method works, but it’s far from accurate. We proved this by highlighting the bright regions and focusing on the actual overlaps.


EIS contours overlaid on AIA

This showed us that something was wrong with either the way we were updating the metadata or the co-alignment method. While discussing this, we realized that an assumption in the match_template method was that the maps should have the same type of WCS.

The Celestial Fix

In our initial approach, we focused on adjusting the CRPIX values instead of the CRVAL values. CRPIX specifies the position of the reference pixel in the image, shifting the image in pixel space without correcting the world coordinates directly. This method led to apparent alignment issues because it didn’t address the celestial coordinate system. After consultation, we realized that adjusting CRVAL, which defines the world coordinates of the reference pixel, is essential. This adjustment ensures that the reference points align correctly in world coordinate space, maintaining consistent mapping between pixel and celestial coordinates.

Furthermore, for accurate co-alignment, the observers (instruments) should be within a tolerable angular separation to minimize parallax effects. When observers are too far apart, solar features can appear differently due to relative positions, complicating alignment. By correcting CRVAL values and considering angular separation, we achieved precise co-alignment, as evidenced by the corrected overlay of EIS contours on the AIA image, allowing for reliable scientific analysis.


EIS contours overlaid on AIA (corrected)

Blog 3: Side Quests Week

Deus1704 — Wed, 03 Jul 2024 10:59:31 GMT

While the mentors are waiting for community reviews on the new coalignment API, I’ve been diving into some fun side quests.

Side Quest 1: ASDA Example

The long-needed Automated Swirl Detection Algorithm (ASDA) example gallery in the sunkit-image is now ready. The ASDA module is for identifying the swirls or vortices in the 2D flow field of the solar atmosphere. ASDA offers a robust tool for detecting and analyzing the vortices in the solar atmosphere.

Swirls in the solar atmosphere provide insights into dynamic solar processes such as solar flares and coronal mass ejections. Detecting and analyzing these swirls helps understand the mechanisms driving these solar activities.

The Gamma(Γ1 & Γ2) values are used for identifying the vortex center and the vortex edges respectively.

Gamma1 (Γ1): Identifies vortex centers by quantifying rotational motion.
Gamma2 (Γ2): Detects vortex edges by measuring the coherence of swirling motion.

These can be visualised as;


Gamma values (Γ1 & Γ2)

Using these, the final map with the swirls identified looks like:


Swirl Map with Velocity Field

The swirl map is crucial for visualizing fluid flow dynamics in the solar atmosphere, helping to identify the distribution, size, and characteristics of vortices.

Magnifying a particular section of these for better understanding of the streamlines:


Magnified Swirl Map Region with Streamlines

Magnifying a specific region and overlaying streamlines allows for detailed analysis of flow patterns around swirls, aiding in understanding solar atmospheric dynamics and the interactions between different vortices.

Side Quest 2: Rotation Matrix

One of my very first pull requests in SunPy received reviews after a long hiatus. The feedback highlighted some implementation issues with the way the SpicePy API was being used. For instance, the sxform function only accepts a single ephemeris time, which required a workaround to obtain the correct state transformation matrix. With the invaluable help of Albert, these issues have been resolved and the test cases have been updated accordingly.

Blog 2: All Pieces Falling into place

Deus1704 — Thu, 20 Jun 2024 09:59:31 GMT

Update On New Coalignment

The very first version of the new Coalignment API has been completely implemented, with all the test cases written! The architecture of the proposed co-alignment API closely mirrors the design principles found in SunPy’s rotation module, where a decorator pattern is employed to register specific methods designated for executing co-alignment tasks. Detailed documentation and guidelines on utilizing the new API are available for review at this docs.

Ideas regarding the Future of Coalignment & Major Concerns

We have received numerous comments concerning the new co-alignment API, with the majority emphasizing the revision of the coalignment methods rather than the API structure itself. Consequently, I will be delving into the specific modifications occurring at the World Coordinate System (WCS) level throughout the coalignment process and the request of returning only the shifts and not applying the shift to the actual data.

Although we are afraid that the suggestiong might be aiming to reinvent the reproject but again the fundamentally the approach for the two is different and needs to be thought of in brief before taking any major steps. We might be discussing this issue in the next community meet once Stuart is back.

Update on STARA

The STARA module and the examples are completely ready, after along time of resolving conflicts, planning testing. We have come a long way with these test figures.


Test HMI Continuum Map	Sunspot identified by STARA on the Test Map

Blog 1: Rethinking Coalignment from Ground-up

Deus1704 — Sat, 15 Jun 2024 13:59:31 GMT

The Very Essence of Coalignment

At its core, coalignment is all about making sure our solar images match up as best they can. It uses a few different techniques, like template matching and solar rotation correction, to get this done. This is really important when we’re trying to accurately track and study what’s happening on a part of sun over time. We re-thought the entire process of what our new API be providing to the users as well as its structure.

This was one of the idea that was very naive version, but focused entierly for the user’s comfort in applying the coalignment methods.

The Basic Structure we finally agreed Upon

We agreed to have this as the very basic structure which we would be working/developing upwards. The following example demonstrates the way it would work.

aia_map1 = sunpy.map.Map(sunpy.data.sample.AIA_193_CUTOUT01_IMAGE)
aia_map2 = sunpy.map.Map(sunpy.data.sample.AIA_193_CUTOUT03_IMAGE)
### Creating a template from aia_map1
bottom_left = SkyCoord(600 * u.arcsec, -500 * u.arcsec, frame=aia_map1.coordinate_frame)
top_right = SkyCoord(800 * u.arcsec, -200 * u.arcsec, frame=aia_map1.coordinate_frame)
submap = aia_map1.submap(bottom_left, top_right=top_right)

coaligned_map = coalignment_interface("match_template",aia_map2, submap)

I am currently implementing the decorator structure for the sunkit-image, but that would be covered in another blog.

Blog 0: The First Two Weeks

Deus1704 — Sat, 25 May 2024 13:59:31 GMT

A summary of activities completed in the two weeks leading up to the start of the coding period.

After completing my end-semester exams, I was overwhelmed by the outstanding tasks that I had let accumulate over time. In order to address those, I tried resolving dependencies and package issues of the previous environments of sunpy & sunkit-image but, ultimately gave up and initialised a new environment. After discussions with mentors, the project priorities were clarified, allowing me to efficiently plan & complete my tasks during the community bonding period. What follows is the detailed account of all the tasks completed during these two weeks.

Incorporating suggestions for Sunspot Tracking and Recognition Algorithm (STARA)

Incorporated suggestions and made several changes to the STARA example. Attempted to create a mock HMI continuum data with an artificial “sunspot” at a chosen location with a certain radius. But it did not meet the exact requirements of the STARA, hence no regions could be identified. The written mock hmi map are demonstrated below,


Mock HMI Map without Sunspot	Mock HMI Map with Sunspot

Discussed alternative approaches with mentor Nabil regarding the mock HMI map. According to his suggestion I tried the HMI continuum test data present in sunpy/data/test directory, but it had some issues with the WCS hence STARA couldn’t properly find the sunspot regions.

Wrapping up the draft pull request

I began working on the transformation of a vector field some time ago but encountered challenges in validating the approach. Using Astropy for the transformation confirmed the accuracy of the function I wrote. The next step is to seek feedback from mentors and the SunPy community. But I still feel a need to test this more rigorously which can validate that this would work with different frames too. I studied about different frames provided by the SPICE toolkit and tried to match the sunpy frames and the SPICE frames and have confirmed that the function works with the static frames.

Fixing the doc failures

Two out of my three open pull requests encountered documentation failures due to incorrect referencing of functions or modules, or example galleries. After studying the conventions of Sphinx and SunPy, I resolved these issues. Additionally, I discovered how to set up GitHub CI actions on my forks.

First Meet with Mentors

In our initial meeting, my mentor Will provided a brief overview of the purpose of sunkit-image, emphasizing its primary function: coalignment. He explained the fundamental concepts of coalignment, user expectations, and desired functionalities of the new API. We also reviewed the current coalignment API’s shortcomings. We agreed that an improved API was necessary and discussed my proposed structure for this enhancement. We also drafted a plan for the entire rethinking and redesigning the new API over the span of 4 weeks.

Blog 0: The First Two Weeks

Deus1704 — Fri, 24 May 2024 23:00:00 GMT

/*@jsxRuntime automatic @jsxImportSource react*/ const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0]; const {useMDXComponents: _provideComponents} = arguments[0]; function _createMdxContent(props) { const _components = Object.assign({ hr: "hr", h2: "h2", a: "a", span: "span", ul: "ul", li: "li", code: "code" }, _provideComponents(), props.components); return _jsxs(_Fragment, { children: [_jsx(_components.hr, {}), "\n", _jsxs(_components.h2, { id: "title-blog-0-the-first-two-weeksdate-2024-05-25featured-description-a-summary-of-activities-completed-in-the-two-weeks-leading-up-to-the-start-of-the-coding-period", children: ["title: 'Blog 0: The First Two Weeks'\ndate: '2024-05-25'\nfeatured: ''\ndescription: 'A summary of activities completed in the two weeks leading up to the start of the coding period.'", _jsx(_components.a, { "aria-hidden": "true", tabIndex: "-1", href: "#title-blog-0-the-first-two-weeksdate-2024-05-25featured-description-a-summary-of-activities-completed-in-the-two-weeks-leading-up-to-the-start-of-the-coding-period", children: _jsx(_components.span, { className: "icon icon-link" }) })] }), "\n", _jsxs(_components.h2, { id: "table-of-contents", children: ["Table of Contents", _jsx(_components.a, { "aria-hidden": "true", tabIndex: "-1", href: "#table-of-contents", children: _jsx(_components.span, { className: "icon icon-link" }) })] }), "\n", _jsxs(_components.ul, { children: ["\n", _jsxs(_components.li, { children: ["\n", _jsx("a", { href: "#draft_pr", style: { textDecoration: 'none' }, children: "Wrapping up the draft pull request" }), "\n"] }), "\n", _jsxs(_components.li, { children: ["\n", _jsx("a", { href: "#issue1", style: { textDecoration: 'none' }, children: "Incorporating suggestions for STARA & its examples" }), "\n"] }), "\n", _jsxs(_components.li, { children: ["\n", _jsx("a", { href: "#fix_doc_fails", style: { textDecoration: 'none' }, children: "Fixing the doc failures" }), "\n"] }), "\n", _jsxs(_components.li, { children: ["\n", _jsx("a", { href: "#first_meet", style: { textDecoration: 'none' }, children: "First Meet with Mentors" }), "\n"] }), "\n"] }), "\n", _jsx("p", { children: "After completing my end-semester exams, I was overwhelmed by the outstanding tasks that I had let accumulate over time.\nIn order to address those, I tried resolving dependencies and package issues of the previous environments of\nsunpy & sunkit-image but, ultimately gave up and initialised a new environment. After discussions with mentors, the project priorities were clarified, allowing me to efficiently plan & complete my tasks during the community bonding period. What follows is the detailed account of all the tasks completed during these two weeks." }), "\n", _jsx("h2", { id: "issue1", children: "Incorporating suggestions for Sunspot Tracking and Recognition Algorithm (STARA)" }), "\n", _jsx("p", { children: "Incorporated suggestions and made several changes to the STARA example. Attempted to create a mock HMI continuum data with an artificial \"sunspot\" at a chosen location with a certain radius. But it did not meet the exact requirements of the STARA, hence no regions could be identified. The written mock hmi map are demonstrated below," }), "\n", _jsxs("figure", { style: { display: 'flex', flexDirection: 'row', alignItems: 'center' }, children: [_jsx("img", { src: "/images/mock_wo.png", width: "400", height: "400", style: { margin: '0 auto' } }), _jsx("img", { src: "/images/mock_withspot.png", width: "400", height: "400", style: { margin: '0 auto' } })] }), "\n", _jsxs("p", { children: ["Discussed alternative approaches with mentor Nabil regarding the mock HMI map. According to his suggestion I tried the HMI continuum test data present in ", _jsx(_components.code, { children: "sunpy/data/test" }), " directory, but it had some issues with the WCS hence STARA couldn't properly find the sunspot regions."] }), "\n", _jsx("h2", { id: "draft_pr", children: "Wrapping up the draft pull request" }), "\n", "\n", "\n", _jsx("p", { children: "I began working on the transformation of a vector field some time ago but encountered challenges in validating the approach. Using Astropy for the transformation confirmed the accuracy of the function I wrote. The next step is to seek feedback from mentors and the SunPy community. But I still feel a need to test this more rigorously which can validate that this would work with different frames too. I studied about different frames provided by the SPICE toolkit and tried to match the sunpy frames and the SPICE frames and have confirmed that the fucntion works with the static frames." }), "\n", _jsx("h2", { id: "fix_doc_fails", children: "Fixing the doc failures" }), "\n", "\n", "\n", _jsx("p", { children: "Two out of my three open pull requests encountered documentation failures due to incorrect referencing of functions or modules, or example galleries. After studying the conventions of Sphinx and SunPy, I resolved these issues. Additionally, I discovered how to set up GitHub CI actions on my forks. " }), "\n", _jsx("h2", { id: "first_meet", children: "First Meet with Mentors" }), "\n", "\n", "\n", _jsx("p", { children: "In our initial meeting, my mentor Will provided a brief overview of the purpose of sunkit-image, emphasizing its primary function: coalignment. He explained the fundamental concepts of coalignment, user expectations, and desired functionalities of the new API. We also reviewed the current coalignment API's shortcomings. We agreed that an improved API was necessary and discussed my proposed structure for this enhancement. We also drafted a plan for the entire rethinking and redesigning the new api over the span of 4 weeks" })] }); } function MDXContent(props = {}) { const {wrapper: MDXLayout} = Object.assign({}, _provideComponents(), props.components); return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, { children: _jsx(_createMdxContent, props) })) : _createMdxContent(props); } return { default: MDXContent };

The trial post

Deus1704 — Mon, 13 May 2024 23:00:00 GMT

To code or debug that is the question