The last few weeks have been a whirlwind of ideas, coding, and rethinking as we figured out the best way to build my dashboard project.

Our First Idea:

My initial plan seemed pretty logical. I was going to download all the raw data (the "event files"), process them into lightweight h5 files full of handy PNGs, and stick them all in an S3 bucket. The dashboard would just grab these pre-made files.

It was a neat and tidy plan, but the more we thought about it, a static, deployed app . We decided to make something that a user could clone and run right on their own local machine.

The Next Step:

So, that's what we did. The new idea was to build a dashboard that anyone could just grab from a repository and run on their own computer. No big deployment , In this version, the user would download the source data they were interested in, and my dashboard would help them create the plots and data products right there on their machine.

But that created a whole new problem: asking people to download gigabytes of data and clog up their own hard drives. That didn't feel very user-friendly, and it was a major headache waiting to happen.

The "Aha!" Moment: Why Not Go Where the Data Already Lives?

This is when we had the real breakthrough. It was a total game-changer. Instead of moving massive datasets around, we realized we could just move our little dashboard to the data.

By running everything on SciServer, where all the telescope data already lives, the user doesn't have to download a single thing. Suddenly, the headache of huge downloads and local storage just... vanished. All the complex processing happens on SciServer's temporary "scratch" storage. It's faster, way more efficient, and saves everyone a lot of trouble. The only time we spend now is on the actual science—the calibration and corrections.

So, Where Are We Now?

I'm really happy to say that I have a working version of the dashboard running with this new, much smarter approach! All the major functions are in place.

To be honest, the user interface is still a bit basic—it looks like an engineer built it (which, well, I did). So my next big task is to clean it up and make it look more polished.

To give you a little peek, here’s what the initial draft looks like:

As we get closer to the mid-term evaluations, my goals are coming into focus. First up, I'll be syncing with my mentors, @matteobachetti & @Gullo, to make sure we're all on the same page with going all-in on SciServer. After that, I'm excited to start making the UI more dynamic and adding some cool new features.