Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for NNJA-AI Data #128

Open
jacobbieker opened this issue Jan 13, 2025 · 6 comments · May be fixed by #129
Open

Add support for NNJA-AI Data #128

jacobbieker opened this issue Jan 13, 2025 · 6 comments · May be fixed by #129
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@jacobbieker
Copy link
Member

BrightBand has released the NNJA-AI dataset now here: https://www.brightband.com/data that is cloud-optimized data of observations from NOAA's data assimilation process.

Detailed Description

Context

This could be a really nice way of training on pure observations or just including observations in a consistent format.

Possible Implementation

They have a package that supports querying and getting the data. This would be wrapping that in a dataset to be able to load it on the fly, or generate examples on disk.

@jacobbieker jacobbieker added enhancement New feature or request good first issue Good for newcomers labels Jan 13, 2025
@yuvraajnarula
Copy link

@jacobbieker can I work on this issue?

@jacobbieker
Copy link
Member Author

Yeah, go right ahead!

@yuvraajnarula
Copy link

hey @jacobbieker, I was trying to work on this issue and I had a doubt related to this do you want more of a nnja-ai wrapper for the dataloader.py and show observation in the form of graph?

@jacobbieker
Copy link
Member Author

Hey, yeah, what we want is a data loader to returns the observations and their Metadata. Visualizing it would be helpful but not needed. But we do want it to be an output of shape (time, latitude, longitude, metadata-channels) for each observation.

@yuvraajnarula
Copy link

@jacobbieker

Image
In the first output, I tried a batch size of 16; in the second one, I tried without it and ran a loop to show you the output of the first iteration.

@jacobbieker
Copy link
Member Author

Hey, yeah, exactly like that!

@yuvraajnarula yuvraajnarula linked a pull request Jan 20, 2025 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants