Skip to content

MJO xcdat conversion#1091

Merged
lee1043 merged 23 commits into
mainfrom
feature/1084_lee1043_MJO-xcdat
Jun 6, 2024
Merged

MJO xcdat conversion#1091
lee1043 merged 23 commits into
mainfrom
feature/1084_lee1043_MJO-xcdat

Conversation

@lee1043

@lee1043 lee1043 commented May 1, 2024

Copy link
Copy Markdown
Contributor

This PR will resolve #1084.

Draft PR made to track changes. PR is not ready to merge until it is explicitly marked as ready.

@lee1043 lee1043 self-assigned this May 1, 2024
@lee1043

lee1043 commented May 9, 2024

Copy link
Copy Markdown
Contributor Author

Progress note:

The new code was able to run the demo notebook without noticeable error, but with differences in result metrics.

A rigor test (run the code for all CMIP5/6 models/realizations) conducted.

  1. Calendar-related error for non-standard calendar using models is found.
  2. Some differences in the results are found. (Some sensitivity is not surprising because numbers used for the ratio (EWR) are very small.)

Screenshot 2024-05-08 at 6 01 23 PM

Code revision:

  1. To avoid date comparison error from non-standard calendar using models, more simple and straightforward date comparison is applied. It is confirmed the error from the non-standard calendar models is resolved.
  2. Common grid is changed from uniform grid to Gaussian grid to make the xCDAT code more consistent to the CDAT/CDMS version.

Next step:

  • Another rigor test (run the code for all CMIP5/6 models/realizations) is launched. Its result will be analyzed once the test run is completed.
  • Custom season capability when crossing calendar year is not been fully tested, and there to be potential issue (maybe using discontinued months, which could affect to NDJFMA season) → There should be no issue with custom season because for MJO it sub-slice segment using length of segment from the starting day of each year. (here)
@lee1043 lee1043 added this to the 3.5 milestone May 9, 2024
@lee1043

lee1043 commented May 14, 2024

Copy link
Copy Markdown
Contributor Author
  • Usage of more consistent grid (uniform --> Gaussian) did not dramatically changed results (Compare below with above, which are for first ensemble members of CMIP5 (upper row) and 6 (lower row) models).

  • CDAT vs xCDAT results seem to be acceptable (with high correlation and p<0.05), but more investigation for a few outliers would be helpful because their diagnostics fields are looking like identical (see second below figure).

Screenshot 2024-05-14 at 1 00 01 AM
Screenshot 2024-05-14 at 1 00 15 AM

…requency domain to be consistent with that of cdms version
@lee1043

lee1043 commented May 14, 2024

Copy link
Copy Markdown
Contributor Author
  • Progress note: after f455307, result of demo notebook 5 became much more consistent than before.
  • The 3rd round of the rigorous test (running across all CMIP5/6 models) is in progress.
@lee1043

lee1043 commented May 15, 2024

Copy link
Copy Markdown
Contributor Author

The above fix completed resolved the inconsistency! The 3rd round of he rigorous test results below.

  • East West Power Ratio:

Screenshot 2024-05-14 at 10 10 22 PM

  • East Power:

Screenshot 2024-05-14 at 10 10 31 PM

  • West Power:

Screenshot 2024-05-14 at 10 10 40 PM

@lee1043
lee1043 requested a review from acordonez May 15, 2024 05:17
@lee1043
lee1043 marked this pull request as ready for review May 15, 2024 05:17
@lee1043

lee1043 commented May 15, 2024

Copy link
Copy Markdown
Contributor Author

@acordonez I believe this PR is ready. Can you please try install code in this PR and run the demo 5 notebook to see if that works okay to you as well?

@acordonez

Copy link
Copy Markdown
Collaborator

@lee1043 Sorry for taking a few days to get to this! I ran the notebook and found no issues there. I remember you said it was expected to see some slight differences in the metrics? E.g. an east power of 0.016568144258499968 versus 0.016600374830362644?

@lee1043

lee1043 commented Jun 6, 2024

Copy link
Copy Markdown
Contributor Author

@lee1043 Sorry for taking a few days to get to this! I ran the notebook and found no issues there. I remember you said it was expected to see some slight differences in the metrics? E.g. an east power of 0.016568144258499968 versus 0.016600374830362644?

@acordonez thank you for reviewing. Yes, I consider that the difference you found is subtle enough to be neglected.

@lee1043
lee1043 merged commit b354fbe into main Jun 6, 2024
@lee1043
lee1043 deleted the feature/1084_lee1043_MJO-xcdat branch June 6, 2024 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

2 participants