NASA Cloud Modeling and Analysis Initiative (CMAI):
NASA Modeling Framework
First Work Plan Draft (21 Dec 2005)
- Comment from Donald Anderson, 21 Dec 2005
- Comment from Anthony DelGenio, 02 Jan 2006
- Comment from Bruce Wielicki and Kuan-Man Xu, 18 Jan 2006
- Comment from Bruce Wielicki, 18 Jan 2006
- Comment from Michele Rienecker, 22 Jan 2006
- Comment from Anthony DelGenio, 22 Jan 2006
- Commment from Arthur Hou, 23 Jan 2006
Second Work Plan Draft (15 Feb 2006)
Third Work Plan Draft (21 Feb 2006)
Comment from Donald Anderson, 21 Dec 2005
This looks like a good next step. Maybe shorten/edit examples. As some will know, we are already implementing some of these thoughts in the context of A-Train, etc. I have also been working ‘in the weeds’ to foster some specific cross-center and cross agency collaborations that will be integral to CMAI.
The assembled on this email also are well aware of, or directly involved in NEWS. I expect that NEWS and CMAI will be tightly coupled at the science interface. In addition, the CMAI is not GEWEX or CLIVAR. It knows about both, it will benefit from and inform both. So we’re not designing a new wheel.
We’re attempting to implement all those bullets that CCSP is shooting back and forth over our heads. This is three year funding, so a longer term investment requires CMAI to deliver NOW. We should clean up this latest draft and then distribute to the MAP-selected team, plus others I may know about and you don’t, plus specific non-funded, but important players, like those who served on the panel, Bill Collins, Phil Rasch, Dave Randall,,,.
Schedule a CMAI meeting for early March, have specific guidelines and
goals for that meeting. A second meeting on/about annual report
time. I want to use that second meeting to provide the opportunity
to augment/change direction.
By that time we may have a peek from GMAO/Bosilivich on MERRA test runs, which will be an important element of CMAI and MAP ME in coming years.
Comment from Anthony DelGenio, 02 Jan 2006
I've made a few additions/changes to Bill's writeup beyond the few things I gave him before Christmas, mostly in the areas of what I think is needed to make it feasible for the modeling centers to really do CMAI, and in the area of some possible science themes for the different kinds of models to focus effort on.
In reading over the document, I was struck by how similar some of the goals and approaches seem to those of the NOAA/NSF Climate Process Team that has been working to understand low-latitude low cloud feedbacks in the NCAR and GFDL models. Three years and $3M later, that effort has yet to yield much fruit. They don't understand why the models had different climate sensitivities when they started, they don't understand why they have similar sensitivities now, they implemented a state-of-the-art new parameterization of shallow cumulus in one model and it had little effect. And published papers are almost non-existent. We might do well to discuss why CPT hasn't been more successful before embarking on a grand adventure of our own.
Comment from Bruce Wielicki and Kuan-Man Xu, 18 Jan 2006
My major problem was the strongly negative comments toward NCAR and GFDL efforts, and the indication that we were doing something totally new in a complete attack on the problem. In fact, putting on my NCAR or GFDL hat for a minute:
a) NCAR has a single model it develops using a large community at the universities and NCAR including well developed working group structures, meetings, evaluations, etc.
b) GDFL now uses a single coupled model (not 6 or 7) for development and has weekly to bi-weekly team meetings of the entire climate model team to focus on schedules, issues, comparisons, developments, etc.
c) UKMO has a "full" team approach pretty close to what you were arguing for, while they are probably twice the size of NCAR or GFDL groups.
d) NCAR, GFDL and other groups also carry out a wide range of intercomparison projects (AMIP, CMIP, etc) that are formal attempts to confront models with data, investigate reasons for different behavior, etc.
e) It may be impossible to EVER devise a cloud parameterization at 100km spatial scale that will be accurate enough for climate. Randall's push toward global CRMs, and Kuanman and Tao's complementary effort using the FvGCM are independent efforts to cover the range of scales as you suggest. Randall's new NSF center is built around developing this concept: he currently is planning to start with a full 3-D CRM as the benchmark.
So I tried to take a more positive approach to the motivation and differences of what CMAI is trying to do to help the situation, and why NASA should be doing it.
We may now have too much confusion between my itemized lists and yours, so perhaps some gelling or clarifying is in order. I also realize that this is now more specifically tied to the cloud modeling initiative than what you intended, but I've never been a fan of one size fits all anyway, so using this as a start and modifying as necessary is a good thing in my mind.
Comment from Bruce Wielicki, 18 Jan 2006
Bill has some good questions about the CMAI workshop. and how it should be most effectively run.
Here are my thoughts:
1) primary purpose is to get everyone on the same page. this means aware of who/what else is being done (e.g. plenary presentations), who might be key to collaborate with, and where the overall program is going (iterate and discuss CMAI framework text that bill started).
2) the CMAI framework discussion has to primarily get agreement on the objectives, how it relates to existing nsf/noaa/doe efforts, what key holes currently exist (e.g. effect future Roses NRAs) and how early results might better define the holes and approaches to their reduction in size. Since most people proposed and were selected based on that proposal, and since we don't have lots of spare cash at NASA HQ, major changes in directions of the existing proposals can only take place through future NRAs.
3) any meeting called by the funder to assess progress on your proposal and plans for future activities acts to spur the same: focus/action/progress.
4) keep the meeting short, not long: 1.5 to 2.5 days. After 3 days I typically see lots of glazed eyes....
5) I would suggest 20min presentation per proposal (15 + 5 for discussion/questions): 1 day, half day for discussing the framework document and what we do/modify as a group, holes, etc. If we went for a full 2 day meeting, perhaps getting a half day discussion on how to fill the holes and interface issues (DIME, ESMF, etc). The list of things we "could" do for even CMAI is long enough to dwarf the 15-20 proposal group, so prioritization of which interface and hole issues get dealt with will be a sizeable discussion. Some of the desired CMAI actions will link back to NEWS. This is a good thing.
Bottom Line: we already have a good and innovative group of proposals that will attack many of the things listed in the framework. The group will more completely engage the NASA satellite data than has been possible before. The group will also be well posed to continue improved attacks on the cloud problem as the EOS, NEWS, and A-train data sets mature (accuracy, calibration, data availability, key subsets, etc).
My only current block out dates in March are 21-23 for the CCSP Observations Working Group (OWG) Workshop in DC.
Comment from Michele Rienecker, 22 Jan 2006
Just a few comments from the GMAO (Max, Julio, Michele)
We think it is important that a focus on cloud modeling like the CMAI be put in the context of the existing national multi-agency effort, viz., the CPT Tony refers to below. We say 'multi-agency' because we at GMAO have joined the collaborative effort, though in an unfunded capacity (with Tsengdar's approval) as part of our own development effort. I think Tony has done something similar. As the current write-up points out, NASA has a unique contribution to make by bringing satellite observations to model development and evaluation. However, with the CMAI, NASA should not go off on its own, shunning the CPT with the other major national modeling efforts. Instead, to avoid appearing to be naive by the non-NASA modeling groups, we could use the concept of the CMAI to re-energize the CPT, contribute to it, and to capitalize on it to meet NASA's modeling goals.
Although Tony questions the success of the cloud modeling CPT to date, in fact there have been notable successes:
The CPT has brought the major modeling groups together in collaboration with observationalists and has brought different types of models together to address the problem of improving boundary layer clouds. The fact that there has not been a significant improvement in cloud modeling is not surprising at this stage - it's an indication of how difficult the problem is and of why it is so important for NASA to be heavily involved in the collaboration. This is obviously recognized by Don and all of us - the impetus for a discussion of the NASA CMAI.
The CMAI can re-energize the CPT by
- specifically focusing on the information satellite data bring to the process
- identifying the observation questions as well as the modeling questions (esp those that have implications for model evaluation)
- providing resources so that GEOS-5 (and successors), ModelE, GSFC and LaRC CRMs can participate in and influence CPT experimentation and evaluations
- bring other MAP (or NASA) efforts (such as assimilation, reanalyses) to answer some of the questions posed in the write-up.
Any CMAI Workshop should include representatives from the CPT, as
well as the MAP investigators.
Comment from Anthony DelGenio, 22 Jan 2006
I agree that it would be a good idea to have CMAI join the CPT effort. I apologize for being blunt, but my comments were designed to head off what I worry is a forthcoming train wreck.
First of all, the success/failure of CPT. We all agree that cloud modeling is a very hard problem. But there's more to the CPT experience than that. Chris Bretherton has in my mind made a tremendous effort to engage himself in GCM modeling, both from a diagnostic and a parameterization development standpoint. I have been greatly impressed at his willingness to do so. CPT should be re-funded if for no other reason than the good faith effort Chris has made to make it succeed. But only a couple of the other CPT team members have engaged the project in the same spirit. The rest have simply used their funding as an opportunity to keep doing what they were doing in the past. My experience with the MAP NRA was quite similar - many people wanted to "collaborate" with me on proposals to MAP, but all except one (who was funded, I'm happy to say) envisioned me doing all the work to diagnose and improve the model, while they would just contribute the necessary data and/or data analysis to do so.
My second point, then, involves the modeling centers themselves. CPT realized after a short time doing business that the existing cloud modeling infrastructure at GFDL and NCAR was not going to be able to support the kind of diagnosis and development interaction they had in mind, so by their second year they hired Ph.D.-level liaison personnel at the two centers whose full-time responsibility is to connect the two models to CPT efforts. That was done despite the already substantial funding that supports modeling at GFDL and NCAR. Note by comparison the statement that GMAO is participating in CPT in an unfunded capacity. And GISS is not particpating at all except for my attending one CPT meeting per year (at my own cost) as a member of the Advisory Committee. Chris has approached me about more active GISS involvement in CPT, but I have been non-committal, because we simply do not have the personnel at GISS to hold up our end of the bargain. And I believe the same to be true at GMAO. Julio has done what he can to participate, but by himself he has not been able to do the extensive diagnosis that is being done on the GFDL and NCAR models, nor has he been able to conduct the full suite of climate change experiments that GFDL and NCAR have.
Now consider the difference between CPT and CMAI. CPT is mostly a modeling exercise, a lot of model diagnosis, hypothetical (e.g. aquaplanet) studies, and model development; every once in a while they remind themselves to look at data. CMAI, as I understand it, is intended to bring data front and center into the model evaluation and development process. So if we have, say, 10 NASA satellite cloud datasets and 10 NASA cloud field experiment programs, and funded investigators of each have their own take on *the* way to analyze the data to reveal what's important for the models to reproduce, and those 20 separate approaches to data analysis get funneled into 2 NASA modeling centers where there is perhaps one or less than one FTE of effort devoted to cloud modeling, what is going to happen? (See train wreck statement above.) Perhaps the answer is that the modeling centers will not be doing it themselves in CMAI, the funded CMAI investigators from outside will be diving into the models to do it. But that's where we can learn from CPT, where ostensibly *all* the investigators were explicitly supposed to engage with the models, and in reality, maybe 3 out of 10 or so actually did - despite the PI leading well by example. Can CMAI expect a higher batting average?
In conclusion, let me say that I agree completely with Michele's 4 points about how CMAI can contribute to CPT. Clearly NASA can play a valuable role in the CPT process, and it just makes sense to have all the U.S. modeling groups playing in the same sandbox. But I can't emphasize Michele's 3rd point enough ( - providing resources so that GEOS-5 (and successors), ModelE, GSFC and LaRC CRMs can participate in and influence CPT experimentation and evaluations). It sounds self-serving, I know, but the fact is that as currently constituted, the U.S. modeling centers are nowhere near the critical mass needed to solve the cloud modeling problem. I'll stop holding my breath once I understand how CMAI is going to deal with that (or mabye I won't, depending on the answer).
Commment from Arthur Hou, 23 Jan 2006
This is a very ambitious proposal but that can also mean opportunities if a viable framework can be established to integrate information from models and observations. I would like to see proposals of specific "pathways" for linking models to observations, and how they are better than the business-as-usual approach.
One of the new pathways that I would like to see pursued is using cloud-scale observations to estimate semi-empirical parameters in model physics within the framework of data assimilation.
This is not traditional data assimilation, which uses observations to optimize initial states rather than model physics. For data assimilators, model development is 'someone else's business' - they don't touch it. But discrepancies in clouds & precipitation between models and observations contain more information about model deficiencies than initial condition errors. This is something NASA as a science research agency can pursue, but not so easy for operational NWP outfits.
The advantage of model physics parameter estimation in an online data assimilation system is that you are optimizing the parameters wrt to the analyzed (observed) atmospheric state instead of a simulated (and usually biased) model state.
Anyway, more details on modeling and analysis techniques as a foundation for such an initiative can only increase its change of success.