Full and Open Access to Data

The following statement(s) have expired and are here for historical purposes and do not represent statements of the AMS that are “in force” at this time.

A Policy Statement of the American Meteorological Society
(Adopted by AMS Council on 4 December 2013)

Introduction and Background

Full and open access to environmental data benefits science and society and is critical to the three main sectors of the American Meteorological Society (AMS) enterprise,   namely, academic, government, and commercial interests. While there are similarities, there are also fundamental differences in the ways the three sectors collect, use, and share data.  Research is common and important across all the sectors.  Academia is distinct with a primary mission aligned in education; academic users primarily collect data and develop products with limited scope, and rely heavily on data from the government and in some instances from the private sector.  Government, especially at the federal level, carries the major responsibility for maintenance of observing systems with national and global reach, operational data collection and quality control requirements, provision of operational products for the United States, and sponsoring research that in turn may generate additional data.  Data collection in the private sector tends to occur on a smaller scale and is often coupled with business requirements.

The above basic view also has wider dimensions; the World Meteorological Organization (WMO) World Weather Watch was founded over 50 years ago in recognition of the fact that weather forecasting beyond the immediate range is a global activity and requires access to observations across many nations, continents, and oceans. The reliance of modern society on accurate weather forecasts for protection of life and property and as a basis for routine decisions has long been recognized and would not have been possible without the full, open, and timely exchange of data among the 191 WMO Member States and Member Territories under the World Weather Watch Program, per policies stated in WMO Resolution 40 and Resolution 25 i. Earth System Science (ESS) research, education, and business also must have access to global environmental data. The increased globalization of ESS, especially in solving complex interdisciplinary problems, including climate projections, requires that research be conducted collaboratively by distributed teams of investigators, often involving sharing of data, knowledge, and other resources.

The AMS encourages the ESS community to provide full, open, and timely access to environmental data and derived data products, as well as all associated information necessary to fully understand and properly use the data (metadata). These data are at the foundation of efforts to ensure public safety and national security, as well as efficient management and use of weather- and climate-sensitive sectors and systems such as water resources, transportation, and agriculture. Environmental data are used to protect critical infrastructure and support scientific publications, and are essential for routine and high-impact weather forecasting and warning, and climate monitoring. The spectrum of data is diverse and includes primary measurements collected by in situ and remote-sensing instruments and observational networks; products derived from single and multiple sources, such as ocean wind speed at 10 meters; atmospheric water vapor; cloud liquid water;  rain rate derived from radiative brightness temperatures observed by microwave-sensor-equipped satellites, and information associated with environmental predictions generated by numerical models. Also, feedback on data use can be a significant motivator in establishing effective data exchange policies, particularly in traditionally hard-to-obtain data sources, such as those related to water resources, air quality, and human health. In all this, the provision of metadata and generation and maintenance of data in widely accepted standard forms is critical.

In addition, the AMS expects all scholarly papers published in its journals to contain sufficiently detailed references to public sources of information (literature and data) and methodology such that independent research can test the paper’s scientific conclusions. This expectation assumes that the data and metadata upon which the conclusions are derived are properly cited and readily available to the scientific community.

The AMS concept of full and open access is to promote, to the greatest extent possible, data availability at minimal or no cost. However, the AMS recognizes that this cannot always be the case.   For instance, significant data collection and value-added product generation in the private sector supports specific client needs and rightfully has an associated cost to be recovered.  Similarly, if users request specific actions be taken on their behalf to prepare data, data providers may need to levy a modest recovery cost.

With regard to the private sector, the AMS encourages public–private partnerships to ensure data availability to support public safety, protect critical infrastructure, and advance ESS and related services. Many U.S. research agencies, including the National Science Foundation (NSF), the National Oceanic and Atmospheric Administration (NOAA), the National Aeronautics and Space Administration (NASA), and the Environmental Protection Agency (EPA), already have data policies in place and have taken steps to ensure  that data upon which scientific studies are based are preserved and made available at little or no cost to users. It is the view of the AMS that all sectors can contribute more toward the overarching goal of full, unrestricted, and open access to data. For example, the private sector can operate specialized data collection instruments (e.g., lightning detection stations) and make that data available at reasonable cost to users. Government agencies can find ways to link and feed more data into national and international data-sharing systems to ease the burden, and cost, of data sets. Finally, federal agencies that fund scientific research can require their grant recipients to develop and comply with data policy and management plans that describe how the investigators will provide for long-term preservation of, and access to, scientific data resulting from their research.

The AMS recognizes that government agencies, academia, and private-sector entities face challenges in attaining the above goals.  Some of these challenges include policies that inhibit the widest possible use of data, such as resource constraints, intellectual property rights, and inadequate cyber-infrastructure to properly maintain and administer data sets.

To address these challenges, the AMS proposes the following core principles and recommendations to inform national and international data policies and influence standard practices in academia, government, and the private sector in the pursuit of full and open access to data:     

Principles

  • Public and private entities engaged in activities that ensure public safety, protect critical infrastructure, and support commerce should have, to the greatest extent possible, full, open, and timely access to environmental data.  
  • Data transparency should be encouraged, and the interests of data providers and users must be recognized and adequately addressed in a data-exchange policy.
  • Unrestricted access to data used and generated by scientists is fundamental to advancing basic and applied science and the independent validation of research findings.
  • The creation and maintenance of authoritative metadata and data in accepted standard forms are fundamental to data exchange and effective analyses.
  • In the most general sense, all data and data products should be archived, preserved, stewarded, and accompanied by metadata that will ensure they will be fully usable and understandable for generations to come.  However, because of limited resources, not all data and products can be managed long-term. This barrier is not a reason for inaction; the data community, including collectors, providers, archivists, and users, must assume responsibility and work together to decide what data should and can be saved.
  • To the maximum extent possible, all data and metadata should adhere to agreed upon formats, conventions, and standards at the community and international level.
  • Data curation and stewardship are essential for ensuring that the data are fit for use, readily available to the communities that can use it, and preserved as long as they serve the need of science and society.
  • In times of crisis or emergency, the government and supporting private-sector organizations should be able to activate preplanned value-added joint data collection capabilities that enhance the ability to address the crisis, ensuring fair compensation and protection of intellectual property rights.
  • Collection and archiving of data incur costs, which must be recovered in some form. The user community obtains maximum benefit when this cost recovery is structured in a way that minimizes both the cost and cost-related uncertainty for users. Maximum benefit for education and research is achieved when data are available freely or at a minimal cost.
  • Increased public–private partnerships are needed to ensure more open, long-term affordable access to data in times when fiscal pressures continue and enabling technologies rapidly evolve.

 

Recommendations

  • The AMS must design programs to reduce data-sharing barriers among its three main sectors: academia, government, and private sectors.
  • Funding agencies, scientific societies, and planning organizations are encouraged to fully recognize the data management costs and support those activities.
  • The advantages of full and open data access should be documented and promoted as best practice by the AMS.
  • Data preservation and stewardship should be supported and recognized as essential for scientific research and providing expanded opportunities for sharing ideas on these topics at national and international meetings.
  • The AMS should promote the use of community and international conventions and standards for data and metadata for providing, archiving, referencing, and sharing to increase data use and interoperability.
  • In the interest of open access and scientific integrity, the AMS should strongly encourage an environment in which scholarly papers published in scientific journals contain sufficient detail and references to data and methodology to permit others to test each paper’s scientific conclusions. The practice of citing data that are referenced in publications, via Digital Object Identifiers, would foster and promote a culture wherein researchers' published works in scientific journals share the data and metadata used in their research, including derivative data generated in their research if such data are essential in supporting conclusions.
  • Governments should adopt policies and practices that encourage private-sector innovation and partnerships through applied research, licensing, data exchange, and other means to ensure full and open access at the lowest possible cost.
  • The U.S. Government should develop a process to identify privately collected environmental data and data collection capabilities that may be useful in a time of crisis or emergency and develop plans to access those data, ensuring fair compensation to entities that own the data and/or operate proprietary data collection systems.
  • The AMS should encourage more data sharing from all data collection and product development systems that are part of the ESS endeavor, including data from traditionally hard-to-obtain observing networks (e.g., hydrologic and air-quality observations from federal and state government networks).
  • Since not all data and data products can be archived, preserved, and stewarded for generations to come, the AMS should promote a best practice that includes ongoing community discussion, coordination, and shared responsibility between, data collectors, providers, archivists, and users. Open processes, driven by the stakeholders, that lead to data value assessment, data quality assurance and control, and protocols for reasoned decisions concerning what data to preserve within the resources available need to be established. 
     

The purpose of this statement is to reaffirm the American Meteorological Society’s commitment to a policy of full and open access to environmental data that are critical to the advancement of atmospheric and related sciences, the provision of products and services for the benefit of society, and the promotion of commerce and private-sector activities. Adopting such policies could accelerate scientific discoveries, broaden and enhance participation in scientific enterprise, promote entrepreneurship, and benefit society.

[This statement is considered in force until December 2017 unless superseded by a new statement issued by the AMS Council before this date]