STI: Metrics That Matter
by Nicolas Sacchetti
At the P4IE 2022 Conference on Measuring Metrics that Matter, Kaye Husbands Fealing has done a keynote address on Science, Technology and Innovation (STI) Metrics and Intersectionality: Assessing the Public Value of Innovation.
She is the Dean of the Ivan Allen College of Liberal Arts at the Georgia Institute of Technology and specializes in the science of science and innovation policy, the public value of research expenditures, and the underrepresentation of women and minorities in STEM fields (science, technology, engineering, and mathematics) and workforce.
Kaye Husbands Fealing also belongs to the Science and Technology Action Committee and has been participating in the Science and Technology Action Plan for more than a year. From the outset, she explains that « the purpose of the committee was to talk about what needed to be known in order to buttress fiscal spending on STEM. » They did a survey with Zogby Analytics. The results show the interest of the public about how science is funded by governments and the effects on the private sector.
She points out the intersection of the combine issues of the survey question where 77% of the respondents agreed the U.S. is at a critical point for committing to a major new initiative to assure health, security and prosperity for the nation. She states: « We need to be able to understand, not just have it funded. We need the data and the metrics on what goes to funding and what is the return on investment from that funding. »
She also points out the 83% positive responses of Americans saying they would strongly or somewhat recommend that their child, family member, or other young person enters a STEM field.
Challenges S&T Policy Can Inform
On this topic, she refers to the former science advisor at the Obama administration John P. Holdren 2008 article titled Science and Technology for Sustainable Well-Being. He addresses, « the status of five specific challenges in which S&T have particularly important roles to play: meeting the basic needs of the poor [Dean Husbands Fealing calls it ‘inclusive innovation’]; managing the competition for the land, water, and terrestrial biota of the planet; maintaining the integrity of the oceans; mastering the energy-economy-environment dilemma [she underlines the intersectionality here]; and moving toward a nuclear weapon–free world. »
How do we measure STI?
More than ten years ago, as the Study Director at the U.S. National Academy, her research panel was asked to look at metrics for STI, especially related to policy. In 2014, the panel published the report: Capturing change in Science, Technology, and Innovation. She explains that the social returns to the public and private expenditures was the start of their reflexion to find out what people want to know and what needed to be measured. « We wanted to know the social outcome of the funding, and therefore what policy needed to be put in place to ensure the highest return on investment, » elaborate Kaye Husbands Fealing.
Also as a follow-on question, given expenditure on STI activities, they wanted to look at the impact on economic growth, competitiveness, and jobs. As STI indicators, « Users want to know the drivers of innovation that could be encouraged through funding mechanisms and creative organizational structures. […] Indicators are also sought for trends as to which countries will be generating the most scientific research that can be commercialized or which countries are likely to attract the most R&D investments in the near future, » tells the report.
The other STI indicators are about « advances in science on the horizon or vulnerabilities in the innovation ecosystem that could impede the commercialization of new ideas. […] People are also interested in the nature of cooperative relationships that foster collaboration [culture/climate] […] and distributional questions, including geospatial hot spots for entrepreneurial activities; potential avenues for broadening the participation of women, minorities, and people with disabilities in STEM fields; contributions to S&T breakthroughs from the social and behavioral sciences; the uptake of ideas for innovation from consumers; and the inclusivity of growth for various rungs of society », the report explains.
Kaye Husbands Fealing also presents equity – diversity – inclusion (EDI) research metric examples from the National Research Council (NRC): cites per publication of interdisciplinary faculties, non-Asian minority faculties and female faculties; Graduate Record Examination – Quantitative (GRE-Q) of first-year students with full support. Furthermore, how many of the first-generation students are reached.
In 2019, Kaye Husbands Fealing was part of the S&T Methods Subcommittee of the Council of Canadian Academies (CCA). The subcommittee had the role to « examine the challenges and limitations of current methods used to assess Canada’s strengths in S&T and R&D. » From everything they have looked out, she pulled out collaboration rates to understand advancement and discerning what can be used: collaboration at/and between universities and among nations on S&T (e.g. better vaccines, the Internet for all, tackle climate shifts).
She was also pleased to see the CCA’s Scientific Advisory Committee (SAC) didn’t just look at traditional R&D avenues but also looked at the impacts of the social sciences, arts and humanities as well as how to get at the impacts that these areas have on STI. Kaye Husbands Fealing: « We may have to use different methods including interviews and case studies to assess the impact of social sciences, arts and humanities on a scientific framework. »
AI, Ethics, and the environment
Kaye Husbands Fealing brings up the question of what is the intersection of AI, ethics and the environment. « What are we doing to assess that within the innovation system? » She shows different pieces of news to demonstrate the advancement of data science and the importance to look at the intersectionality of AI, ethics, and the environment.
« Those examples are different aspects of using AI to understand what is happening environmentally and thinking about the ethical aspect of how change is occurring within our environment, » she says: Keeping coral reefs healthy with AI – AI could help to predict harmful alga blooms – Machine learning algorithms are used to track movement patterns and predict extinction risk of species
She then poses questions of reflexion: « Are we able to assess what this innovation is and how it is being used and also what are the ethical principles behind how it is being used? Looking at the innovation system, we need to be able to capture this information and understand how investment at this intersection is bringing us forward and see if we need to target different types of research or implementation of these types of techniques. »
There is a challenge. « The underrepresented classes in datasets could lead to bias that misses the existence of rare species and could result in false environmental impact assessments — loss of habitat could result or misalignment of resources for conservation, » advise Kaye Husbands Fealing.
Underrepresented Groups in STEM
Kaye Husbands Fealing is also the vice-chair of the National Science Foundation (NSF) Committee on Equal Opportunities in Sciences and Engineering. She quotes the 2019-2020 biennial report Making Visible the Invisible, Bold Leadership Actions: « NSF demonstrate and promote bold leadership actions to create, integrate and make visible elements within and across its programs to enhance broadening participation of underrepresented and underserved groups in STEM Leadership […] broadening participation is not a problem but a strategy to promote and advance scientific research, learning and innovation. »
On gender and innovation, Kaye Husbands Fealing quotes a paper she is co-authoring with on this topic: « History shows that female scientists have posed new questions, launch new theories and open novel fields of study. »
Going forward, she says that we need Community Innovation Surveys that allow us to incorporate gender disaggregated data into established indicators. « Future questionnaires could pose queries related to workers’ gender and ethnic identities and associated labor roles and responsibilities. » She also mentions the OECD’s Women’s leadership in environmental action report conclusion saying : « Future work could focus on causal links between the distribution of gender data collected and the adoption of environmental policies, their stringency, and gender relevance. »
This content has been updated on 2023-10-27 at 21 h 39 min.