Measures of Academic Performance … the Good and the Bad
03.24.10 by Michelle Kienholz
Nature, in addition to adding PLoS-like commenting to its entire journal, has an opinion piece by Julia Lane on the long-discussed need for better metrics for measuring academic productivity. She starts off:
Measuring and assessing academic performance is now a fact of scientific life. Decisions ranging from tenure to the ranking and funding of universities depend on metrics. Yet current systems of measurement are inadequate. Widely used metrics, from the newly-fashionable Hirsch index to the 50-year-old citation index, are of limited use. Their well-known flaws include favouring older researchers, capturing few aspects of scientists’ jobs and lumping together verified and discredited science. Many funding agencies use these metrics to evaluate institutional performance, compounding the problems. Existing metrics do not capture the full range of activities that support and transmit scientific ideas, which can be as varied as mentoring, blogging or creating industrial prototypes.
Better metrics will require clean, more easily accessible data not scattered among proprietary data sets … and more appropriate data for gauging performance. For example, “MESUR (Metrics from Scholarly Usage of Resources, http://www.mesur.org), a project funded by the Andrew W. Mellon Foundation and the National Science Foundation, record details such as how often articles are being searched and queried, and how long readers spend on them.” In suggesting other alternatives, she recognizes that:
Knowledge creation is a complex process, so perhaps alternative measures of creativity and productivity should be included in scientific metrics, such as the filing of patents, the creation of prototypes and even the production of YouTube videos. Many of these are more up-to-date measures of activity than citations. Knowledge transmission differs from field to field: physicists more commonly use preprint servers; computer scientists rely on working papers; others favour conference talks or books. Perhaps publications in these different media should be weighted differently in different fields.
Of course, a major factor in hiring and P&T decisions especially will continue to be grant funding, which in turn requires a solid publication record, no matter what productivity metric is used. One wonders if/when the US will be forced to adopt measures (possibly serving as a different sort of new metric) recently implemented in the UK to curb grant submissions by “repeatedly unsuccessful” PIs. Imagine how fun that discussion would be with your Chair come evaluation time….
FIC ARRA RFA for Global Human Subjects Research Oversight
01.29.10 by Michelle Kienholz
This one just has Fogarty ( and its typical pittance for budget) plus NIAID, NIDA, & NIMH.
Program to Enhance NIH-supported Global Health Research Involving Human Subjects (S07)
LOI Receipt Date: February 22, 2010
Application Receipt Date: March 22, 2010
FOA invites applications from US institutions for one year of support for resources & activities that will strengthen oversight of NIH-supported human subjects research conducted collaboratively with institutions in low- to middle-income countries.
Direct costs may not exceed $50,000 for a one-year project period … NO F&A (indirect) COSTS ALLOWED … 12-13 awards anticipated.
12-p Research Strategy narrative
Applications will be supported to develop collaborative processes and training as well as jointly used tools and systems to address the specific needs and capabilities for improved review and monitoring of protocols for NIH-supported research conducted at a developing country institution. The grants will provide one year of support for an applicant US IRB to collaborate with a developing country counterpart IRB that reviews some of the same NIH research protocols to do any of the following:
- Develop sustainable electronic systems, procedures and communication methods to facilitate collaboration on review and monitoring of protocols sent to both IRBs;
- Increase administrative, scientific, socio-cultural and ethical competencies of IRB members and staff related to research reviewed by both IRBs through joint workshops, short-term exchange and training activities;
- Create sustainable international research ethics training resources for global health researchers at both institutions to improve research protocols and practices involving human subjects in the collaborating developing country.
Scientific/Research Contact:
Barbara Sina, PhD
Telephone: (301) 402-9467
Email (preferred): sinab@mail.nih.gov
RFAs from ARRA & NCMHD
01.15.10 by Michelle Kienholz
First, yet another specialized ARRA funding opportunity, and then an unusual NCMHD R21 opportunity for Innovative Faith-Based Approaches to Health Disparities Research. For the ARRA K12ish FOA, NCI, NCRR, NCCAM, NIA, NIAAA, NIAID, NIAMS, NIDA, NIDCR, NIDDK, NIMH, NINDS, NINR, and, interestingly, NLM are on board.
Institutional Comparative Effectiveness Research Mentored Career Development Award (KM1)
Application Due Date: March 25, 2010
The mentored career development approach will permit research-intensive institutions to develop an interdisciplinary environment catering to the needs of diverse populations of scholars including (but not limited to) those with backgrounds in medicine, pediatrics, surgery, dentistry, nursing, allied health, pharmacology, health care administration and management, clinical research design, epidemiology, biostatistics, biomedical informatics, economics, quality improvement, modeling systems, ethics, behavioral science, engineering, and law. Scholars supported through this program could include recent doctoral graduates who are entering the research workforce as well as established investigators who are seeking to extend their expertise or experience in CER in a mentored environment.
Total costs cannot exceed $2,500,000 for the 3-year project period; 8-10 awards are anticipated
The narrative cannot exceed 25 p in addressing: Background; Program Plan; Recruitment and Retention to Enhance Diversity; and Plan for Instruction in the Responsible Conduct of Research. Please see RFA for appropriate program contact.
Innovative Faith-Based Approaches to Health Disparities Research (R21)
LOI Receipt Date: February 9, 2010
Application Due Date: March 17, 2010
Standard R21 budget ($275k-2y)/narrative length (6 p) – up to 8 awards are anticipated
NCMHD intends to cultivate empirical, formative, evaluative and transdisciplinary intervention research on faith-motivated initiatives, concepts and theories that have played an important role in addressing health disparities. The focus on exploratory, evaluative and/or intervention research will allow studies to assess the impact of faith-based initiatives and programs in health disparity populations; formulate hypotheses about the role and unique characteristics of faith communities in addressing health disparities; design targeted interventions; and track the efficacy of community and faith-based or faith-motivated programs that result from a participatory approach to research in the community.
Scientific/Research Contacts:
Irene Dankwa-Mullan, MD MPH
Acting Director, Office of Innovation and Program Coordination, NCMHD
Telephone: (301) 402-1366
Email: dankwamullani@mail.nih.gov
RFAs from ARRA & NCMHD
01.15.10 by Michelle Kienholz
First, yet another specialized ARRA funding opportunity, and then an unusual NCMHD R21 opportunity for Innovative Faith-Based Approaches to Health Disparities Research. For the ARRA K12ish FOA, NCI, NCRR, NCCAM, NIA, NIAAA, NIAID, NIAMS, NIDA, NIDCR, NIDDK, NIMH, NINDS, NINR, and, interestingly, NLM are on board.
Institutional Comparative Effectiveness Research Mentored Career Development Award (KM1)
Application Due Date: March 25, 2010
The mentored career development approach will permit research-intensive institutions to develop an interdisciplinary environment catering to the needs of diverse populations of scholars including (but not limited to) those with backgrounds in medicine, pediatrics, surgery, dentistry, nursing, allied health, pharmacology, health care administration and management, clinical research design, epidemiology, biostatistics, biomedical informatics, economics, quality improvement, modeling systems, ethics, behavioral science, engineering, and law. Scholars supported through this program could include recent doctoral graduates who are entering the research workforce as well as established investigators who are seeking to extend their expertise or experience in CER in a mentored environment.
Total costs cannot exceed $2,500,000 for the 3-year project period; 8-10 awards are anticipated
The narrative cannot exceed 25 p in addressing: Background; Program Plan; Recruitment and Retention to Enhance Diversity; and Plan for Instruction in the Responsible Conduct of Research. Please see RFA for appropriate program contact.
Innovative Faith-Based Approaches to Health Disparities Research (R21)
LOI Receipt Date: February 9, 2010
Application Due Date: March 17, 2010
Standard R21 budget ($275k-2y)/narrative length (6 p) – up to 8 awards are anticipated
NCMHD intends to cultivate empirical, formative, evaluative and transdisciplinary intervention research on faith-motivated initiatives, concepts and theories that have played an important role in addressing health disparities. The focus on exploratory, evaluative and/or intervention research will allow studies to assess the impact of faith-based initiatives and programs in health disparity populations; formulate hypotheses about the role and unique characteristics of faith communities in addressing health disparities; design targeted interventions; and track the efficacy of community and faith-based or faith-motivated programs that result from a participatory approach to research in the community.
Scientific/Research Contacts:
Irene Dankwa-Mullan, MD MPH
Acting Director, Office of Innovation and Program Coordination, NCMHD
Telephone: (301) 402-1366
Email: dankwamullani@mail.nih.gov
ARRA Administrative Supplements
01.4.10 by Michelle Kienholz
When it rains, it pours … in odd and extravagant ways …
Administrative Supplements for Comparative Effectiveness Research Workforce Development
Receipt Date: March 1, 2010
The maximum sum that may be requested is $500,000, as total costs, in FY2010 only. Up to 16 awards may be made, likely by September 2010 and, where possible, shortly after their approval.
This administrative supplement provides the opportunity for eligible grantees to develop, expand, and/or increase CER training, education, and career development programs within existing U.S. NIH-supported grants. Some types of awards from NIH (eg, S10 awards and many R and P awards) are not eligible for administrative supplements. Institutional awards that support the scope of training envisaged in this supplement announcement include the K12, K30, T32, T35, and T90 mechanisms, together with their KL and TL equivalents. Questions about eligibility should be addressed to the program officer and/or the grants management specialist for the award.
To be eligible, the parent grant must be active, and the training, education, and career development activities proposed in the supplement must be accomplished within the current competitive segment. The proposed supplement MUST be within the general scope of the peer-reviewed activities and aims approved within the parent grant, including projects on a no-cost extension.
Some examples of the types of supplements that could be appropriate include, but are not limited to: adding scholar or training slots for CER education and training through short courses, Certificate programs, and advanced degree-awarding programs; creating a course curriculum for early or midcareer researchers to develop or enhance skills in CER; creating an outreach training or course for community based research to develop or enhance skills in CER; and developing a CER training or course to include related fields such as communication and information dissemination science, medical decision-making, and outcomes and evaluation research as long as the proposal’s specific aims supports Federal Coordinating Council-defined CER.
Science Works For Us
11.18.09 by Michelle Kienholz
Literally, considering ARRA research awards are tax-payer funded.
ScienceWorksForUS, which highlights all aspects of stimulus funding for university-based research activities, is brought to you by the Association of American Universities, the Association of Public and Land-grant Universities, and The Science Coalition. You’ll find the expected news feeds about the economic stimulus program generally, ARRA-funded research anecdotes, and research findings stemming from ARRA-funded efforts.
The level of detail at the state level is nicely organized: total dollars and number of awards plus links to individual universities (the Web pages on which they report their ARRA awards and whatnot), a breakdown by funding agency (NIH, NSF, DoE), and more state- and university-specific news releases related to ARRA-funded research. You can run your cursor over the US map to quickly compare who’s getting what out of this initiative and click on individual states for the aforementioned details.
As a reminder, NIAID invites you to contribute your own story of how ARRA funds have helped you, as does the US DHHS, which invites you to submit stories or comments about ARRA funding.
ResearchMatch.org
11.13.09 by Michelle Kienholz
No, not an online dating service … ResearchMatch is an NCRR-funded secure registry that allows individuals to sign up to receive alerts about clinical research in which they might be interested and for which they might be eligible — and researchers to sign up to find potentially eligible participants for their studies.
Currently, 40 of the 46 current CTSA sites are participating, and the registry will eventually expand to include non-CTSA institutions. As described in the NIH’s news release:
After an individual has self-registered to become a volunteer, ResearchMatch’s security features ensure that personal information is protected until volunteers authorize the release of their contact information to a specific study that may be of interest to them. Volunteers are notified electronically when they are a possible match and then make the decision regarding the release of their contact information. It also will promote choice as there are no obligations on the volunteer to participate in studies.
According to the ResearchMatch.org participant FAQ, “[approved] researchers will not be given access to begin looking for potential study Volunteers through ResearchMatch until approximately January 2010.” As further explained by the researcher FAQ:
ResearchMatch is a not-for-profit activity and is free for any participating site & their researchers. … Researchers at participating sites will be given access to register through the ResearchMatch system. Upon registration, researchers may request either feasibility or recruitment access … your access to recruit via ResearchMatch will last only as long as your IRB-study approval.
After you have been granted recruitment access, you will be able to search for appropriate matches amongst the non-identifiable ResearchMatch Volunteer profiles in the system. You will enter your study’s criteria in the ResearchMatch Search Builder which will yield a list of these potential matches. You will send out IRB-approved content in your initial recruitment message to these potential matches through ResearchMatch. The secure ResearchMatch clearinghouse will route your message to each of these potential matches and they will have the option of replying yes, no, or no response. Your study’s home page will feature all those Volunteers who say yes and show aggregate figures/charts demonstrating the response rate to your initial recruitment message. Once the Volunteer has authorized ResearchMatch to release their contact information to you, you will be responsible for managing this contact information as called for by your IRB-study protocol.
Vanderbilt, which maintains and whose IRB oversees ResearchMatch.org, also has its own DNA bank called BioVu, which includes leftover blood from all patients seeking treatment unless they opt out when signing the the Vanderbilt Consent for Treatment and Agreement to Pay form.
Science Outreach and Framing Science
10.21.09 by Daniel Gaddy
Yesterday I wrote about influenza and the influenza vaccines. In this post, I referenced two articles demonstrating how popular talk show hosts, on both sides of the political aisle, are irresponsibly promoting irrational fears of the influenza vaccines. I believe this is, at least in part, due to some issues that FundScience was founded to address. Throughout human history, it seems that science has been demonized. Over the past few decades, this has become particularly relevant. As science attempts to address important problems that affect all of us, from climate change to influenza pandemics, science is met by stubborn yet impassioned resistance from people who have no idea what they are talking about!
While writing yesterday’s post, I was reminded of two important articles I read a couple of years ago. The first was written in 2007 by Seth Shostak of the SETI Institute, entitled When Did Science Become the Enemy? In this article, Dr. Shostak attempts to trace the origins of the “mad scientist” perception of scientists. While these perceptions have likely been around throughout recorded history, they have become particularly prevalent over the past few decades due largely to Hollywood.
The peculiar thing is that American heroes aren’t often very good at science. Indeed, in much popular culture, it’s only the villains who’re conversant with Maxwell or Einstein. The “mad scientist” has become such a cultural icon that the Royal Society held a special lecture on the subject. Some of the mad men of science (and they are, overwhelmingly, men) are just evil characters intent on destroying the world, taking over the world, or simply rearranging the world according to their personal predilections. Dr. No and Dr. Evil come to mind, as do Lex Luthor, Dr. Octopus, the overly Teutonic Dr. Strangelove, and the Green Goblin.
How did scientists become the enemy? I mean, really: who would you rather have help you take a calculus final… or for that matter, cure the common cold or figure out the nature of dark energy: Spiderman or Green Goblin? Science is useful.
And if the scientists in popular media haven’t slipped entirely to the dark side, they’ve at least gone bonkers. They’ve become obsessed with some narrow field of research, and lost sight of the big picture. When a prehistoric monster is shambling through a major metropolis, wreaking havoc and destruction, there’s always some lab-coated PhD who’s interfering with the steely-eyed military types, screaming “we have to save it for science!” And just to make sure that these howling academics won’t become your role model, they’re usually portrayed as short, ugly bald guys with social grace and sex appeal on a par with Ben the rodent.
Certainly Hollywood has done its part to denigrate honorable scientific endeavors, but the reasons for that may be partially our fault. This brings me to the second article I would like to quote. Also in 2007, Matthew Nisbet and Chris Mooney wrote about Framing Science. In this article, the authors attempt to inform scientists how to “frame” scientific issues to not only catch the attention of lay-persons, but to “resonate with core values and assumptions.”
Without misrepresenting scientific information on highly contested issues, scientists must learn to actively “frame” information to make it relevant to different audiences. Some in the scientific community have been receptive to this message. However, many scientists retain the well-intentioned belief that, if laypeople better understood technical complexities from news coverage, their viewpoints would be more like scientists’, and controversy would subside….
…the scientific theory of evolution has been accepted within the research community for decades. Yet as a debate over “intelligent design” was launched, antievolutionists promoted “scientific uncertainty” and “teach-the-controversy” frames, which scientists countered with science-intensive responses. However, much of the public likely tunes out these technical messages. Instead, frames of “public accountability” that focus on the misuse of tax dollars, “economic development” that highlight the negative repercussions for communities embroiled in evolution battles, and “social progress” that define evolution as a building block for medical advances, are likely to engage broader support.
The evolution issue also highlights another point: Messages must be positive and respect diversity. As the film Flock of Dodos painfully demonstrates, many scientists not only fail to think strategically about how to communicate on evolution, but belittle and insult others’ religious beliefs.
On the embryonic stem cell issue, by comparison, patient advocates have delivered a focused message to the public, using “social progress” and “economic competitiveness” frames to argue that the research offers hope for millions of Americans. These messages have helped to drive up public support for funding between 2001 and 2005. However, opponents of increased government funding continue to frame the debate around the moral implications of research, arguing that scientists are “playing God” and destroying human life. Ideology and religion can screen out even dominant positive narratives about science, and reaching some segments of the public will remain a challenge.
It is true that scientists tend to let the facts speak for themselves. This is what we are taught, and what we are good at. But now we have to face the facts. The people making the decisions to give X dollars to science and Y dollars to, say, war are lay-people. The need exists to reach out to society at large and educate people in a way that is not boring. This is not an easy concept for scientists, because the research that is exciting to scientists is not necessarily exciting to non-scientists. Furthermore, there is a fine line between simply reaching out, attempting to appeal to the public, and making a fool of yourself and science in general. If you do not go far enough, you miss an opportunity to gain public support, and possibly even increase funding for your field. If you go too far, you become a quack or a mad scientist, and you risk alienating the public even more and making them even more suspicious of science than they already are. We must find ways to frame science appropriately, while doing our best to protect science from biases. FundScience is trying to be a part of this science outreach effort, and we would like to hear ideas for doing this from our community. Let us know what you think in the comments.
Related articles
- Saving science from “Unscientific America”
- The Vanderbilt Center for Science Outreach
- Stanford Office of Science Outreach
- Science outreach is becoming hip
- Perceptions of Science – Scientist vs Public
Science2009 Recap
10.19.09 by FundScience
The University of Pittsburgh’s Science2009 event took place on Thursday and Friday of last week. Despite inclement weather, the event was a success. The FundScience booth attracted approximately 50 visitors over the two days. These visitors included graduate students, postdocs and faculty members, all of whom shared very good ideas for funding implementation, the direction of the organization, the website/blog, and other issues. Of note, Dr. Victor Ambrose of the University of Massachusetts, Science2009 Plenary Speaker and winner of the Dickson Prize in Medicine, stopped by and had some very insightful and inspiring comments for us.
Dr. Victor Ambrose meets with FundScience cofounders David Vitrant and Mark Friedgan.
We appreciate the time all of our visitors took to share their ideas for FundScience development and their excitement for our cause! The University of Pittsburgh hosts these events annually, so we look forward to an even greater turnout for Science2010! In addition, watch this blog and follow us on Twitter and Facebook for announcements of our participation in other upcoming science events!
Science2009
10.13.09 by FundScience
The University of Pittsburgh’s Science2009 event happens this Thursday and Friday. The event is free and open to the public, and includes lectures on a variety of scientific topics, including cutting edge technology such as microRNAs, evolutionary biology, renewable energy, and infectious diseases. Anyone in the area should take a look and consider attending sessions they are interested in.
FundScience will have a presence at Science2009 in the form of a booth where you can meet FundScience members and obtain more detailed information about our mission and our upcoming funding opportunities. We are excited to meet members of our community and share the progress we have made over the past few months, so please stop by and, as always, spread the word!
Science2009—Unplugged, the University’s ninth annual celebration of science, technology, and research, is coming on Thursday and Friday, 15 and 16 October, to Alumni Hall (with two special preview events on Wednesday, 14 October). Here are six great reasons to attend:
1. Highlighting the program will be plenary lectures by four distinguished scientists: Victor Ambros, PhD, of the University of Massachusetts Medical School; Michael Graetzel, PhD, of Ecole Polytechnique Federale de Lausanne; Cori Bargmann, PhD, of the Rockefeller University; and Bruce Beutler, MD, of The Scripps Research Institute.
2. Pitt and Carnegie Mellon researchers will present Spotlight Sessions on a dozen timely topics in which they are actively engaged. Likewise, five Science at Work Sessions will draw on expertise from local industry.
3. Some of the latest laboratory equipment that makes today’s research possible will be on display at the Science2009 Supplier Showcase from 10 a.m. to 3 p.m. Wednesday, 14 October, at the Petersen Events Center.
4. The Opening Reception and Technology Showcase returns this year with a special exhibit of new technologies recently developed by local academic researchers. Come and mingle with them and potential investors, venture capitalists, and entrepreneurs from 5:00 to 7:00 p.m. Wednesday, 14 October.
5. Maryrose Franko, PhD, of the Howard Hughes Medical Institute will be the featured speaker at a special professional development workshop designed especially for researchers in the early stages of their careers.
6. Two poster sessions highlighting recent research by students, post-docs, and faculty will be complemented this year by the Undergraduate Research Poster Reception.
For details, go to www.science2009.pitt.edu. All Science2009 events are free and open to the public.
| Posted in FundScience News, Research Resources | 1 Comment »