The following are the reply send to a member
of NSERC Grant Selection pannel who has
requested clarifications about the Sliding
Funding Scale suggested by Donald Forsdyke.
I felt, the reply may be of interest to many
others and thus decided to post it.
Alex Berezin
---------------------------------
Dear Dr...,
Thanks for experssing some interest in our
proposals and Forsdyke's Sliding Scale.
Below this comment I attach two more files,
the first are "NSERC QUOTES" (delails are
explained there), the second are the "REFERENCES"
with references on peer review and funding issues
with (some) of Forsdyke's papers listed.
However, I can explain his scheme in brief:
In essence, Forsdyke's idea is pretty straight.
At first glance one may even say that it is
"almost" the same as the NSERC's present funding
scheme. Almost, but not quite. Let me explain
the difference.
Forsdyke starts from the obvious premis that all
university professors who are active in research,
should be given SOME opertaing funds to do their
work. Very few peopele, if any, will dispute this
_in principle_ (after all, the expectancy of doing
research is #1 reason why almost all of us were
hired in the first place). Of course, what exactly
means "to do research" has some dergee of
subjectivity, and there are some few odd cases,
but they are pretty rare and certainly nowhere near
35-40 % which is presently the unfunding (NIL) rate
in science and engineering.
Now we got to the real problem. Beacause for the vast
majority of university researchers NSERC is the ONLY
funding source, and because it runs a single-grant
(operating) system, the above means that the prime
requiremnt (that the research activity needs some,
let basic, operating funding) is not satisfied by
NSERC. As a government organization, NSERC simply
fails its mandate (to fund Canadian university
professors doing research in science and enginering),
in the same way as a would-be school system which
would refuse to enroll 1/3 of all students of the
the proper age.
(NSERC would probably be a bit closer to it if
it had a multiple operating grants system, as NSF or
NIH have, but for some reason [ or without it ]
NSERC allows only one operating grant per person).
Forsdyke proposes grant ranking on the combined
scores on a track record (TR) and the proposals
(P). He suggests that TR and P ranking is done by
different peer reviewers, to mimimize the errors
and achieve some kind of a robust score. Because
of this, he calls this 'bicameral review' (as if
two separtae evaluating chambers assess TR and P).
You can loose on one side and still maintain on
the other.
Only in those (undoubtedly, rare) cases when
the scores are clearly below the minimal passing
threshold on BOTH counts of TR and P scores, there
is a justification for a NIL grant. Overwise, it
is a finite and the amount is defined by some
weigthing combination of components. It appears
almost like the present scheme, but the
similarity is misleading because in NSREC it
largely translates into a yes-no scheme.
And don't worry for the total NSERC budget: if
the sliding scale is introduced, there is likely
to be a lot more downgradings of grants than
upgradings. I know, some will likely to see this
prospect as an unpleasant one, but that's where
the housecleaning is ought to be done.
In short, Forsdyke proposes to replace the present
step-function scale with a sigmoid (sliding) scale.
With a sigmoid scale people can move up and
down, depending on the performance, there is a
dynamical feedback to their actual work. Not so in
the present (NIL) system. The latter does not really
provide a constructive fedback. For as long as you
are a good boy, you are funded, but once you have
stumbled (got NIL, often for a minor criticism), you
are disrobed and (almost always) forever. Cases of
return are quite rare. As you know, the present
figure is about 30 % NILs, but ADD TO THIS those
people (actually, very many !) who after few
attempts are simply giving up and no longer apply
for NSERC grants. Even at McMaster (which is a very
research-intensive univeristy), there are several
NILs in practically every science and engineering
department, and one should have a pretty weird
imagination to classify all of them as a dead wood.
There are several superficial "explanations" which
NSERC administartion uses to justify the rejection
of sigmoind scheme while insisting on the
selectivity principle. I can go at length on all of
these "reasons", but in summary, all of them are
unsupportable by facts and/or logic and are
self-contradictory.
In my (and Forsdyke's, and many others) view, the
most detrimentally wrong thing about NSERC is that
it is calling its funding panels, GSC, grant SELECTION
committees. It's not to say that there should be no
NIL awards at all (certainly, there will be some, we
are not inviting research welfare system), but the
very fact that this term (selectivity) is used in
a TITLE of a structural unit (granting panel) sends
an utterly distorted, wrong message. What if we call
PhD defence committees a "doctorare SELECTION
tribunal" ? (despite, that in pactice, yes, some
defences do occasionally fail).
One may say, who cares which words are used.
Does it really matter ? Unfortunately, it does, and
a lot. As you may or may not believe (I do), this
is a classical case when the (wrong) WORD forms
a (wrong) REALITY.
The main assumption (also deadly wrong) which NSERC
sells in this connection is their assertion that
"small grants are useless" and hence it is better to
dephase "problematic" people altogether than to give
them small (say $ 5,000 to $ 7,000 per annum ) grants.
(of course, it will be much cheaper for the
hospitals to send patients to the gas chambers
than to treat them: a same kind of philosophy).
In the "NSERC QUOTES" attached below there is a
compilations of remarks (circulated by e-mail
earlier) from the responses we (CARRF: Canadian
Association for Resposnsible Research Funding)
have received from all over the country. This was
circulated a while ago. We are still receiveng
new comments, but we did not update this compilation
for a long time.
You perhaps may especially notice the last quote
(# 21). This is from another GSC member. I am
certain that he is quite correct in indicating
the pressures from NSERC administartion to be
even more selective, the pressures which some
individual GSC members apparently resist and
would prefer much lesser NIL rate.
I wonder, what your comments on this would be.
To conclude, I don't see that anything truly
positive can emerge unless and until GSCs will
be renamed to GRC, Grant RANKING Committees
(or equivalent term) and the ACTUAL POLICY will
be changed correspondingly. NIL awards should
rarity, in the most acute cases only. Furthermore,
those who are NILed should be given very specific
reasons and objectives to meet and if efforts to
meet them are evident in the next applications,
they should be given probationary awards. Almost
all systems in our society operate on this
principle.
Overwise, finite awards must be given. No grant
is too small (who ever refused to accept a grant,
because it was "too small" ?).
I (and many others) frankly don't see most other
NSERC-run activities (like 'realocations') as
having too much real value unless the NIL probem
is resolved as a first priority, by Forsdyke's
method or else. Who is going to get busy on a
redecorating of the house and landscaping when
one third of the building is in a collapsed
state ? The NIL problem is thus a number one
issue you (GSC members) have to work on
collectively with the NSERC administartion.
Sincerely,
Alex Berezin
----------------------------------------------
CANADIAN PROFESSORS COMMENT ON NSERC POLICIES
The following are some quotes from the responses
received by Berezin and Hunter regarding their
two publications:
1) "Open Letter to Canadian researchers - funding
system needs reforms" (CAUT bulletin, February 1994,
page 15) (CAUT= Canadian Association of the
University Teachers)
2) "Myth of Competition and NSERC Policy of
"Selectivity" (Canadian Chemical News, March 1994,
p.4-5)
--------------------------------------------------
[1] ... I received your newsletter through the
Faculty of Medicine which is circulating it here.
Most colleagues agree with your sentiments.
Good luck !
Professor of Medical Sciences, Western Canada
-------------------------------------------------
[2] ... As an NSERC grantee, I fully support your
idea of creating the Canadian Association for
Responsible research Funding. I would be pleased to
be a member of this Association.
Professor of Physics, Ontario
---------------------------------------------------
[3] ... I would like to consider constucting a case
for the damage done to unity of Geography by NSERC
policy of discriminative funding ...
Professor of Geography, Ontario
----------------------------------------------------
[4] ... We are behind you 100 % ... In my department
people in [one group] are so much funded, that they
cannot even spend all their money, while other people
working in basic research (with much higher level of
productivity) are not funded at all or very little.
It is a shame that our scientific community is
manipulated by an irresponsible and selfish clique
backed by scientifically illiterate bureaucrats.
It is time to act !
Chairperson, Science department, Quebec
--------------------------------------------------------
[5] ... As one of my collegues recently stated, you
never propose anything that is innovative to NSERC
because they'll surely cut you off.
Professor of Engineering, Windsor
------------------------------------------------------
[6] I support the formation of the Canadian Association
for Responsible Research Funding, and wish to be
associated with you in your endeavours...
Professor of Philosophy, Ontario
------------------------------------------------------
[7] Congratulations for your excellent article on NSERC
policies and your proposals for reform. Well done !
I totally agree with the need for reforms along the lines
you suggest. ... Enclosed is my article which makes the
case for "useless" research ... If I can help the cause
in any way, please let me know.
Professor of Mathematics, Quebec
----------------------------------------------------
[8] Dear Professor Berezin,
It is with great joy and enthusiasm that I have read
your news on the canadain Association for Responsible
Research Funding ... I can only applaud your efforts
and would like to get involved, if possible ...
I wish you much success in your quest for justice
and democracy for all and I'll appreciate keeping me
informed (and involved) on future developments
[2 page letter]
Professor of Physics, Alberta
-------------------------------------------------------
[9] I fully support your aims and share your concerns ...
Professor, McMaster
------------------------------------------------------
[10] ... it is very easy to ostrich out from the issues
which concern Hunter and Berezin, while under the
publish-or-perish threat... to have a really original
ideas about [area of research] is to place oneself in
a position of academic suicide, since it is unlikely
to be perceived as relevant by a peer review committee ...
Professor, Biomedical sciences, Ontario
--------------------------------------------------------
[11] Thank goodness sombody has at last started this
Society ! ... We must get back to the fundamental idea
that all univerisy researchers should be funded for
INDIVIDUAL RESEARCH. We must stop referees from deciding
that if you are not in a center of excellence your ideas
and work must be inferior. We must separate the research
grants from the support of graduate students ...
... FIGHT ON. GREAT IDEA! GOOD LUCK !
Professor of Science, McGill
-------------------------------------------------------
[12] I believe that it is more cost effective to support
more researchers at low funding levels than fewer
researechers at high funding levels ... I don't believe
that funding a greatefr number of scientists would result
in mediocre research. The peer review system of journals
will certainly help to maintain standards. If the
"publish-or-perish" attitude can be broken, I suspect
that the quality of research would actually increase.
Many researchers would likely prefer to publish fewer
but more substantial papers ...
Professor and Chaiperson (Biomedical),
Maritimes university
------------------------------------------------------
[13] I promised myself not to write any more on a
subject about which I have already payed my dues.
However, Berezin's and Hunter's "Myth" ... has pushed
me again into that old sin: writing. My first suggestion
was to change the name of GSC, "Grant Selection
Committee" into more appropriate one: GRANT SEGREGATION
COMMITTEE. ...
Professor and dean, Maritime university
-------------------------------------------------------
[14] I want to congratulate you for your article. I found
the article well written, appropriate and right on target.
I feel that Canada is wasting a tremendous amount of
research talent because of the presently ill-conceived
funding system that leaves the door open to any kind of
nepotism and corruption ... I hope that you will be able
to set up an independent study group to monitor NSERC
activities. It is about time that we have a breath of
fresh air in this field.
Professor of Science, Concordia
--------------------------------------------------------
[15] Dear Dr. Hunter:
Bravo to you and to Dr. Berezin for your commentary ...
... too much bureaucracy, too many committees ... They ...
don't give a damn for the quality of a researcher but
judging solely by his confirming to the present day fads ...
Professor of Science, Universite de Montreal
---------------------------------------------------------
[16] I just read your letter in the CAUT Bulletin.
I greatly appreciate this very timely contribution and
strongly support the views you express ... I would be
most happy to help...
Professor of Science, Ottawa
-------------------------------------------------------
[17] Dear Dr. Berezin,
First of all I must compliment you and Dr.Hunter for some
truly innivative ideas regarding funding of academic
researchers by NSERC and other agencies ... I believe it
is unnecessary to fund so-called productive researchers
to a very high level. By definition, such research should
have no problem obtaining funds from other sources ...
Professor of Engineering, McGill
-------------------------------------------------------
[18] Dear Alexander and Geoffrey:
I respond to your "Open Letter". First a note of thanks
for your time and dedication. After reading it I felt
intantaneously better and I would even dare to hope that
things might change if a dedicated group sets to work
on it. The policies of censorship, suppression of
innovative thought, reduction of our intellect by a
staightjacket of "fundability" imposed on us by NSERC
in the name of "excellence" are real ... The total
lack of free exchange of opinion and enforced confirmity
to the mainstream is indeed the consequence of NSERC power
politics. I find this academic environment oppressive and
intimidating and the need to shut-off one's intellect and
creativity, in order not to alienate the mainstremers that
include the mamagement, a severe price to pay for survival ...
I agree wholeheartedly with (and support) all the points
you made in your letter ...
Professor of Science, Winnipeg
------------------------------------------------------
[19] Dear Professors Berezin and Hunter:
We read with great interest your letter (CAUT bulletin).
Both of us are in accord with the ideas that you expressed;
indeed we have often voiced them ourselved with local
colleaugues. You may consider publishing this letter in
the Globe and Mail to bring the issue to the most appropriate
audience - the taxpayers across the nation ... We fully
support your endeavour and will continue to provide you
with any information which comes our way.
Two professors of biomedical sciences, Toronto
---------------------------------------------------------
[20] Dear Professor Berezin:
Your recent E-mail message was forwarded to me by a
colleague. Having read that, and the previous letter in
the CAUT Bulletin, I am writin to tell you that I agree
wholeheartedly with the views expressed in both ... I am
certainly enthusiastic about joining your association.
... I just hope that the proposed association is not "too
little too late" as I, and many of my colleagues, are
thoroughly disillusioned and disappointed in the way that
research funding is allocated in Canada and are actually
contemplating leaving research and/or Canada. As you
rightly point out, any attempt to discuss the matter
objectively is met with the "well, its the best system
in the world" response and is either ignored or
suppressed ....
You have my full support in your efforts to get an equitable
system in this country and if there is anything I can do to
help, please let me know. Good luck.
Professor and Chair (biosciences), Maritime university
--------------------------------------------------------
[21] I think the majority (overhelming majority) of the
scientific community is in broad agreement with you ...
First on the matter of small grants. The pressure from
the NSERC bureaucracy on the GSC to be more selective
is enormous and they keep telling us that every small
grant we award will affect our future funding ... The
unelected dictatorship is the NSERC bureaucracy
itself ... there is no support for pure research
in NSERC ...
Professor of science, current member
of NSERC committee
------------------------------------------------------
ADDITIONAL COMMENT (from Berezin):
The above quotes are just a small sampling from
over 150 letters and e-mail messages received by
Hunter and Berezin since the founding of CARRF
(May, 1994).
Most letters are unquestionably supportive to the
CARRF initiative. The only exception are some few
e-mail messages (3 or 4) which can be qualified as
a "shut up" hate mail. Invariably, they were from
well funded elitarian grantees, usually members of
Grant Selection Committees who have vested interests
in keeping the selectivity system (their system)
as it is. This, however, should not imply that all
GSC members are against CARRF objective. In fact,
several very supportive letters were obtained from
GSC members (former or active) who naturally asked
not to be quoted under their names. One such quote
(# 21 above) points that NSERC bureaucracy resists
the awards of small grants, contrary to the wishes
of many GSC members. Some of the above quoted people
actually had no objections to be publicly quoted
under their true names. However, for the purpose
of uniformity, I have decided to provide this
compendium anonymously. The latter should not be
seen as my agreement with the principle of anonymous
peer review assessment which, I believe, is one of
the most damaging misconceptions in science.
--------------------------------------------------
SOME REFERENCES ON PEER REVIEW AND FUNDING MODELS
Berezin, A. A. (1993). The SSC and peer review. Physics World
(Dec.), 19.
Berezin, A. A., R. Gordon & G. Hunter (1995). Anonymous peer
review and the QWERTY effect. Amer. Physics Soc. News,
March 1995.
Berezin, A. A. & G. Hunter (1994). Myth of competition and NSERC
policy of selectivity. Canadian Chemical News 46(3), 4-5.
Forsdyke, D. R. (1983). Canadian medical research strategy for
the Eighties I. Damage-limitation or superelitism? Med.
Hypotheses 11, 141-145.
Forsdyke, D. R. (1983). Canadian medical research strategy for
the Eighties II. Promise or performance as the basis for the
distribution of research funds? Med. Hypotheses 11,
147-156.
Forsdyke, D. R. (1989). Sudden-death funding system. FASEB J.
3(10), 2221.
Forsdyke, D. R. (1989). A systems analyst asks about AIDS
research funding. Lancet 2(December 9), 1382-1384.
Forsdyke, D. R. (1991). Bicameral grant review: an alternative to
conventional peer review. FASEB J. 5, 2312-2314.
Forsdyke, D. R. (1992). Bicameral grant review: how a systems
analyst with AIDS would reform research funding.
Accountability in Research 3, 1-5.
Forsdyke, D. R. (1993). On giraffes and peer review. FASEB J. 7,
619-621.
Forsdyke, D. R.(1994). Authorship and misconduct. Nature 370, 91.
Forsdyke, D. R. (1994). A theoretical basis for accepting
undergraduate academic record as a predictor of subsequent
success in a research career. Implications for peer review.
Accountability in Research 3, 269-274.
Gordon, R. (1993). Grant agencies versus the search for truth.
Accountability in Research: Policies and Quality Assurance
2, 1-5.
Gordon, R. (1993). Alternative reviews. University Affairs
(Assoc.of Universities and Colleges of Canada) 34(6), 26.
Horrobin, D. (1981/1982). Peer review: Is the good the enemy of
the best? J. Res. Communic. Stud. 3, 327-334.
Horrobin, D. F. (1990). The philosophical basis of peer review
and the suppression of innovation. J. Amer. Med. Assoc.
263(10), 1438-1441.
Kenward, Michael. (1984). Peer review and the axe murderers",
New Scientist, 102 (1412), p. 13 (31 May, 1984).
McCutchen, Charles W. (1991). Peer Review: Treacherous Servant,
Disastrous Master. Technology Review, vol. 94, #7,
(October 1991), 28-40.
Osmond, D. H. (1983). Malice's Wonderland: research funding and
peer review. J. Neurobiol. 14(2), 95-112.
Savan, Beth. (1990). Science Under Siege (The Myth of
Objectivity in Scientific Research, CBC Enterprises,
Toronto, 1988.
Szent-Gyorgyi, Albert. (1972). Dionysians and
Apollonians, Science, 176, 966 (1972).
----------------------------------------------------------
