Tag Archives: CIHR

Update on withdrawn CIHR trials policy

In an only somewhat-overdue update (thanks to conference season interrupting my regular blogging activities – I do write on the road, but need to get sleep & give a read over before I can push “publish” on a post) the Canadian Instutites of Health Research (kind of the Canadian NIH, for US American readers) has put out a new message regarding the missing CIHR trials policy that we’ve been following since late March.

To backtrack a bit, while I was on the road and having fun with research and colleagues over the past month, there was more coverage of the CIHR trials policy disappearance, including Michael Geist’s blog and the National Post. Additionally, “rapid response” letters from around the world continued to roll in to the BMJ related to their article, some with great titles such as, “Canadians step back from the well of transparency while the World is thirsting for it” and “CIHR decides it must compromise my interests as a patient.”

Then, right around June 15 or so (I saw it on the 17th), CIHR President Dr. Alain Beaudet issued a “Message from the President – Policy on ethical conduct of research involving humans.” Go read it: http://www.cihr-irsc.gc.ca/e/43756.html. If you’re anything like me, you may need to read it a few times over, because it’s not the clearest statement ever made.

Here’s what I think it’s saying:

  1. That the March removal of the trials policy (“Registration and Results Disclosure of Controlled and Uncontrolled Trials”) was about “harmonization” and deference to the TCPS-2
  2. The TCPS-2 has some requirements for trial registration and public disclosure
  3. While the trials disclosure requirements in the TCPS-2 and the former CIHR trials registration policy were in the same spirit, the CIHR policy had more specific directions about what needed to be done
  4. CIHR will (at some unspecified point) be  integrating certain of these more stringent operational requirements as part of the terms and conditions of its “relevant programs.” These include: a) publication of the systematic review used to justify the trial, b) registration and compliance with WHO requirements for minimum data disclosure, and c) submission of final reports in CONSORT format.
  5. CIHR will propose 4 revisions to the TCPS-2 for “prompt consideration” (not clear on how soon this can/will happen): a) applying to all trials, not just clinical trials, b) requirement to update trial registration when the trial protocol changes, c) requiring that serious adverse events be reported in post-trial publications, and d) and a requirement to deposit aggregate data in an unbiased, publicly accessible database.
  6. In the interim, CIHR will specify that researchers have to “comply with all the requirements mentioned above” (not sure whether this means 4a-c or is also inclusive of 5a-d).

So, what does this mean? Are we all good now?

Well, we’re better than we were before the press coverage, I think. We’re not as better as we would have been, had the trials policy never been pulled.

Remaining questions:

  1. Really??? I’m still kind of skeptical that 3 months after the trials policy and the TCPS-2 came out, both of which had been in development for a looong time, someone suddenly just went, “Oh, gosh, you know what? It’s not okay to have both of these policy statements!” Why am I skeptical? Well, because it just doesn’t make sense. CIHR had a trials policy that wasn’t 100% the same as the TCPS-1. Tri-council funders have all sorts of different policies that are more stringent than the TCPS, and it’s not a problem (e.g., the beloved CIHR Access to Research Outputs policy). It’s just not adding up, and at this point the message seems to be that it doesn’t matter if it’s not adding up, CIHR is sticking to their story.
  2. When will the CIHR be implementing these new requirements for relevant grant programs, and how will this implementation be different from the trial policy rules?
  3. What’s the process for revisions to the TCPS, and how long does it take?
  4. When are we going to see the requirement to make individual-level/micro/”raw” data publicly available? The item listed above in 5d, which currently has no teeth, only requires aggregate data deposit. What does this mean? How aggregate? Does this have to include all adverse events? We need to be able to reanalyse this data to look for harms to specific groups. This is one of the most important parts of the scrapped trials policy, and there is no mention of it in the new statement from CIHR.

I think the international attention and public pressure on CIHR over the withdrawal of the new trials policy likely contributed to these developments, which seem like a step back in the right direction. However, without teeth in the current requirements, and a return of the publicly-accessibly micro-level data archivng requirement, it seems like 3 steps forward, 2 steps back at this point.

-Greyson

Previous posts on this topic, in case you haven’t been following along:

Leave a comment

Filed under democracy, ethics, funding, government, Health, OA, research

BMJ article about the CIHR trials policy disappearance

More follow up from these previous posts about the surprise disappearance of the Policy on the registration and results disclosure of controlled and uncontrolled trials funded by CIHR.

Thursday, the British Medical Journal (BMJ) published a News article by Ann Silversides, titled: Withdrawal of clinical trials policy by Canadian research institute is a “lost opportunity for increased transparency.”

For consistency’s sake I must note my dismay that this BMJ article about open data is not freely accessible online. Open access is as open access does; journals who live in glass open access houses should be cautious about criticism, folks.

Since some readers of this blog undoubtedly lack subscription access to the BMJ, here are a  few choice quotes from Silversides’ article, interspersed with my commentary:

“The CIHR [Canadian Institute for Health Research] policy
certainly was leading the drive towards increasing transparency,”
said An-Wen Chan, a scientist with the Women’s College
Research Institute in Toronto and co-author of the Ottawa
Statement on Principles and Implementation of Clinical Trial
Registration and Results Reporting

If the institute’s policy is permanently rescinded, the result
would be “a lost opportunity for a federal funding agency to
make a statement that increased transparency is important for .
. . ensuring that publicly funded research has maximal impact,”
said Dr Chan.

I don’t know Dr An-Wen Chan, but I agree entirely. The policy seemed to follow naturally along the progressive path forged by the CIHR’s 2004 RCT registration policy and 2007 research access (OA) policy.

Trudo Lemmens, of the University of Toronto law school, said
the decision to remove the policy “sends a bad message.” The
TCPS-2 requirements are more general and vague than the CIHR
policy, and it is not clear who will implement and enforce
TCPS-2, he said.

Again, I don’t know Dr. Lemmens, but I agree that the rescinding of the policy sends a “bad message.”

Silversides also reports that:

The CIHR has recently been “in discussions” with Rx&D, the
trade association for Canada’s brand name drug companies, on
renewing the CIHR/Rx&D collaborative research programme
(which funds awards, grants, and clinical trials) and finding
ways “to improve the clinical trial environment,” Rx&D
president Russell Williams stated in an email.

I suppose what is an improvement to the clinical trial environment depends on your point of view. Making clinical trials more transparent, and thus more likely to be audited, by more sets of eyes, for human safety issues, seems to be an improvement to me. But if my primary concern was the ease of bringing new technology/product from trial to market, my perspective would likely be different.

Professor Lemmens, who has published widely on transparency
and clinical trials, said that at an international level, the
pharmaceutical industry has been critical of trial registration
and requirements about having to give details of results. He
speculated that “maybe it is not pure accident” that the CIHR
policy, widely regarded as pushing for more transparency in
clinical trial reporting, has been withdrawn “when there is a
clear push (in CIHR) to promote collaborative research with
industry.”

And this possibility that the timing of the trials policy’s disappearance might not be “pure accident” is exactly what we don’t want to think about when it comes to our national health research funder, with a mandate:

“To excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened Canadian health care system.”

-Greyson

Leave a comment

Filed under ethics, funding, government, Health, privatization

Details: TCPS-2 vs the CIHR trials policy of 2010

Thanks to a few days’ time and some help from people with more experience reading science policy, I now feel that I can expand on my previous post about the TCPS-2 “superseding” the Dec 2010 CIHR trials policy.

First of all, I note that I am putting “supersede” in quotes not merely because it is a really cool word to which I wish to draw attention (although it is) but because the TCPS-2 was officially launched before (not, as I had previously thought, simultaneously with) the new Trials Policy, and as such, would not likely have been intended as a replacement for another policy that had not yet been unveiled.

Non-exhaustive list of differences between the TCPS-2 and the CIHR trials policy of 2010:

  • Unlike the TCPS-2, the CIHR policy required that the systematic review that justifies doing the trial be publicly cited.
  • TCPS-2 is specifically about clinical trials, whereas CIHR could be interpreted to apply to a wider scope of “controlled and uncontrolled trials.”
  • The CIHR policy required following the WHO international standards, whereas the TCPS-2 only requires that trials be registeres in registries that meet the criteria of WHO/ICMJE. Seems like a small distinction, but the difference is the minimum dataset required.
  • This one is kind of nitpicking/pointing out an oversight, but the CIHR required the name of the trial registry as well as the registation #, whereas the TCPS-2 only asks for the # (kind of useless without the name, and presumably they want the name too, but didn’t specify).
  • The CIHR policy took steps to prevent duplicate/multiple registration, whereas the TCPS-2 does not address this potential issue.
  • The TCPS-2 says that “Researchers should also promptly share new information about an intervention with other researchers or clinicians administering it to participants or patients, and with the scientific community – to the extent that it may be relevant to the general public’s welfare,” which does not require public disclosure. The CIHR policy required the data to be made publicly available to all.
  • Research design amendments (changes) require ethical approval under the TCPS-2. The CIHR policy required that amendments  to the research design be reported within 30 days after the ethics approval.
  • The TCPS-2 asks for “new risks” or “unanticipated issues that have possible health or safety consequences for participants” during the trial. CIHR policy did not ask for adverse event info until after the trial.
  • TCPS-2 asks for info that might merit or lead to early stopping of  a trial. CIHR policy wanted notification and public disclosure within 30 days of stopping a trial early.
  • The TCPS-2 tells researchers “to make reasonable efforts to publicly disseminate the findings of clinical trials in a timely manner by publications and by the inclusion of raw data and results in appropriate databases,” whereas the CIHR policy specified reporting guidelines (e.g., CONSORT for RCTs) and required reporting and public disclosure within 12 months of the end of the trial, building on the existing CIHR research access policy.
  • The TCPS-2 encourages researchers and institutions to publish results in a timely manner. The CIHR policy required public disclosure within 12 months and reserved the right to disclose the final report themselves within 18 months.
  • The TCPS-2 asks for “the inclusion of raw data and results in appropriate databases” whereas the CIHR specified what appropriate databases are and that both micro (aka “raw” aka “participant level”) as well as macro (aka “aggregate” aka “summary”) level data are necessary.
  • The TCPS-2 talks about ethics with regard to confidentiality clauses and PI access to trial data, whereas these issues were not addressed by the CIHR trials policy.
  • The CIHR policy talks about after-trial follow-up, including submission of any “severe adverse events or harm” to the publicly-available trial registry and a requirement to “retain all trial information including original micro-level data and metadata data for twenty five years unless they are deposited in a freely accessible data repository (to align with the Health Canada requirements).”

Generally speaking, major differences are:

1) Insider vs public disclosure of information

TCPS-2 is concerned with sharing info with the research ethics board and “to other researchers or clinicians administering it to participants or patients, and with the scientific community.”

CIHR was concerned with reporting and disclosing info “to CIHR and the trial registry” – and the trial registry had to be openly accessible to the public.

2) Whether to specify how much should be reported, and how soon to report it

The TCPS-2 advises on ethical matters across disciplines. It makes few specific mandates as to timeliness or data specifics.

The CIHR policy was intended specifically for trials funded by CIHR, and similar to other funding policies could and did specify details of what kind of data should be reported (all, macro and micro), to whom (CIHR and a publicly accessible registry), and when (within 12 months of trial completion, or with some types of data 30 days of early stoppage of a trial).

TCPS-2  on its own vs. as part of a comprehensive approach

I’m not saying the TCPS-2 is bad. It’s pretty good, overall, and after years and years of collaboartive work and revision seems to do an admirable job of doing what it’s supposed to do – which is to set out general, interdisciplinary, national ethical guidelines, not details of practice requirements. Without specific procedural policies designed to instruct researchers in various disciplines, the TCPS-2’s appropriately limited scope leaves us with somewhat vague directions.

The two policies, the TCPS-2 and the CIHR trials policy of 2010, are clearly intended to complement with each other. As I noted before, the TCPS-2 specifically states that:

“[Trial] registries, in addition to agency policies, editorial policies, ethical policy reforms, and revised national and institutional ethics policies and results disclosure requirements, contribute to a multi-faceted approach to eliminate non-disclosure.” (emphasis mine)

I continue to be very disappointed that the CIHR has apparently seen fit to retroactively withdraw their facet of this approach, and I do think that as a public agency they do owe the public a decent explanation of what happened here. What has been made publicly available thus far does not add up. As I have stated previously, there may well be valid reasons for killing the new policy on clinical trial data, but the lack of transparency around the policy retraction continues to be troubling.

-Greyson


2 Comments

Filed under censorship, democracy, ethics, funding, government, Health, OA

Follow-up: CIHR trials transparency policy

Here is the official word from the CIHR on the clinical trials transparency policy that was so transparent that no one could see it:

According to  Dr. Ian Graham, Vice-President, Knowledge Translation, the new Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS-2) supersedes the Policy on registration and results disclosure of controlled and uncontrolled trials funded by CIHR.

The trials registration & disclosure policy page that was recently a 404 page is now a redirect page to the TCPS-2.

I find this response unsatisfying. Here’s why:

  • I don’t understand why a policy would supersede another policy announced at the same time. Both policies were in development for years, but they didn’t notice until three months after they were simultaneously unveiled that they were redundant? Seems unlikely.
  • It also seems odd that CIHR would send a KT Senior Advisor on a tour around the country educating people all about a new policy they would then suddenly decide was obsolete.
  • While I have been assured that “CIHR is committed to transparency in clinical trials,” I have not gotten a clear answer as to whether or not all the exact requirements of the apparently-obsolete CIHR policy are included in the TCPS-2. (Am working on going through it with a fine-toothed comb, but it is >200 pages long and I have a head cold, so it’s slow going. ) I am particularly concerned about the preservation & disclosure of individual-level data and reporting of all adverse events in clinical trials.
  • The TCPS-2 clearly states (Article 11.3) that “[Trial] registries, in addition to agency policies, editorial policies, ethical policy reforms, and revised national and institutional ethics policies and results disclosure requirements, contribute to a multi-faceted approach to eliminate non-disclosure.” It is concerning that CIHR apparently does not feel that the  CIHR policy facet of this “multi-faceted approach” is important.

If the TCPS-2 does ensure the same clinical trial reporting standards as the recently-retired-at-a-young-age CIHR policy, then this is possibly just an embarrassing right-hand-doesn’t-kn0w-what-left-hand-is-doing situation.

If, on the other hand (er, that would be the third hand in the middle, I guess?) the TCPS-2 is not as specific in mandate for clinical trial registration and reporting as the CIHR policy was, this is a great loss for open access, open data and public health and safety. Making detailed clinical trial data publicly available for third-party re-analysis is the best way we have to provide safeguards on drug approval, safety & effectiveness processes – processes that CMAJ rightly criticizes as opaque and secretive.

CMAJ has its own conflicts of interest, certainly, and I haven’t let them off the hook for that. But they are not the only ones. We must be vigilant not only about transparency in our scholarly publishing processes, but also about our government health agencies, including those that fund scientific research.

I would hope that CIHR’s relationship with, say, pharmaceutical companies (and their former employees), would never dictate their clinical trials policies. But I want the transparency to know that is true, for sure. And I’m certainly not getting it around this question right now.

If anyone has more info on this, I’m still looking for insight here, so please contact me.

-Greyson

3 Comments

Filed under democracy, digitization, ethics, funding, government, Health, OA

The mystery of the missing CIHR trials policy

Who stole the Canadian Institutes of Health Research’s trial transparency policy?

Canadian health researchers report that the policy, only four months old, went missing sometime in mid-March. The policy’s full name is Policy on the registration and results disclosure of controlled and uncontrolled trials funded by CIHR.

It was last seen in the vicinity of the CIHR/IRSC website, at http://www.cihr-irsc.gc.ca/e/42831.html. The policy is described as an expansion of a previous CIHR policy, aimed at increasing clinical trial transparency and reducing biased disclosure of trial results.

We have evidence that the policy was alive not long ago.

1) Canadian health researchers around the country have received the following announcement of this new policy via e-alerts and newsletters over the past few months:

CIHR announces new policy
Policy on the registration and results disclosure of controlled and uncontrolled trials funded by CIHR (http://www.cihr-irsc.gc.ca/e/42831.html)
In 2006 the CIHR endorsed, in principle, the World Health Organization (WHO) international standards for clinical trial registration. Consequently, CIHR has updated its policy effective 20th December 2010. The new Policy will apply to all competitions with application deadlines after 1st January 2011.
The new Policy requires researchers awarded CIHR funding to:
* Register a trial in one of the WHO primary registries or      ClinicalTrials.gov prior to participant recruitment;
* Regularly update the information during the trial;
* Report and publicly disclose trial results; and
* Retain all trial information for 25 years.

The updated Policy expands on the 2004 policy on RCT registration. This Policy complies with the WHO International standards, ICMJE requirements and the Declaration of Helsinki.The Policy will help ensure that clinicians, researchers, patients and the public have access to information about CIHR-funded trials. The aim is to increase transparency and accessibility of trials by their prospective registration and disclosure of results, thereby reducing publication bias and fulfilling ethical responsibilities.
The new Policy can be accessed via the CIHR Funding Policies web page (http://www.cihr-irsc.gc.ca/e/204.html)

2) The CIHR glossary refers to this policy (under “Results”).

3) There’s a nice archived abstract and PDF of the slides from a presentation explaining the origins, development and details of the new policy, “Towards Greater International Transparency of Clinical Trials – Short Term Efforts for Long term Benefits: CIHR Trial Policy 2010” given in February by Karmela Krleza-Jeric,MD, M.Sc., D.Sc.

But the policy hasn’t been seen in at least a week, possibly longer.

The URL to which the above sources refer as the policy document on the CIHR website is currently a 404 Error page.

At first, it might seem to be merely a website glitch. However, the policy is also missing entirely from the above mentioned CIHR Funding Policies web page, despite the fact that the other recently added policy on Gender and Sex Based Analysis is listed, and the page currently says it was last updated a week ago on March 28, 2011.

Whodunnit?

Policy wonks continue to search for the missing policy, last seen at least one week ago.

At this point, authorities can only speculate on motives for this disappearance. However, the spectre of institutional conflict of interest has been raised.  At this point, we are making an appeal to the public to please contact us with any information you may have about the policy’s disappearance.

Anyone with information on the current whereabouts of the CIHR Policy on the registration and results disclosure of controlled and uncontrolled trials is asked to please leave a comment below. We hope to see this young policy safely back at home again as soon as possible.

Thank you.

-Greyson

ETA – Cached copy of the policy text is now available here.

ETA2 – Follow up post here. And another here. Also here.

3 Comments

Filed under censorship, digitization, funding, Health

Hark – PubMedCentral Canada on the horizon!

Thanks to Dean Giustini for the original heads-up on this:

In a press release titled “Canada joins international effort to provide access to health research,” the NRC (parent organization of CISTI, the de facto Canadian national library of science & medicine)

PubMed Central repository will open new pathway to Canadian health research

July 06, 2009, Ottawa, Ontario

Accelerating the development of discoveries and innovations and facilitating their adoption through free and open access to research findings. This is the aim of an important new initiative that will provide researchers and knowledge users free access to a vast digital archive of published health research at their desktop and connect them to an emerging international network of digital archives anchored in the United States.

The National Research Council’s Canada Institute for Scientific and Technical Information (NRC-CISTI), the Canadian Institutes of Health Research (CIHR), and the US National Library of Medicine (NLM) have announced a three-way partnership to establish PubMed Central Canada (PMC Canada). PMC Canada will be a national digital repository of peer-reviewed health and life sciences literature, including research resulting from CIHR funding. This searchable Web-based repository will be permanent, stable and freely accessible.

Yay!

This has been a long time coming, and I know I’ve been but one of a gazillion nags pestering folk at CISTI and CIHR about when PMCCanada was going to actually happen. Between translation issues (the two existing PMC repositories are both English only), funding cuts, and who knows what all else, it’s taken quite a while for PMCC to go from drawing board through discussion to reality.  And to be sure, this is still short of reality, but an official press release gives reason to hope!

PMCCanada could really help Canadian health research funders, many of whom now have OA policies requiring free access to research outputs within 6 months of publication, guide grantees to a specific deposit location if they so desire (or at least offer a clear option to those without institutional repositories). CIHR sort of already guides researchers to PMC, but the CIHR policy came about much more quickly than the Canadian PMC repository did, so this could only be one of multiple options.  I wonder if CIHR will now require deposit in PMCCanada, similar to the NIH’s PMC deposit policy.

I know some Canadian journals have worked to become accepted into PMC by now, and I’m not sure if they will get shifted over, or exactly how the territory between PMC affiliates is divvied up/shared.  It’s a bit of a confusing time for authors these days, with some journals depositing automatically into PMC, some not (or not fast enough to comply with all funder mandates), some not even eligible to do so, and multiple deposit processes still required to deposit into multiple repositories (e.g. a university institutional repository and PMC).

Anyway, kudos to those who have worked long and hard on this project. It will be great in principle to have a PMCCanada, it may make it easier for CIHR (and possibly other finders) to check on compliance with their access policies, and we’ll just have to figure out what the existence of PMCCanada means in practice.

-Greyson

5 Comments

Filed under government, Health, OA, preservation, publishing

CIHR using OSS for learning modules

The Canadian Institutes of Health Research (CIHR) recently unveiled three knowledge translation learning modules, the first of their new CIHR Online Tutorial courses.

I don’t know much about the background or driving force behind creation of these modules, but from the website it looks like the plan is to develop learning tutorials in several categories, with KT leading the pack.

CIHR has been a Canadian leader in encouraging Open Access to research outputs, and I was pleased to notice that they are also showing leadership in using Open Source Software, as the Learning Modules are run with the open source course management software Moodle.

I use Moodle in some classes I teach, and my non-systems-administrator perspective is that while it’s not perfect, it’s no harder/more frustrating to use, and certainly more versatile than proprietary software packages such as Blackboard or WebCT.

Kudos to CIHR for efficient use of public resources and for quietly demonstrating professional use of OSS!

-Greyson

Leave a comment

Filed under government, Health, OSS, technology, tips and tools