OERA Peer Review Process

The OERA draws on independent subject matter experts as peer reviewers to assist both with its proposal vetting practices, and (once a project is underway), with its report review processes.  Engaging peer reviewers is an integral part of the OERA’s business model, that allows for independent arms-length oversight to help guide its research decision-making.

Peer reviewers are sourced from government (federal and provincial), academia, not-for-profits and industry, and can be local, regional, national and/or international.  All peer review work is undertaken on a volunteer basis.  The selection of experts is dependent on the research subject area and scientific and technical requirements of the proposal or project.  The intent is to ensure that research decision-making is guided by knowledgeable individuals who are actively working in the sectors that are the subject of research proposals or projects.  Soliciting external expertise is core to ensuring a fair and independent research process with the goal to fund the best and most relevant research to develop the offshore energy sectors.

Section A: Peer Review Process for Proposals:

Open Call Program and Smaller Scope RFPs:

As a first step, all proposals received under the Open Call program ($20,000 cap) and smaller scope requests for proposals (RFPs) (<$75,000), are evaluated by OERA staff to ensure the minimum eligibility requirements and criteria have been met.  This is to ensure only qualified submissions are forwarded to the external reviewers.

Following, the OERA will then solicit two or three external subject matter experts to complete a peer review for each proposal that has met eligibility.    One of the reviewers is always sourced from the Department of Energy (either from the Renewables or Petroleum Resources group as appropriate to the proposal), who reviews all submissions.  The peer reviewers are given approximately three weeks to complete the proposal evaluation.  For each proposal, the peer reviewer completes a (quantitative) evaluation using a standard Evaluation Form.

Once all the evaluation forms have been submitted to the OERA, OERA staff collate and summarize the results then consult and discuss with DOE to arrive at its recommendations for funding.  The recommendations are then presented to OERA’s Executive Director and subsequently to OERA’s Board of Directors for final approval.

The peer review and approvals process for Open Calls/smaller RFPs takes approximately 11-15 weeks to complete.

Larger Scope RFPs/Competitive Calls:

For larger scope RFP competitions, the OERA first receives a Letter of Intent (LOI) followed by a full proposal from those invited to do so. Both must be peer reviewed.  The OERA assembles a Special Committee to complete this two stage review. The OERA selects individuals to participate on each Special Committee depending on the needs of the research competition and any technical requirements expressed by the Department of Energy. The Committee types used are one of: Project Management Committee (PMC), Research Program Technical Committee (RPTC), or a Technical Advisory Group (TAG).

Special Committee recruitment follows the same practice as for Open Call peer reviewers, that is, talent is drawn from government, industry, and academia from across the world (noting that most Committee members are sourced locally from Nova Scotia).  Typically, each Committee comprises five or six members, including mandatory participation by DOE and OERA representatives.

Each Committee member is asked to review and evaluate all LOIs and invited full proposals, using the standard evaluation form or a modified form tailored to the nature of the proposal and scope of work.  OERA staff too, undertake an evaluation of each LOI and invited full proposal, with a particular focus on eligibility requirements.  Three to four weeks are given to complete the evaluations.

In some instances, Special Committee reviewers or OERA staff may have questions or conditions for the proponent to address.  Where applicable, any new or updated information acquired from the proponent is then shared with the Committee for a final assessment.  This step may influence whether the proposal can move forward for funding consideration or removed from further consideration.

Once all the required information has been collected, the Special Committee convenes to discuss and make recommendations for funding.  The recommendations are then presented to OERA’s Executive Director and subsequently to the OERA’s Board of Directors for final approval.

The peer review and approvals process for larger scope RFPs can take approximately 18-23 weeks to complete.

Section B: Peer Review Process for OERA-Funded Project Reports:

Proponents awarded OERA funding are obliged under contract to submit project interim reports and a final report.  The number of interim reports required depends on the length and complexity of the project.  For Open Call projects, each report is peer reviewed by two to three subject matter specialists, inclusive of one DOE representative.  Again, peer reviewers are drawn from a range of organization types and different geographies.  Often, the OERA will use the same individual(s) who completed the initial proposal peer review.

Reports received under larger scope projects may be reviewed by the either the entire Special Committee, or select Committee members or other external peer reviewers.  The number of and sourcing of peer reviewers depends on the complexity of the project and/or required expertise to address a specific project issue.  The peer review of an interim or final report considers four questions, specifically:

  1. Based on the information provided in the Report, is the project on track to meet the targeted objectives? Please highlight any concerns/issues related to meeting targeted objectives.  (For final reports, has the project met the objectives);
  2. Is the research proceeding in a technically/scientifically sound manner? – please highlight any concerns.  (For final reports, is the completed research technically/scientifically sound?);
  3. If any, please comment on project risks and how they are/have been managed; and
  4. Please provide any additional general comments on the Report.

Where necessary, OERA staff may add to the above list of questions for the peer reviewers to consider while undertaking the report evaluation.   Such report reviews are essential to the overall research process and the OERA’s due diligence.  For interim reports, the evaluation will determine if a next payment can be issued, or for a final report, if the holdback amount can be released.   Peer review evaluations are not typically shared with project researchers.  All report peer reviews are kept on file and/or archived.

General:

All peer reviewers are bound by a Confidentiality Agreement to ensure proprietary information is not inadvertently shared without permission.  To ensure independence, the names of the peer reviewers are kept confidential in most circumstances although exceptions can be made provided all parties agree.

All evaluations completed by the OERA and external peer reviewers and/or Special Committees are kept on file and in confidence by the OERA.  OERA staff members are responsible for coordinating, managing and documenting all evaluations as justification to support funding and gauge proponent compliance with RFP requirements.

See Flow Chart Here