Frequently Asked Questions
Improving Teacher Quality - Research
Are controlled experiments required for the research portion of the grant? What level of statistical power is required?
The research portion of the grant should include an experimental or quasi-experimental design with sufficient controls to establish that the results measured at the project's completion can reasonably be inferred to be linked to the intervention. Most projects have been able to identify similar cohorts to that which receives the intervention in order to secure data for control purposes. There are no specific requirements regarding statistical power, but projects should aim for the best result they can obtain within the fiscal parameters of the project and practical issues on the ground. Overall, projects will be expected to gather and report evidence of impacts on teacher practice and knowledge and on student achievement that result from the intervention.
Are there sources of assistance for project directors in developing their research plans?
The Commission has recently published a field guide to assist project staff in determining their research needs and developing a plan to meet them. The Field Guide offers a basic review of scientifically based research and offers many checklists to help project staffóboth program managers and researchersódevelop a solid research plan. Download the Field Guide on our website at: http://www.cpec.ca.gov/FederalPrograms/FieldGuide.pdf
Can an IHE partner perform the evaluation research?
There is no rule against an IHE partner (including a member of the leadership team) serving as research director. If this is proposed, steps must be taken to insure against bias, and evidence will required of the person's qualifications. In the event the project will seek research assistance outside, there are many resources to identify organizations and individuals who are qualified as research professionals. See Part II, Chapter 2 of the SBR Field Guide for more information about locating and working with research consultants: http://www.cpec.ca.gov/FederalPrograms/FieldGuide.pdf
Could we involve the campus IRB as a system of checks and balances?
The Institutional Research Board (IRB) on your campus is not expected to be a participant in your grantóthat is not their role. You will have to assure that your evaluation research is submitted to and approved by the IRB. Suggestions and recommendations made by the IRB may help you identify and address problems in the research plan, which can serve as a form of "checks and balances."
Does the whole school model rules out random assignment? Can you give us an example of what type of research design we might use?
The whole school model does not automatically rule out random assignment. Many feel that random assignment is too hard to do in educational research, but there are creative ways to think about random assignment which might be of particular use in this initiative. See Part 1, chapter 3 of the Scientifically Based Research Field Guide for more on this topic:
The specifics of research design are so individual to each proposal that we cannot make a general statement about which design might be best. Finding an experienced and thoughtful researcher to work with who will listen closely to what you are trying to do and will be thoughtful about developing a robust research plan is the best place to start. Guidance on research design are available at CPEC's technical assistance workshops prior to a competition, and the audio recordings of one meeting are archived on the CPEC website. To register to attend a workshop and for more information, visit the Improving Teacher Quality
How do we provide research references in the proposal?
In those sections of the project narrative that cite research studies, proposers should use APA style to provide the citations (parenthetical citations at the end of the material cited). The list of references must be included as a separate document following the project description, and is limited to a maximum of 2 pages. It will not count against the total page limit for the narrative.
In addition to IRB approval, do we need to get parent permission?
You should be able to develop a research plan that will meet with campus IRB approval without having any need to seek parental permission if you plan to use only aggregated statistical data on students. Student data provided by the LEA can and should be "scrubbed" of all student identifier information and provided in a form that does not permit links to any particular student. If your research design includes student interviews or similar data collection, or it deals with students enrolled in an "academy" model, you may need to secure parental permission. IRB requirements, in general, will be determined by the research design.
What are you getting at in terms of dissemination? Who are we disseminating to, etc?
Dissemination is largely up to the project, and most normal forms of dissemination include journal articles, reports at conferences and meetings, and the like. It is highly desirable that project directors also seek to disseminate information on the project outcomes among their own partners or others in the region, and that the research findings are made available for possible replication.
What are you looking for in a control school?
Proposers should enlist the assistance of a professional research consultant in designing their control plan. There are many "correct" ways to approach this issue, but the bottom line is that you will be expected to get as good a match as you can possibly find.
What percent of the total budget is assumed to be allowable for the research portion of the grant?
A figure between eight and fifteen percent of the total grant is generally considered within a reasonable range. Any amount above that range will require additional justification based on project specifics.
Where can someone find assistance in preparing the research plan?
Proposers are strongly encouraged to consult the CPEC publication: Examining Educational Experiments: A Field Guide for Conducting Scientifically Based Research.
This publication can be found on the CPEC website at: http://www.cpec.ca.gov/FederalPrograms/FieldGuide.pdf
Who should projects utilize to manage the research portion of their grants? Is it required (or prohibited) to contract with research experts outside of the institution submitting the proposal?
Many projects secure the assistance of external research experts to help design and implement their research plan. This is not required, and is up to the discretion of the institution. Whoever serves as research director should have qualifications and experience that substantiate their ability to design and manage credible evaluation research.
View all FAQs.
E-mail a question to Webmaster@CPEC.CA.Gov