More On John White’s Spring 2015 PARCC Fraud

Mercedes, myself and a few others discussed the implications of these no bid micro-contracts this morning. I don’t expect the majority of BESE that unwaveringly supports John White and all his fraud and malfeasance to reign him in over contracting for a test the approve of anyway. I would hope that Kristy Nichols would take a look at this situation however. Over the summer Nichols and DOA reduced John White’s contracting authority from 20,000 to 2,000 on July 12th to prevent John White from circumventing state procurement laws and procedures for contracting services. If anyone is going to put a halt to this obvious subversion of the Governor’s will and the spirit of the State’s contracting laws it will have to be DOA, Nichols and Jindal. I would contact those folks and let them know what you think about White’s latest scheme and ask them to intervene.

deutsch29

When it comes to Louisiana’s supposed “PARCC test,” we are flying solo– and by the seat of Louisiana Superintendent John White’s pants.

Not at all an encouraging thought.

Louisiana is not included among the ten states plus DC scheduled to administer the legitimate, Peason-developed PARCC common assessments in Spring 2015. (Read here, and here, and also here for the evidence.)

White has been downplaying this fact before an oblivious media and a stunned Louisiana public– administrators, teachers, and parents alike who have been subtly misled by White to believe that Louisiana is somehow connected to the 2014-15 PARCC common assessments.

It just is not so.

Yet White knows he must concoct some arrangement to test Louisiana students and convince all involved that his 2014-15 testing product is “PARCC-like” in order to keep the “PARCC” ruse alive so that he can try to really get Louisiana into legitimate PARCC in 2015-16. To carry this off…

View original post 655 more words

Advertisements

I’m back. Did you miss me?

Hi guys!

I’m back from an outstanding cruise I did not warn you all I was taking for fear of being robbed whilst being out of town. Sorry about that, but I tried to keep my time away to a minimum and I had this planned for around 11 months.  I gave this to my wife as a Valentine’s Day present and since I attended I campaign event on our anniversary this year I really needed to make sure I followed through well on my pseudo-apology tour.

Fellas: I learned no matter how many times your wife tells you it’s okay to go to somewhere on your anniversary, even if you took her out the night before and bought her jewelry, it’s never okay. It’s a test and you are eternally doomed to failure if you believe her words and not your history together.  (If you catch yourself thinking, “wow, I never thought she would go for that!”  Chances are she didn’t and those clipped sentences and terse demeanor are your last warning before things go topsy-turvey –  so make sure whatever it is your are doing is worth the silent treatment and cold doghouse dinners you will be getting for a while.)

Ladies: we will probably promptly forget this advice for your birthday, my birthday, father’s day, mother’s day and some anniversary we met but we will ask you first and want to believe you are telling us the truth.  You can make this easier on both of us by just telling us “no.”  We probably shouldn’t have asked, but we really wanted to do whatever it was and want to believe you will tell us the truth.

Oh well, I doubt I’ve improved Martian – Venusian relations much, but I tried.

On the plus side, I may have had an impact on a local election.  Rickeem Jackson of the non-resident running for local elective office Rickeem Jacksons, who has never paid taxes or voted in an election prior to the one he was running for public office in, (I’m assuming he actually showed up and voted for himself – which might be a stretch.) has been defeated at the polls by an educator and pastor that actually lived in Jefferson parish, paid taxes and voted in elections.  Go figure!

http://ballotpedia.org/Rickeem_Jackson

I was planning on writing a piece on how ridiculous elections have become to have allowed someone so eminently unqualified to get elected based on his personal appearance and war chest, but I was pleasantly surprised.  This victory has flipped the Jefferson Parish school board away from the education Reformers.  Jefferson is/was LDOE’s biggest ally in the school reform movement.  They were given little oversight and wide latitude in how they operated.  Now that the Reformer superintendent has announced he is leaving, so this will mean the new superintendent will be selected by this new board.  I expect some hidden stories to be revealed that might explain Jefferson’s oft touted meteoric-rise in scores came to pass.  It appears the public does not buy the narrative education reformers are selling with hundreds of thousands of corporate and out-of-state of dollars.  The newspapers are reporting on thousands spent by Teacher’s Unions, they have been lax in their reporting of out of state billionaire’s and corporate funded Teflon child advocacy groups like the often sanctioned Stand For Children – which is run by former LDOE staffer Rayne Martin  I received this info from one of my regular sources. (Another source confirmed that Rayne Martin did run at least one Stand radio spot for Rickeem. It would not surprise me if this was not properly reported.)

We found some info about our old friend Rayne Martin’s Stand for Children outfit, and its support for Chamber of Commerce favorite Mark Jacobs.

·         Jim Walton and Michael Bloomberg contributed to the Stand for Children LA IEC in late October. Bloomberg gave $125,000 on October 20. Walton gave $25,000 on Oct. 17.

·         The biggest expenditure was $37,000 a few days after those contributions to Buisson Creative for a TV ad for the Jefferson Parish school board. Mark Jacobs, but not Rickeem Jackson.

·         We can track the spending a little further. The commercials ran on:

o   WWL  (~$15,000)<https://stations.fcc.gov/collect/files/74192/Political%20File/2014/Non-Candidate%20Issue%20Ads/Stand%20For%20Children%20La%20IEC/Stand%20for%20Children%20La.%2010-29-11-4-14%20%2814150520861707%29.pdf.>

o   WDSU (~$13,000)<https://stations.fcc.gov/collect/files/71357/Political%20File/2014/Non-Candidate%20Issue%20Ads/Stand%20for%20Children%20Amended%20October%2029-November%204%20%2814146084489527%29.pdf>

o   Not sure where the other $9,000 or so was spent. Probably on another station.

Also of interest, Stand for Children has been dinged recently (ahem, seriously reprimanded by government bodies) for campaign finance violations. Stand for Children, in its zeal to support pro-Common Core candidates, illegally attack Core opponents, failing to properly report its independent expenditures. http://www.dailyjournal.net/view/story/e7f9b1df42d14f10adb8ba67d0d3a633/OK–Education-Ethics-Violation/. I’ve also attached the file (3 files because it loaded one page at a time) with the actual settlement agreement so we have a primary source.. By the way, Stand for Children spent nearly $365,000 in Oklahoma and lost. Earlier this year in Oklahoma, Stand for Children was caught delivering a pro-Common Core petition with fake signatures:<http://www.news9.com/story/25071850/fake-signatures-found-on-okla-common-core-petition>.

Back in Louisiana, Stand for Children modified its WDSU request. At first, it had Stand for Children in the  “Candidate” line. One day later, that was whited out and Stand for Children was put in the “Committee” line with its address added. Also, the amended filing fills in a box that had been left blank: “Spot makes reference to vote for six Jefferson Parish candidates up for election primary on 11/4 because they are for children’s education.” This may have been a correction of an oversight, or it may have been an attempt to tighten up or cover up once they knew they were going to get hit with an ethics violation in another state.

So, Mark Jacobs is being supported by out-of-state billionaires funneled through the independent expenditure committee of Stand for Children, an organization that violated campaign finance laws in Oklahoma.

LDOE has not audited themselves since Katrina

According to a new article written by Jessica Williams at the Times Picayune, the LDOE has not performed an internal risk assessment since 2006, the same year the state was impacted by Katrina.  They could also not show how they were auditing any programs internally.

http://www.nola.com/education/index.ssf/2014/12/louisiana_education_department_3.html

The Louisiana Department of Education did not effectively audit its $5 billion operation in the 2014 fiscal year, state auditors said. According to a legislative audit released Monday (Dec. 1), state education officials had an internal audit plan for that year, but could not show that they had actually audited any programs.

This information jives with what I learned when I was investigating some payroll fraud claims by current and former employees of LDOE. When Paul Pastorek and then John White took over management of LDOE, they did not want anyone snooping in their business (legislatively required or not.)  They allowed the audit department to dwindle to zero supervised employees (as employees left or retired they reallocated those audit positions to their own executive staff) and just a single audit manager responsible for auditing the disbursement of over 5 billion dollars annually as well as the oversight of all day to day operations of the 500-700 or so LDOE employees and their paper timesheets.

I tipped Daryl off to LDOE’s inability to perform even the most basic of audits, the lack of personnel, and the rampant abuse by John White’s executive staff of “working” from home (as reported by numerous time keeping staff), over the summer.  We discussed the issues with proving payroll fraud with a complicit boss covering for them, as well as the possibly of performance audits.

I learned from internal staff that the auditors from Darryl’s team had stumbled across rampant disregard for rules and abuse and were aggressively auditing LDOE –  for the first time in at least a decade.  LDOE is responsible for one of the largest chunks of our 25 billion dollar annual budget (if not the largest) and for years had been escaping any oversight whatsoever.  Most of their metrics for evaluating themselves are flawed, their conclusions are biased and often quite absurd non sequiturs –  as pointed our recently by a professor from Harvard.

Are you happy with the current BESE rubberstampers that approves expenditures like this?

The report also highlights the misdeeds of an embattled eastern New Orleans non-profit director, as previously outlined in a June audit. Former Open World Family Services Inc. executive director Kim Cassell spent $307,500 in public education funding on travel, pet-grooming supplies and other personal expenses, state Legislative Auditor Daryl Purpera noted.

The state Education Department distributed nearly $1.6 million in federal grants that Cassell’s organization used to operate. She now faces three felony criminal charges.

The education department’s failure to effectively audit increases “the risk that errors and/or fraud could occur and remain undetected,” Purpera added. The department managed $5.1 billion in public revenue in 2014.

The audit reveals that 5.1 billion dollars of your tax dollars may be going anywhere as far as John White and 8 of the 11 BESE members may care.  Daryl could not audit all of the programs, he was only able to audit the auditors and their audits (or lack thereof).  Most fraud and abuse should be detected by the department long before Purpera’s team needs to get involved.

That means there are many more dog grooming expenditures out there yet to be discovered, and which probably never will be.

Special thanks to BESE member Jane Smith for putting me in contact with Darryl and fighting so valiantly for the cause she has recently joined.  I am also very thankful for BESE member Lottie Bebee for being a fierce and tireless defender of our state and children and BESE member Carolyn Hill for fighting alongside them.

Are Jefferson Parish Judges Corrupt?

I thought I would provide an update to my Jefferson Parish School Board Race post. I received a lot of feedback from the public about how outraged they were as well as how fed up they were with the corruption in the parish. I honestly had no idea although I had my suspicions based on how Jefferson became the poster parish for education reform.

Education reform flourishes in corrupt places because of all the opportunities for money to change hands.

I have been working on a number of stories simultaneously, that are requiring a lot of research and interviewing. Some of the groups I am investigating have a record of suing or threatening to sue whistleblowers, so I’m trying to piece together public records and witness statements that back up my claims.  Incidentally I believe that intro segues into my next point and a comment from a reader from Jefferson Parish and what they had to say about my story on Rickeem:

This makes so much sense to me with recent experience I had with the group that runs the campaign for these guys who are clearly not qualified.

Byron Lee is the puppet master here and has his hands in everything and controls Mark Spears, (controlled) Derik Shepard and some judges.

Just research Byrons construction company and see how the money trickles through everyone’s hands due to awarded parish contracts. Byron lee and judges I won’t mention have control of all the churches and with money not accounted for and sheep following baptist church members, push these puppets into office.

Just check Adrian Adams'(newly elected judge) campaign report. He is affiliated with this secret group (Byron, Mark, Derrick Zeno) and to this day has not filed a campaign report that shows where he got over $25,0000 (in 15 days) to cover radio, tv, two billboards, several mail outs. It took him 3 months to raise that much and used at least that much more in the final two weeks of election.

So sad Jackson will win. This group will not be stopped. They almost control the black community as a whole because you have to listen to pastor and vote for who they say. Very sad and frustrating.

Will this reader’s comment be prophetic? I hear Rayne Martin and Stand for Charters have gotten on the Rickeem Bandwagon from this reader:

While traveling yesterday, I heard a radio campaign add for Mr. Jackson on WWL. Guess who paid for it? Stand on Children led by Rayne Martin. So predictable.

Below is a link to the judge that made the ruling in the Rickeem Jackson case, Judge Lee Faulkner Jr. He is also probably connected to Derrick Zeno since Faulkner replaced a “Melvin Zeno”, according to his bio. That seems like a unique name but I was not able to find much online. (Can anyone clue me in on the Zenos?)

http://louisianajudgesnoir.org/lee-v-faulkner-jr

 

Judge Lee V. Faulkner, Jr.

Faulkner

The Honorable Lee V. Faulkner, Jr., was born in New Orleans, Louisiana and is a life long resident of Jefferson Parish. He graduated from Xavier University of Louisiana and the Southern University Law Center in Baton Rouge, Louisiana.

Following law school, Judge Faulkner entered into private practice, handling personal injury, criminal, civil and domestic cases.  He also served as a public defender for the Parish of Jefferson.

Prior to being elected as Judge, he was appointed by the Louisiana Supreme Court to serve as Judge Ad Hoc for Second Parish Court for the Parish of Jefferson.  He also served as a Traffic Hearing Officer for Second Parish Court.

He was elected as a District Judge in 2008, following the retirement of the Honorable Melvin C. Zeno.

He is a member of numerous social, judicial and legal organizations, including but not limited to the National Bar Association, American Judges Association, Past President of the Jefferson Bar Association, Louisiana State Bar Association and the Omega Psi Phi Fraternity, Incorporated.

I wonder if any reporters from Jefferson Parish will look into these tips?

I also wonder how Judge Faulkner determined his own “life-long” residency in Jefferson Parish? In Rickeem’s case he appears to have decided “intent” to live there is enough to meet the 2 year residency requirement for school board candidates.

For some reason this situation and these reports brought to mind this bible passage. Matthew 7:15 and 7:16

15”Beware of the false prophets, who come to you in sheep’s clothing, but inwardly are ravenous wolves. 16”You will know them by their fruits. Grapes are not gathered from thorn bushes nor figs from thistles, are they?

Matthew is definitely my favorite apostle.

This seems an appropriate passage for Stand for Children and Education Reformers and the cess-pool of corruption I’m told Jefferson Parish has become. What fruits do you see them bearing?

LDOE’s statistical methods and reports are beyond useless according to Harvard professor

(Provided by Mary K. Bellisario)

 

Question: Have press releases issued by the LA Dept. of Education been accurate,especially regarding specific analysis ofresults of student AP tests, possibly ACT tests,  PARCC preparation, and the VAM– Value-added Method for teacher evaluations? 

 

Further:  Should legislators base their decisions, and vote their support, oninformation contained in state press releases about public education whichcould be inaccurate?

 

An independent, nationally-recognized data analyst, Dr. A. J. Guarino, is doubtful that data from the LA Dept. of Education is being analyzed or presented accurately.  He gives his reasons below. He calls into serious question analysisprocedures and “selective” reporting in three specific areas:  AP test results (and possibly ACT results), PARCC preparation, and the VAM method of teacher evaluation.

 

Attached above and below is an unsolicited critical analysis of information that has recently come from the LA Dept. of Education, describing Louisiana’s purported progress in public education.  The analyst, Dr. A. J. Guarino,is a professor of biostatistics at the Harvard University teaching hospital, Massachusetts General Hospital of Health Professions. He and his graduate students routinely collect and analyze media reports from around the U.S. for their accuracy.   Dr. Guarino has ties to Louisiana, where he formerly served as president of the Louisiana Education Research Association (LERA). 

 

If you have further questions, or want to verify his credentials, Dr. Guarino can be reached at:  ajguarino@mgh.harvard.edu

 

Among Dr. Guarino’s conclusions are: 

“When looked at collectively, there is a troubling trend of either the misrepresentation of statistical outcomes or the incorrect application of statistical principles.”

 

Further, Dr. Guarino concludes: 

“Based upon my professional experience, I believe the Department of Education and Superintendent White have created both of these scenarios.  My intent is to bring attention to these problems so that legislators and policy makers can make informed decisions moving forward.”

__________________________________ 

Teaching, learning, and assessment: The closed circle

A. J. Guarino*

I certify that there is no conflict of interest with any financial organizationregarding the content of this letter.

The intent of this posting is to report on findings and policies that are based uponincomplete information or misapplication of statistical principles.

        I am a professor of biostatistics at the Massachusetts General Hospital Institute of Health Professions in Boston (teaching hospital for Harvard) where I teach basic and advanced quantitative methods to graduate students in the health sciences. To document the need to be “statistically literate,” I present reports from media sources purporting major findings while failing to provide essential information, which often contradicts their “major findings.” As the former president of the Louisiana Education Research Association (LERA), I continue to follow the educational developments in Louisiana.  Over the past weeks and months, I noticed “press releases” and other media containing only selective information, which made the reported findings suspicious. After securing the full information, my analyses did not replicate those of the press-releases.

        Decisions must never be derived from findings that fail to provide complete information. At this stage of my career, my community-service is to report possible data misrepresentations that appear in the media. I would never seek nor accept compensation to provide statistical consultation.   

1.      Reporting of AP Test Scores

A recent press-release by Superintendent White noted that AP passing rates increased 24.6%, the highest in the nation, from 5,144 in 2013 to 6,407 in 2014. What is conspicuously missing is the ratio between successful tests and unsuccessful tests for each year.  The first number (5,144) represents a reported 5.3% passing rate while the second number (6,407) represents a 4.1% passing; in other words, the AP passing rate actually Decreased 22.64% from 2013 to 2014.

(5.3 – 4.1) /5.3 = 1.2/5.3 = .2264 (22.64%)

The failure to provide the essential information grossly distorts the interpretation. In order to review the actual passing rates for the latest round of tests, I had to obtain that information from a newspaper article the next day.  Here is the summary:  Of the 55,000 or so additional tests in 2014, only 1,263 (approximately 2.2%) passed. These data were never mentioned in the press release. 

Calculating the percent increase from the raw data without taking into consideration the contexts of those numbers is complicity. This is like saying more people finished the Boston Marathon this year than last year only because more people entered the race.  Given the ever increasing monies expended by the state to provide for students to take AP exams, it would be appropriate for those holding the data to inform those who need the data to make informed decisions that, last year, approximately 156,000 total AP tests were given and around 6,407 passed (i.e., 4.1%).

6407/156000 = .0410 (4.1%)

One final note on this matter: It appears that the ACT results have been reported in a similar fashion to the AP scores.  If that is the case, and the increases in the number of tests that were scored at 18 or higher are due to increased numbers of students taking the tests then those results are equally distorted.

2.      Preparation for PARCC Exams

A recent press release from the Superintendent of the State Department of Education states: Since 2010, teachers and state assessment staff in more than a dozen states and the District of Columbia have developed questions, accommodations, and policies that make for an improved blueprint for standardized testing.

At first glance, the statement appears to be quite informative. However, my graduate students were quick to notice the missing essential information, i.e.,Curriculum. It is significant to note that a “Test” is a type of assessment that evaluates academic achievement. Downing & Yudkowsky (2009) explain, “…assessment and instruction are intimately related. Teaching, learning and assessment form a closed circle, with each entity tightly bound to the other” (p. 9) .

Superintendent White broke the “closed circle” by suggesting that neither Eureka Math nor any other curriculum is favored for state testing.  In a note to a high level government official who was expressing concerns about Eureka Math, Superintendent White replied:  “We will also continue to insist, as you urge, that state tests be aligned to high expectations for our students’ skills and not to any particular curriculum (emphasis added).” 

In other words, Superintendent White is suggesting that Louisiana should give high-stakes exams to studentsirrespective of the fact that students, teachers, and stakeholders have not been given a clear curriculum on the first day of school.  How do you test what you are supposed to know at the end of the year if you haven’t communicated those expectations at the beginning of the year?  Lack of clarity for curriculum guarantees that profit-driven entities will provide their own curriculum, all of which purportedly provide the best means of preparing students to meet expectations laid out by high-stakes exams.  This guarantees a process that is costly, inefficient, and ill-defined.

The DOE discreetly recognizes this unfortunate reality by designating certain curriculum derived by outside sources as “Tier 1.”   However, no outside entity can provide to parents and teachers on the first day of school what students should know and how they will be expected to demonstrate that knowledge; this responsibility falls squarely to the DOEand to this point in time it is a responsibility that has not been met. TheDOE tacitly acknowledges its failure to generate curriculum when it continues to insist that school systems are free to use whatever curriculum they choose. 

The lack of a state generated curriculum that is available on the first day of school short circuits end-of-course exams..Results are tainted since there has been no defined path presented to articulate the criteria, much less communicate how the criteria will be met. 

3.  The Evaluation of Teachers via a Value-Added Model (VAM)

The Louisiana Department of Education has implemented the Value Added Measures (VAM) as part of a teacher’s performance evaluation. However, implementation of any educational interventions must be supported by the best available research results (evidence) or more formally known as Evidence-Based Practice (EBP). For example, the Individuals with Disabilities Education Act (IDEA) and Elementary and Secondary Education Act (ESEA) require that schools use programs, curricula, and practices based on “scientifically-based research” “to the extent practicable.” The best available research from the American Educational Research Association (AERA)  reports the following, “…weak to nonexistent relationships between state-administered value-added model (VAM) measures of teacher performance and the content or quality of teachers’ instruction.” Implications of these resultsfail to support the utility of VAM data for teacher evaluations. Concurring with the AERA findings is the American Statistical Association , the largest organization in America that represents statisticians and related fields.  The Louisiana Department of Education’s implementation of theValue Added Measures (VAM) as part of a teacher’s performance evaluation is in conflict with Evidence-Based Practice (EBP).

Conclusion

When looked at collectively, there is a troubling trend of either the misrepresentation of statistical outcomes or the incorrect application of statistical principles.  If the results aremisrepresented, you do not have all the necessary data to make an informed decision (#1 above).  If the statistical principles are incorrectly applied, you have created a fundamentally flawed process that provides no meaningful interpretations (#2 and 3 above). 

Based upon my professional experience, I believe the Department of Education and Superintendent White have created both of these scenarios. 

My intent is to bring attention to these problems so that legislators and policy makers can make informed decisions moving forward.

________________________________________________ 

Bio:  Dr. A.J. Guarino

*A. J. Guarino presently teaches biostatistics at the Massachusetts General Hospital Institute of Health Professions in Boston. In 2011, Dr. Guarino was awarded the 2011 Nancy T. Watts Award for Excellence in Teaching – the highest prize given to a faculty member at Boston’s health sciences graduate school.  He received his bachelor’s degree from UC Berkeley and his doctorate in Educational Psychology with an emphasis in statistics and psychometrics. He has published over 50 refereed research articles in a variety of fields in health, education, psychology, assessment, and statistics and presented nearly one-hundred papers at national and regional conferences. Dr. Guarino has also coauthored five graduate level statistics textbooks.  A partial list of recent scholarship is provided below.  Dr. Guarino has also served as president of the Louisiana Education Research Association (LERA). 

He can be reached at: ajguarino@mgh.harvard.edu 

Recent Scholarship

Textbooks:

Meyers, L., Gamst, G. & Guarino, A. J. (2013). Performing Data Analysis: Using IBM SPSS. Hoboken, NJ: Wiley.

Meyers, L., Gamst, G. & Guarino, A. J. (2013). Applied Multivariate Research Design and Interpretation Second Ed. Newbury Park, CA: Sage.

Meyers, L., Gamst, G. & Guarino, A. J. (2006). Applied Multivariate Research Design and Interpretation. Newbury Park, CA: Sage.

Gamst, G., Meyers, L., & Guarino, A. J. (2008). Analysis of Variance Designs. New York: Cambridge Press.

Meyers, L., Gamst, G. & Guarino, A. J. (2009). Data Analysis Using SAS Enterprise Guide. Hoboken, NJ: Wiley Publication.

Refereed Articles and Presentations

Matthews, L. T., Ghebremichael, M., Giddy, J., Hampton, J., Guarino, A. J., Ewusi, A., Carver, E., Axten, K., Geary, M., Gandhi, R. T., Bangsberg, D. R. (2011). A Risk Factor-Based Approach to Reducing the Likelihood of Lactic Acidosis and Hyperlactatemia in Patients on Antiretroviral Therapy. PLoS ONE 6(4), 1-7.

Nahas, S. J., Young, W.B., Terry, R., Kim, A., Van, D. T., Guarino, A.J., & Silberstein SD. (2010). Right-to-left shunt is common in chronic migraine. Cephalalgia 30(5), 535-42.

Chesser, S., Forbes, S. A., & Guarino, A. J. (November, 2011).  Investigation of intra-individual response of the stress hormone cortisol to varying educational environments (single vs. mixed sex groupings). Poster presentation at the annual meeting of the Society for Neuroscience, Washington DC.

Lopez, R. P., & Guarino, A. J. (2013). Psychometric Evaluation of the Surrogate Decision Making Self-Efficacy Scale. Research in Gerontological Nursing 6(1), 71-76.DOI: 10.3928/19404921-20121203-02

Certain, L., Guarino, A. J., & Greenwald, J. (2011). Effective Multilevel Teaching Techniques on Attending Rounds: a Pilot Survey and Systematic Review of the Literature. Medical Teacher, 33, 644-650.

Hastie, P., & Guarino, A.J. (2013). The Development of Skill and Knowledge during a Season of Track and Field Athletics. Research Quarterly for Exercise and Sports.

Lopez, R.P., & Guarino, A.J. (May, 2011). Confirmatory factor analysis of the Surrogate Decision Making Self-Efficacy Scale (SDM-SES). Poster presented at the annual meeting of the American Psychological Society, Washington DC.

Eaves, R.C. & Guarino, A. J. (2006). Dunn’s multiple comparison test. Encyclopedia of Measurement and Statistics, 293-296.