Saturday, October 18, 2014

Merit’s Liquidity

Over the past year, multiple major media outlets and a powerful university president or two have been agitating about the correlation between parents’ income and SAT scores. Over that same time period, editors of various psychology journals were rejecting my regression study to determine the true influence of parents’ income on SAT scores while controlling for other possible factors because the study did not “make clear what the gap or problem is in the literature and exactly how the present study fills that gap.” If I cannot convince a journal that a lack of an examination on the influence of socioeconomic status in the entire 87-year history of America’s most important academic test is a gap in the literature, then what hope remains for my other project on the interplay of body-mass index and perspiration on navel lint accumulation?

Part of the problem resulted from my need to rely heavily on years of state averages, rather than the scores of individual students, but I am convinced that anyone who manages to gain access to individual data will merely corroborate my work at much greater expense. One editor conceded, “I agree that there is something we can learn from these data,” despite rejecting my work anyway. Ultimately, the new open-access journal, Open Differential Psychology published the study.

Last week, the College Board released the latest year’s SAT results. The organization cleverly released its “benchmarks” report the day before their usual, more detailed report. Reporters were too lazy to report on the subject two days in a row, let alone do any analysis. Blasé headlines declared scores “flat,” as they would for any two-consecutive-year analysis. I found the numbers fascinating. They dovetail with my study and other previous writings and flesh out some trends the media missed.

In my study I addressed the overlooked fact that correlations between parents’ education and (individual) SAT score and between parents’ income and SAT score have not been constant over time.
Family educational advantage seems to evince virtually undeviating growth as a predictor of SAT scores, but financial advantage seems to grow as the economy worsens. Rather than postulate that times of economic difficulty almost immediately make wealthy people smarter, one should focus on the exclusivity of the income category.


2014 brought new record advantages to students whose parents have bachelor’s or graduate degrees on the critical-reading and mathematics subtests. This fact can inform discussion about the notion of an “education bubble.” The expansion of college access has raised concerns over indebtedness, falling standards, and underemployment, but, while these concerns are likely to grow, education appears increasingly to benefit the following generation as if to demonstrate that the ability of higher education to sift and inculcate basic skills remains intact, or at least it did one generation ago.

The difference between the yearly trends for the SAT-score benefit of education to a family compared to the benefit of coming from a home with a higher income startled me, especially given the sharp drop in income advantage coinciding with the 2008 economic downturn. So, I graphed income’s SAT advantage with US gross domestic product (GDP) growth rate. The graphs almost perfectly coincide when the income divide compares families above $100,000 to those below that.


My regression study included a control for year, but an even more detailed study of the influence of family income could benefit by controlling for the state of the economy, as I suggested in my concluding remarks.
Perhaps additional or alternative variables could be identified. For instance, year was significant in some regression iterations but had small β values. Perhaps year is a proxy variable for other factors like the state of the economy.

The graph provides convincing evidence that there is some kind of causal relationship between the economy and SAT scores despite the relatively low influence of parents’ income on scores in the regression analysis. Student stress or mood seems more plausible as a mechanism than radical yearly changes in educational quality, but my point about exclusivity concerned whether economic slowdown weeded out low test performers from higher economic strata due to the heritable component of cognitive ability shared between the parents and the student. However, for that to be the mechanism, one would expect GDP to look more like a leading indicator of test performance or for the slope of the growth rate to tend to be the negative of the slope of test scores over time. This graph actually seems to indicate that poor economies weed out high test performers from higher economic strata. In The Bell Curve, Richard Herrnstein and Charles Murray claimed that the twenty-first century would continue “the emergence of a cognitive elite.”
The isolation of the brightest from the rest of society is already extreme; the forces driving it are growing stronger rather than weaker.
Of course, the economy of the 1990’s, when they wrote that, usually appeared to be growing stronger rather than weaker.

I can imagine two mechanisms for economic contraction weeding out smart families from the upper class: circumstances cause the power elite to lose interest in the power of ideas, and some upstarts’ ideas were not that good from the start. I suspect both are at work. The first mechanism supports a leftist or Marxist vision of ensconced aristocracy holding the privileges of power and leisure. If recessions interspersed with periods of anemic growth are “the new normal,” then The Bell Curve was partly wrong, and economic populism should be of interest to the so-called cognitive elite. On the other hand, when bad ideas create bad economies, perhaps the cognitive elite receive their just desserts. Some people who are obsessed with IQ research engage in some paradoxically simplistic thinking. As anyone who has surveyed the differential psychology literature about liberals versus conservatives or even the results for the SAT Student Descriptive Questionnaire can attest, the best indicator of dimness is indifference. Wrong ideas also sometimes require intelligent formulation. Spectacularly wrong ideas receive their negative appraisal in retrospect because they were compelling enough to do damage. Moreover, misbegotten hype might serve as a more commonly available vehicle of upward mobility for smart upstarts than truly transformative ideas.

Wisdom and top-down analytic ability sadly receive fewer lauds than the clichés “street smarts” and “common sense.” To illustrate the distinctiveness of wisdom from intelligence so as to support the notion that smart people might promote unwise ideas, I would like to analogize with my personal experience working with engineers and doctors. Many of the engineers whom I have known work hard on mundane projects but like to engage in fascinating discussions about politics, metaphysics, and, of course, the possibility of alien life forms. They contrast with neurologists and neurosurgeons whom I have met who regularly work with cases of alexia without agraphia and amygdalectomy but remain willfully indifferent to any profound questions these phenomena raise about the soul or free will. If emphasis on analytic ability (as opposed to memory) is analogous to “wisdom,” then the engineers are the aristocrats in this analogy, although engineers often zealously advocate for their upstart ideas. I think most people assume that doctors are smarter than engineers because becoming a doctor is harder and better reimbursed, but I would say engineers are deeper conversationalists, which manifests a certain type of wisdom to me.


Merit’s Mobility

Though the 90’s are long gone, Bill Clinton might still point out that this is not “midnight in America.” The American economy usually does expand, and smart people disproportionately tend to be the proverbial movers and shakers. Hints of this appear as evidence for their geographical mobility based on yearly maps of SAT and ACT data. Racial demographics also relate to test scores and have their own trends of time and place.


Last year, I graphed all data from state SAT reports for SAT scores of different family income brackets according to the proportion of each state’s population that belongs to the two highest scoring racial groups, whites and Asians. As the updated version above shows, students from the wealthiest category have the lowest association between state racial demographics and SAT score. One reader considered this “evidence against a genetic explanation.” Of course, homes earning less than $80,000 per year are not all dealing with “typical environmental problems of poverty.” Part of my study’s regression analysis looked at the effects on SAT scores that were common to both income and race. These effects existed when the income cutoff was $100,000 but not $20,000. Without completely dismissing the environmental hypothesis, I pointed out alternative factors that could be at work. An alternative explanation for the graph could be that higher income families with good test performance are more likely to live in racially diverse states compared to other income levels, since the racial proportions pertain to the states, not the income brackets, themselves. It turns out that SAT data does offer some support for my previous contention that “racially diverse states like California have industries or attractions that pull in successful, educated whites.”

Before revealing the map that demonstrates this, I shall review the latest SAT and ACT data by race. Composite ACT-SAT scores by race are very similar to the year before.


Native-American scores remain just slightly above those of Hispanic Americans after a significant drop in Native-American ACT scores from 2010 to 2013.


Overall, Asian scores have continued their amazing progress. The Asian SAT mathematics subtest advantage over whites rose to 64 points.


Asians are not closer to surpassing whites on the critical-reading subtest only because white scores rose by the same amount this year.


In contrast to all other groups, Asian SAT writing subtest scores rose in seven of the past nine years.


However, Asian progress on the ACT did appear to stop. Their scores on the reading and math subtests even slightly dropped. The average score drop for Pacific Islanders, for whom scores are available only in recent ACT data, was quite severe this year: almost a full composite ACT point. Fortunately for Pacific Islanders, this probably only resulted from a massive, 19-percent increase in ACT participation. Hawaiian students, who were 83 percent Asian-American, according to SAT numbers, bumped up their ACT participation from 40 percent to 90 percent, while their SAT participation rate barely budged at 63 percent. Asian students have been the one racial group who heavily favor taking the SAT rather than the ACT. This is likely due in large measure to foreign students. As the following graph shows, ACT participation surpassed that of the SAT, driven mostly by white students.


It would be tempting to surmise that foreign students are causing the continued progress of Asians on the SAT, but, as I previously discussed, the scores of foreign students (represented on the following graph as colored lines of a negative advantage for American students) still closely resemble Asian SAT scores (represented as gray lines for a negative white advantage), while the number of foreign students could not overwhelm the number of Asian students.


Indeed, Asian progress has been so impressive that it calls into question some assumptions of experts in differential psychology and adherents to the philosophy behind so-called “human biodiversity.” Rather than reveal a unitary Asian-white general intelligence gap, Asians have always had a large mathematics advantage. Rather than maintain a constant mathematics gap as Asians improve their English skills, Asian mathematics, reading, and writing skills improved in tandem. Rather than have SAT scores that coincide with research on the secular IQ gains, known as the Flynn effect, Asian SAT gains have been large at the same time as their IQ gains have been small, and white SAT gains have been small, as large IQ gains in Western societies have continued unabated. Frey and Detterman famously called the SAT an IQ test in 2004, but they offered no explanation for why their IQ-estimation equation essentially eliminated the reading subtest. Murray recently defended The Bell Curve by pointing out how little the black-white test score gap has changed. So, what is the precise meaning of that, given that the Asian-white gap has changed so greatly?

What should be clear from the preceding review of new data is that participation rates greatly matter. Hawaii’s ACT scores fell this year by almost two points on a scale from 11 to 36, following its participation increase. In order to appropriately map composite ACT-SAT scores, I must follow my previously described methodology for adjusting scores according to state SAT and ACT participation rates.

For comparison, here is the map of the percentage of white and Asian SAT-taking students over the years:

 photo sat aw map.gif


















Student diversity increased especially among coastal states. I previously claimed, “Demographic changes correspond to falling test scores, and one can see it, at least in terms of a North-South divide, on these maps.” The most recent years of this participation-controlled composite SAT-ACT score map make me want to amend that assessment.

 photo sat-act part cont.gif


















One can more easily notice the change by just looking at the oldest and newest maps without animation.


The earliest year does suggest a North-South divide, but the coastal states of California, Georgia, South Carolina, and North Carolina improved, while some Mountain states declined. The trend could be a fluke, and a few states buck the trend, but it fits with my previous explanation of diverse states attracting relatively well scoring students from wealthy families. The upper class wishes to live near beaches and in high-status states with impressive cultural and educational institutions. Many of the cognitive elite actually might like some kinds of racial diversity. Southern states like Louisiana have not improved, or, in the case of Mississippi, scores improved but were already extremely low. Perhaps the cognitive elite would be attracted to this Southern coast but find Southern culture too alienating, and maybe such a feeling of alienation from otherwise attractive settings makes liberal condescension slightly more understandable.

In an era that made “big data” a catchphrase, the colossal data pool that describes the colossal sample who took these tests inspires elite news outlets to make bar graphs of simple correlations and reports of flat scores. I call that flat reporting.



ResearchBlogging.org






nooffensebut (2014). Parents’ Income is a Poor Predictor of SAT Score Open Differential Psychology, 1-19

10 comments:

  1. Thanks, most interesting.

    I have my doubts about the income stats for the SAT. Are they collected from students taking the test or from a special study of parents?

    Anyway, assuming they are accurate, I think the nosedive in 2004-2008 was related to the Bush Bubble pushing a fair number of lower IQ families above $100k income due to easy money as mortgage brokers, contractors, rims installers, and the like. Then the house-flipping bubble exploded.

    ReplyDelete
  2. Income data comes from the optional Student Descriptive Questionnaire by mail. You can see a copy of one from the 80s at the end of this. In 1996, only 17 percent of students did not answer it. That number peaked at 47 percent in 2003, fell back into the 30s, and peaked again at 55 percent in 2013. Maybe higher non-response rates indicate greater accuracy among those who respond. A validity study from 2009 (here) determined that student-reported grades were 74 percent accurate. I would expect a mail-in questionnaire for parents' income to be more accurate than grades because the student can just ask his or her parents. The fact that a very large sample of self-reported income matches so well with GDP growth seems to me to indicate real-world validation.

    I agree with your analysis of the economy but not its relationship to SAT scores because the 2008 recession lowered, not increased, the Cohen's d SAT gap between high-income families and those below $100K. The bad economy "made" the wealthy less smart. The actual number of higher-than-$100K students stayed pretty high throughout that recession. There was a dip from 31K in 2009 to 23K in 2012, but that is still higher than the period 1996-2006. Nevada's dip between 2009 and 2013 was 39 percent. Maybe the recession was felt more by ACT students.

    ReplyDelete
  3. I would say the harshest example is looking at the average SAT score by race and family income where blacks from families that earn over $200,000 a year score 3 points higher than whites from families that earn less than $20,000 a year from the college board.
    http://isteve.blogspot.com/2014/03/2008-sat-scores-by-race-by-income.html

    ReplyDelete
  4. nooffensebut,

    I've found your blog interesting over the years. You may be interested in some related analysis I've posted to my blog by combining 4-years worth of California's standardized test score data (CST/STAR tests), US Census (median family income, parent education levels, poverty status, etc by race/ethnicity), and other data sources.

    On concentrated poverty

    Follow up post

    Boxplots by ethnic group

    -FDM

    ReplyDelete
  5. Incidentally, I agree with you that education levels are a better predictor than income (I've found the same in my own data at this level of analysis at least). I used median family income for much of my analysis because the income data are a bit more complete and because the education "effects" are more variable by race than the income effects are.

    I plan on doing a follow up post later where I'll summarize some of my prior work, reduce some extraneous information, and add some new stuff in for education as a predictor and what not.

    -FDM

    ReplyDelete
  6. Yes, I read through this and think it nicely dovetails with my SAT analysis. Keep up the good work.

    ReplyDelete
  7. After I published my study questioning the lumping of income with education in SES, Rindermann and Baumeister seemed to copy my idea without citing my study. :/

    ReplyDelete
  8. Hey buddeh
    write something new for us willya

    - guaranteed 100% non-robot individual

    ReplyDelete
  9. Fascinating stuff. Thanks for writing. Keep it up. One of these days, when we get over this whole not being able to ask certain questions about our species without losing jobs because NAZIS thing so everyone can study data scientifically even if he/she doesn't have balls of steel and a desire to be hated, we are going to have a renaissance of scientific understanding about humanity. Right now I'm learning a lot from those who were brave enough to do this work in the face of hate and financial loss, because their desire to seek truth was more important than careers, money, societal approval, etc. But imagine how much more we could learn if we didn't limit the research to only the brave and the few who don't mind being hated?

    Anyways, I want to say thank you to you, and everyone else who is a scientist seeking truth about our species even if the truth hurts people's feelings. Feelings don't determine scientific truth, sorry!

    ReplyDelete
  10. I am told Razib Khan just called me monomaniacal. I haven't read him or interacted with him in years, which shows his irascibility. Monomaniacal means one lost a debate because the other had enough curiosity to gain expertise. Specialization is not a character flaw. Until I corrected it, the myth that 77% of Chinese have MAOA-L, based on a copy error in the New Zealand Medical Journal, trashed MAOA research.

    "Peer review" error:
    http://www.nzma.org.nz/__data/assets/pdf_file/0009/17838/Vol-120-No-1250-02-March-2007.pdf

    John Horgan repeated the error to criticize MAOA research to the public.
    http://blogs.scientificamerican.com/cross-check/code-rage-the-warrior-gene-makes-me-mad-whether-i-have-it-or-not/

    The source of my conflict with Khan was when I called Horgan on this after Khan interviewed him.

    Horgan subsequently called for banning research on race and intelligence.
    http://blogs.scientificamerican.com/cross-check/should-research-on-race-and-iq-be-banned/

    The New York Times hired, then fired Razib Khan due to his controversial statements on race and IQ.
    https://www.washingtonpost.com/blogs/erik-wemple/wp/2015/03/20/new-york-times-signs-contract-writer-razib-khan-then-dumps-him/

    ReplyDelete