tag:blogger.com,1999:blog-5002675950760488813.post2629453999129833644..comments2024-01-21T14:11:10.779-08:00Comments on The Unsilenced Science: The SAT Zombie Apocalypsenooffensebuthttp://www.blogger.com/profile/02461190919466049463noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-5002675950760488813.post-42039118913669160472012-10-24T09:35:14.340-07:002012-10-24T09:35:14.340-07:00Hello, I simply wanted to take time to make a comm...Hello, I simply wanted to take time to make a comment and say I have really enjoyed reading your site.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-31147711732014259632012-10-06T23:40:05.503-07:002012-10-06T23:40:05.503-07:00Here are the numbers for each score. Race data in...Here are the numbers for each score. Race data in this detail are not available. Based on the relative score declines, I would say that the largest number of “diamonds in the rough” are probably white, but Hispanics would probably not be nearly as lacking as blacks.<br />Score 2011 2012<br />36 5 9<br />35 49 69<br />34 112 136<br />33 168 219<br />32 266 302<br />31 414 430<br />30 578 601<br />29 656 743<br />28 902 990<br />27 1089 1260<br />26 1299 1505<br />25 1560 1623<br />24 1799 1860<br />23 1837 1984<br />22 1950 2203<br />21 2064 2329<br />20 1987 2411<br />19 1885 2315<br />18 1802 2259<br />17 1402 2044<br />16 1111 1888<br />15 849 1735<br />14 661 1437<br />13 412 1208<br />12 206 802<br />11 80 357<br />10 13 79<br />9 4 28<br />8 0 3<br />7 0 4<br />6 0 1<br />5 1 0<br />4 0 0<br />3 0 0<br />2 0 1<br />1 0 0nooffensebuthttps://www.blogger.com/profile/02461190919466049463noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-42894222805026537812012-10-06T17:04:57.246-07:002012-10-06T17:04:57.246-07:00Thanks, for Utah, how large were the increases in ...Thanks, for Utah, how large were the increases in higher scorers in absolute terms (or relative to the entire cohort?) In other words, can we see enough diamonds in the rough being discovered to actually count them, or is just washed away by raw noise?Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-73923647424304895022012-10-06T00:46:00.149-07:002012-10-06T00:46:00.149-07:00Since state reports do not go back that far, I can...Since state reports do not go back that far, I cannot answer that for Illinois in 2002, but I can for Utah in 2012. Overall participation jumped from 73% to 97%. Both those scoring above 30 and above 25 increased in raw numbers but fell as a percentage (from 4% to 3.5% and from 22% to 19%, respectively). The data shows a bell curve that slightly shifted. Each of the two years has a single peak. The percentage increase of white test-takers was 9%. For Asians, it was a drop of 4%. Hispanics increased 28%. Blacks increased 14%. The reports show the percentage of each racial group meeting the “college readiness benchmark scores,” which were 18 for English, 22 for math, 21 for reading, and 24 for science. Asians held steady at 27%, whites fell from 29% to 27%, Hispanics fell from 11% to 8%, and blacks fell from 9% to 4%. I interpret this to probably mean that Asians already had high participation, but the adoption of the state standard forced less prepared students from the other groups to take the test. Hispanics increased in number more than blacks, but their scores did not fall as much because more Hispanics are immigrating and because more Hispanics perform better than do blacks among those who do not go to college. It makes sense to expect that recent Hispanic immigrants underutilize education, relative to blacks.nooffensebuthttps://www.blogger.com/profile/02461190919466049463noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-75271911757682578642012-10-05T21:04:30.257-07:002012-10-05T21:04:30.257-07:00It would be interesting to see if this law made an...<i>It would be interesting to see if this law made any measurable increase in high scorers (above 30 or above 25) and which race they most came from. I'm betting offhand that the law rousted more semi-smart white kids into taking the ACT than other races. Probably kids who were set on going into the Army and had done well on the AFQT, that kind of thing.</i><br /><br />Above 25, white boys, maybe some Hispanic boys. Above 30 a few white boys, mostly in reading and writing.<br /><br /><i>I'd be interested in seeing how the absolute number of people scoring 700 or higher (or its ACT equivalent) have changed over the years because surely virtually everybody who could score a 700 was taking one of the two tests. But, there has probably been a big increase in recent years of individuals scoring highly on both tests, which leads to doublecounting.<br /></i><br /><br />You wouldn't want to use the SAT for this unless you started after 2005. The ACT would be best.It was the first test used statewide, and has made no changes to format.<br /><br />Odd note: the ACT's English score grading has gotten tougher over the past two years---probably because the increase in top tier testers.Mitchnoreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-17459838436160698592012-10-04T21:34:17.106-07:002012-10-04T21:34:17.106-07:00That's fascinating stuff.
So from this we ca...That's fascinating stuff. <br /><br />So from this we can calculate the average score of the incremental test takers. For example, in Illinois, when 71% took the SAT, the average was 21.6, but when 99% took it, the average fell to 20.1. So, all else being equal, that means the incremental 28% averaged a 17, which is 4.6 points lower than what the 71% averaged, and 4.6 is pretty close to one standard deviation. <br /><br />If the additional 28% came from the bottom of the distribution, the 2nd to 28th percentiles, then the median of the incremental test takers would be about the 14th or 15th percentiles overall, while the median of the previous 71% would be about the 64th percentile. That would imply a drop of around 6 to 6.5 points, maybe, but instead the drop was about 3/4ths of that. So, some of the incremental scorers came from above the bottom 28% -- although not too many. <br /><br />It would be interesting to see if this law made any measurable increase in high scorers (above 30 or above 25) and which race they most came from. I'm betting offhand that the law rousted more semi-smart white kids into taking the ACT than other races. Probably kids who were set on going into the Army and had done well on the AFQT, that kind of thing.<br />Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-61696899591278050192012-10-04T10:12:23.012-07:002012-10-04T10:12:23.012-07:00My attempt to punish Arizona was obviously excessi...My attempt to punish Arizona was obviously excessive, but the score drops with state test standard adoption do not strike me as minor. Here are the significant, single-year participation changes: Colorado’s ACT participation went from 62% to 99% in 2002, and its ACT score fell from 21.5 to 20.1. (SAT-ACT composite fell from 501 to 472.) Illinois went from 71% to 99% in 2002 with a drop from 21.6 to 20.1 (498 to 464). Kentucky went from 72% to 100% in 2009 with a drop from 20.9 to 19.4 (478 to 444). Michigan went from 70% to 100% in 2008 with a drop from 21.5 to 19.6 (492 to 448). Utah went from 73% to 97% in 2012 with a drop from 21.8 to 20.7 (493 to 469). Maine SAT participation went from 73% to 100% in 2007, and its SAT-ACT composite fell from 501 to 469. Delaware went from 74% to 100% in 2012, and its SAT-ACT composite fell from 492 to 465. In fact, Arizona’s ACT participation more than doubled between 2009 and 2011 from 15% to 34% with a similar drop from 21.9 to 19.7. (The SAT-ACT composite went from 508 to 474 as SAT participation went from 26% to 28%.)nooffensebuthttps://www.blogger.com/profile/02461190919466049463noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-11535605867679947082012-10-04T01:21:17.376-07:002012-10-04T01:21:17.376-07:00Here's an idea -- look for discontinuities in ...Here's an idea -- look for discontinuities in annual participation rates in states that changed the law and see what impact that had on scores. If a state goes from 60% to 100%, in one or two years, what does that do to average scores?<br /><br />My impression is that the impact is surprisingly minor, perhaps because the really dumb kids already dropped out of high school. Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-4321924039430181082012-10-03T21:02:32.481-07:002012-10-03T21:02:32.481-07:00You have a good point, but you could just ignore m...You have a good point, but you could just ignore most of the states. If you look at the states that adopted one of the tests for universal or near-universal testing, Colorado, Illinois, and Maine are the best of the twelve. Mississippi is worst. Arizona has the same score as the best with a small fraction of the participation. Therefore, Arizona deserves to be "punished." I do wish that I had a better way to account for participation differences. The trend is towards adoption of state standards, so the problem could resolve itself.nooffensebuthttps://www.blogger.com/profile/02461190919466049463noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-36134831204048337292012-10-03T18:20:07.251-07:002012-10-03T18:20:07.251-07:00Okay, thanks.
But, I don't really believe th...Okay, thanks. <br /><br />But, I don't really believe that if every teenager in Texas took the SAT or ACT they'd score 478 on the SAT but in Arizona they'd score 295. A 101 participation rate in Texas means something like 60% took the ACT and 41% took the SAT, but the percentage that took either one is something like 65%. I'm pretty baffled how to adjust for this.<br /><br />The bigger question an adjustment could be used for is the national trends over time.Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-62604829058058229612012-10-03T08:12:06.808-07:002012-10-03T08:12:06.808-07:00I think the weighting formula that I used for the ...I think the weighting formula that I used for the composite scores effectively controlled for most of the effect of students who take both tests, especially in states that have a clear preference. It could be improved slightly with a complex non-linear formula for ACT conversion that conforms to the conversion table. The effect of scraping the bottom of the barrel can be mostly controlled by either ignoring states that have very high or very low participation or by only looking only at the 12 states (and growing) with universal (or near universal) testing. Many of the latter group are adopting the ACT as a high school exit exam.<br /><br />State SAT-ACT Composite, Participation Controlled, Participation <br />Alaska 496 441 89<br />Alabama 461 434 94<br />Arkansas 459 423 92<br />Arizona 476 295 62<br />California 501 400 80<br />Colorado 478 478 117<br />Connecticut 514 514 115<br />DC 457 457 114<br />Delaware 465 465 114<br />Florida 467 467 136<br />Georgia 479 479 133<br />Hawaii 485 451 93<br />Iowa 499 329 66<br />Idaho 497 433 87<br />Illinois 474 474 105<br />Indiana 498 498 101<br />Kansas 497 432 87<br />Kentucky 451 451 106<br />Louisiana 462 462 109<br />Massachusetts 525 525 112<br />Maryland 498 473 95<br />Maine 476 476 102<br />Michigan 456 456 104<br />Minnesota 517 419 81<br />Missouri 490 392 80<br />Mississippi 425 425 104<br />Montana 506 450 89<br />North Carolina 497 437 88<br />North Dakota 467 467 103<br />Nebraska 497 413 83<br />New Hampshire 525 493 94<br />New Jersey 509 499 98<br />New Mexico 461 406 88<br />Nevada 486 403 83<br />New York 499 499 119<br />Ohio 500 450 90<br />Oklahoma 469 399 85<br />Oregon 505 479 95<br />Pennsylvania 497 457 92<br />Rhode Island 494 405 82<br />South Carolina 470 470 130<br />South Dakota 492 413 84<br />Tennessee 453 453 110<br />Texas 478 478 101<br />Utah 469 469 103<br />Virginia 508 493 97<br />Vermont 519 503 97<br />Washington 521 412 79<br />Wisconsin 500 375 75<br />West Virginia 471 400 85<br />Wyoming 460 460 105nooffensebuthttps://www.blogger.com/profile/02461190919466049463noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-42485519137783658922012-10-02T18:47:17.952-07:002012-10-02T18:47:17.952-07:00Another graph would be a line chart over the years...Another graph would be a line chart over the years of the SAT/ACT composite adjusted for participation. <br /><br />A big methodological issue is that both SAT and ACT participation has been increasing for two reasons:<br /><br />- Scraping the bottom of the barrel harder<br /><br />- Testtakers trying both tests to see if they'll do better on one than the other.<br /><br />This is a big problem for disentangling unless the two organizations have shared data in a specific study. <br /><br />For example, to eliminate the effect of bottom of the barrel scraping, we could look at absolute numbers of high scores as they change over time. I'd be interested in seeing how the absolute number of people scoring 700 or higher (or its ACT equivalent) have changed over the years because surely virtually everybody who could score a 700 was taking one of the two tests. But, there has probably been a big increase in recent years of individuals scoring highly on both tests, which leads to doublecounting.<br /><br />Any suggestions for how to deal with this?Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.comtag:blogger.com,1999:blog-5002675950760488813.post-73249756103071310082012-10-02T16:21:02.981-07:002012-10-02T16:21:02.981-07:00Thanks.
I've been looking for a combined SAT...Thanks. <br /><br />I've been looking for a combined SAT/ACT study for a long time. <br /><br />However, the flashing maps are too much info to take in. Can you add table versions of the the last few maps, with just the latest year?<br /><br />You might compare these to NAEP scores to see if your adjustments are making sense.<br />Steve Sailerhttps://www.blogger.com/profile/11920109042402850214noreply@blogger.com