CFED Scorecard

Stay Informed

Methodology

The Scorecard Framework

Methodology for Outcome Data
    Data Collection
    Data Sources
    Precision of Estimates
    Ranking Outcome Measures
    How States are Ranked and Graded
    Timeliness of the Data

Methodology for Policy Data
    Update in Policy Methodology
    Data Collection
    Data Sources
    Policy Ratings
   
New in the 2014 Scorecard
    Calculating Estimated Impacts

Errata

    

The Scorecard Framework

The 2014 Assets & Opportunity Scorecard provides a picture, from a household financial security perspective, of both how families in each state fare and the policies in place to improve outcomes. In developing the Scorecard's framework and measures, CFED draws upon its own hands-on experience in technical assistance and strategic policy design as well as the expertise of external advisors.

To present a revealing portrait of each state and the District of Columbia, CFED compiled 66 outcome measures and 67 policy measures organized into a five-issue area framework: Financial Assets & Income, Businesses & Jobs, Housing & Homeownership, Health Care and Education.

The 67 policy measures are identified by CFED as critical for promoting financial security and opportunity in a state. Taken together, these policies provide a comprehensive view of what states can do to help residents build and protect assets. They are, however, by no means the only policies that states could or should adopt to expand economic opportunity. These policies provide a starting point for individual states to consider in the context of the environment in their particular states.

Return to top

Methodology for Outcome Data

The Scorecard assesses states on the financial security and economic opportunity of households on 66 outcome measures. For each outcome measure, states are compared to each other and ranked. Measures in each issue area are averaged, ranked and graded. To download a spreadsheet of all outcome measures and ranks published in the 2014 Scorecard, click here.

Data Collection

The Scorecard draws on a wide range of data sources to produce a comprehensive picture of financial security and economic opportunity. Data collection for outcome measures included in the 2014 Scorecard took place between June and December in 2013, and most of the measures reference outcomes from 2011 to 2013.

Data Sources

Data on the 66 outcome measures primarily comes from publically available data sources, but data are also purchased and commissioned from private sources when data are not publically available. The U.S. Census Bureau is the largest source of data for the Scorecard – predominantly the American Community Survey (ACS) and the Survey of Income and Program Participation (SIPP) – but we also collect data from other federal agencies such as the Bureau of Labor Statistics. Private data purchased for the Scorecard include data on foreclosed and delinquent mortgages from the Mortgage Bankers Association and data on credit and debt from TransUnion.

All Scorecard measures show data at the state level; however, for 12 outcome measures, data are not available for every state due to issues related to sample size.

Precision of Estimates

A key data source for the Scorecard is the Census Bureau’s SIPP, a national survey that collects data on household wealth, assets and liabilities. While the SIPP is the only available data source that is large enough to produce estimates of net worth and asset poverty at the state level, the sample size for the SIPP is relatively small, particularly for smaller states and for sub-populations – such as households of color or single-parent households. A small sample size means that estimates produced from the data are more likely to be imprecise, which means that an estimate of household wealth may not accurately represent the true wealth that households in that state or population hold. More information concerning the precision of Scorecard data generated from the SIPP can be found here.

Ranking Outcome Measures

Each state is individually ranked for 54 of the 66 outcome measures. The state with the most desirable outcome is ranked 1st, and the least desirable is ranked 51st. For example, the state with the highest homeownership rate is ranked 1st. Similarly, because a low level of credit card debt is desirable, the state with the lowest average credit card debt is ranked 1st.

For measures where data are not available for every state, states are not ranked out of 51; instead, they are ranked out of the total number of states for which data are available. For example, data on liquid asset poverty is available for only 41 states; thus, the state with the least desirable outcome is ranked 41st.

Although data are rounded for presentation purposes, states are ranked based on the actual, unrounded values when available. For example, two states might have different ranks even though the measures appear to be the same. There are a few measures for which unrounded values are not available from the data source, and in those instances, states with the same value share the same rank. The next-best performing state is ranked as if the tie had not occurred. For example, if two states have a rate of college graduates with student loan debt of 69%, each is ranked 43rd, and the next state is ranked 45th.

States are not ranked on 12 outcome measures in the 2014 Scorecard. There are two reasons a measure is unranked:

  1. Insufficient Data: Data for specific states may be unavailable, either due to a small sample size, or because the margin of error for the data estimate is too large to publish or rank that outcome measure (see here for more information on estimates calculated from the SIPP). Using this methodology, if fewer than 35 states can be ranked for a given outcome measure, that outcome measure is not ranked.
  2. Clustering of Data: Two measures in the 2014 Scorecard are unranked because there is too little variation in the data between states to meaningfully compare outcomes between states. These two measures are Homeownership by Gender and Four-Year Degree by Gender, which measure the disparity between men and women. Because the difference between the outcomes of the best and worst states for both measures is relatively small, CFED chose not to rank states on these measures.

How States Are Ranked and Graded

To calculate a state’s overall outcome rank, the state’s ranks for each individual outcome are averaged to generate an overall score. The lower the overall score, the better the state's overall performance in the Scorecard. The overall score for the states is then ranked from 1 to 51.

In prior iterations of the Scorecard, overall outcome ranks were calculated by averaging each state’s issue area ranks. This ranking methodology was designed to weight each issue area equally, but in the new methodology, all outcome measures are now weighted equally. The methodology has been updated in an effort to align the overall policy and outcome rankings.

Individual issue area outcome ranks and grades are calculated using roughly the same methodology as overall ranks – individual measure ranks are summed and averaged within the issue area to generate an score for that issue area, upon which the states are ranked. Issue area grades are assigned on a curve: states that rank from 1 to 10 earn an A; from 11 to 20 earn a B; from 21 to 36 earn a C; from 37 to 46 earn a D; and from 47 to 51 earn an F.

Notes on the Ranks:

  • A state is not penalized in the issue area ranks if data is missing for that state. A state’s issue area rank is calculated by taking the average rank of every outcome measure available for a state in that issue area. For example, North Dakota only has data for ten out of twelve outcome measures in the Education issue area, and as a result, their issue area rank is calculated using the ten available measures instead of twelve.
  • When two states receive the same overall score or average issue area rank, each state receives the same rank, and the next-best performing state is ranked as if the tie had not occurred. For example, if two states have the best score, each is ranked 1st, and the next state is ranked 3rd.

Timeliness of the Data

Some outcome measures in the Scorecard are collected from sources that are updated quarterly or annually (such as foreclosure and unemployment), while other measures are only available every two to five years (such as unbanked households and business ownership by race and gender). Moreover, even recently released data can reference a time period that is several years in the past. Because of this lag, even though the Scorecard draws on the most recent data available at the time of production, it is inevitable for some data to become out-of-date rather quickly. These inherent data issues limit the Scorecard's ability to reflect recent changes that have taken place in the economy. However, such issues do not diminish ability of the Scorecard to tell a story of relative performance of states when compared to each other during the same time period.

Return to top

Methodology for Policy Data

Update in Policy Methodology

In prior Scorecards, CFED assessed states on the strength of 12 “policy priorities,” each of which had four sub-criteria. The Scorecard reported on other policies, which were called “additional policies,” but it did not judge whether these policies met a particular threshold. In 2014, the Scorecard treats all policies equally. Most of the 48 “criteria” became separate policies and many of the “additional policies” are evaluated, rather than just reported. Several new policies were also added to the Scorecard based on input from issue-area experts and Lead Organizations in the Assets & Opportunity Network. As a result, a total of 67 policies are evaluated in the 2014 Scorecard. By expanding the breadth of the policy measures, the Scorecard now provides a clearer picture of the overall strength of each state’s policies.

For presentation purposes, many policies are grouped together for ease of understanding—e.g., all the policies related to a state Earned Income Tax Credit are displayed on a single web page. Regardless of how the policies are grouped, each policy is given equal weight—i.e., one point if the state has adopted the policy and zero points if the state has not adopted the policy.

Each of the 67 policies is drawn from the five issue areas in the Scorecard: Financial Assets & Income, Businesses & Jobs, Housing & Homeownership, Health Care and Education. None of these policies alone is the silver bullet solution, but each contributes to building the financial security of residents. The Scorecard presents a menu of policy options for what state policymakers can do to provide financial security and opportunity for residents. States can consider these 67 policies as the starting point for a proactive asset policy agenda and pursue those policies that reflect the particular needs of their state. 

Data Collection

Data collection for the policy priorities in the Scorecard took place between June and September, 2013. With a few exceptions, the 67 policy priorities reflect policies adopted as of the end of September 2013. 

Data Sources

Policy data for the Scorecard are drawn from research and resources created mainly by policy organizations, academic institutions and think tanks with expertise in the specific issue areas covered in the Scorecard. The policies selected for inclusion in the Scorecard and the criteria for assessing the strength of those policies were identified via one-on-one phone conversations with national intermediaries, funders, issue area experts and researchers; conference calls with a Scorecard Policy Advisory Group made up of nine Lead Organizations in the Assets & Opportunity Network; and data reported from 66 Lead Organizations about their policy priorities. The policy measures span the five issue areas and are either promising or proven in helping families build and protect assets. Where information on the state-by-state status of policies was not already documented by an external organization, CFED consulted with experts and conducted original research, which included state-by-state surveys and interviews and using LexisNexis’ state legislative tracking service.

Policy Ratings

Unlike the outcome measure rankings, which compare states to one another within each measure, individual policy measures assess states on a binary scale – yes or no. As such, there are no rankings within policy measures. Instead, states are ranked on the number of policies each has adopted, relative to the other 50 states, within each of the five issue areas and overall. Both the overall and issue area ranks are derived by calculating the sum of all policies each state has adopted as a percentage of all policies for which data was available. As a result, states are not penalized for a lack of available data, and are instead assessed on a relative scale. The state with the highest percentage of policies adopted is ranked 1st; the state with the lowest percentage is ranked 51st. Like the outcome measure rankings, when two states receive the same overall score or average issue area rank, each state receives the same rank. The next-best performing state is then ranked as if the tie had not occurred.

Return to top

New in the 2014 Scorecard

In addition to the expansion of the policy data and the new policy rankings, the 2014 Scorecard now includes one new outcome measure that was not included in the 2013 edition: High School Graduation Rate. Four outcome measures – net worth by race, income, gender and family structure – were removed, as the availability of state-level data was too inconsistent to warrant continued inclusion in the Scorecard. These data are still available, by request, for some states, and at the national level. Please contact research@cfed.org for more information.

Calculating Estimated Impacts

In the 2014 Scorecard, we have calculated the estimated impact of an improvement in a state’s performance to match the performance of the top ranking state on a select number of measures. For example: If Ohio improved its homeownership rate (66%) to match the homeownership rate in the best performing state, West Virginia (72%), how many more homeowners would there be in Ohio?

The estimates were calculated as follows: for each indicator, the rate of the best performing state was multiplied by the appropriate population (households, adults over 25, labor force, etc.) in the remaining states. The difference between the states’ improved performance and actual performance was then calculated. There are 3,019,382 homeowners in Ohio. If the state’s homeownership rate improved to West Virginia’s, there would be 3,278,420 homeowners – an estimated impact of 259,038 additional homeowners.

Return to top

Errata

Financial Education in Schools

States were given credit if they receive an "A" grade for financial education in Champlain College’s National Report Card on State Efforts to Improve Financial Literacy in High School. An “A” grade indicated the state requires a standalone personal finance course, or the state requires that personal finance topics be taught as part of another mandatory course and that students’ personal finance knowledge be assessed.  

Although the report does not give Maryland an "A" grade, the Maryland State Department of Education has indicated that the state requires personal finance topics be integrated into a required course and that students' personal finance knowledge be assessed. For more information about financial education requirements in Maryland, see the State Department of Education's financial education website: msde.state.md.us/fle/.

As of March 13, 2014, Maryland is given credit for requiring financial topics be taught and assessed.  Maryland continues to rank #1 in overall policies and education policies. However, the total number of policies Maryland has adopted increased from 40 to 41, and the number of education policies adopted increased from 10 to 11. As a result, New Jersey, which was tied with Maryland for first place on education policies, now ranks #2. The policies adopted in Maryland and New Jersey have been updated in all of the custom reports currently available for download online. However, the following is a list of errors in documents printed prior to March 13, 2014.  

State Profiles: 

  • In the state profile for New Jersey, the policy ranking for the issue area, Education, was listed as 1. It should be 2.
  • In the Maryland state profile, the number of education policies adopted was listed as 10. It should be 11.
  • In the Maryland state profile, the total number of policies adopted was listed as 40. It should be 41.

Policy Briefs:

  • In all Financial Education Policy Briefs, the number of states cited as requiring personal finance topics be taught and assessed was seven. It should be eight.
  • In the Financial Education Policy Brief specific to Maryland, the final section, “What Has Maryland Done?” indicates that Maryland does not require personal finance topics be taught and assessed. The section should indicate Maryland does require personal finance topics be taught and assessed.

Return to top

Copyright © 2014 CFED — Corporation for Enterprise Development 1200 G Street, NW Suite 400 Washington, DC 20005 202.408.9788

Powered by ARCOS