Finding Evidence of the Student Grant Effect in a Natural Experiment


Abstract: This study reviews the period 1984-1989 during which the Federal Republic of Germany eliminated grants in favor of loans for financially needy higher education students. A time series analysis shows statistically significant declines in enrollment rates among four social groups for the duration of the no-grant policy. The study is noteworthy for the United States in that the conditions of this "natural experiment" eliminate variables with which researchers have had difficulty in recent attempts to measure the student grant effect. The study also suggests that grants are important to higher education access even in an environment of generous student loan repayment conditions.

Grants to financially needy students to make college affordable have been a staple of U.S. public policy toward higher education for the past quarter century. Federal, state, and institutional grants have increased many-fold since the 1960's to try to achieve higher education opportunity for all who qualify academically, not just those who can afford it. For the preceding generation, grants based on service to one's country, in the form of the GI Bill, were a centerpiece of national economic policy.

It would seem grants are here to stay. Nonetheless, doubts surround the efficacy of grants. Some individuals both in and out of the higher education community argue for lower tuition instead of higher grants. Some argue for loans to students instead of grants, since loans are largely paid back and therefore cost less. Some have questioned if grants really work, citing researchers who have been unable to find empirical evidence that expansion of grants has led to expansion of college access.

The purpose of this paper is to look at the grant question empirically, sorting out the tuition and loan questions to concentrate directly on the efficacy of grants as a means of providing access for students with financial need.

Research Background

Research on higher education access varies widely in techniques and conclusions. Leslie and Brinkman (1988) found support for the efficacy of both grants and low tuition, but not without having to sort through contradictory literature and relying on outdated studies. McPherson and Schapiro (1991) confirmed Leslie and Brinkman for the effect of grants on low-income students at private institutions, but found no statistically significant results at public institutions. U.S. Department of Education (1995) evaluations, attempting to update results using data from the 1980's and 1990's, have been inconclusive on the grant question. The Department relies on dated studies for evidence to support current federal grant policy on access.

Sorting out grants from loans is likewise difficult. The Department of Education (1995) noted that in comparison to grant research, "less work has been done on the effects of loans on postsecondary access.... Several studies...have yielded contradictory results." The Department has solicited proposals for experimental research in which samples of students would be exempted from current grant and loan limitations in order to assess the dimensions of grant and loan effects on access.

Theory and Methodology

Both the theory and methodology of this paper are straightforward. The incidence of college attendance among low-income individuals of college age should be affected by the presence or absence of grants, if grants indeed make a difference. The methodology relies on identifying "natural experiments" where this can be observed.

It is difficult to find such natural experiments in the United States. Grants are not shut off and on like a spigot, they increase and decrease incrementally. Grants in the U.S. are not under the control of one authority. An increase in federal Pell grants, for example, can be offset by a decrease in state or institutional grants. Additionally, the effect of grants on net cost to a student is a function of tuition, which can go up to offset a grant effect or discounted to increase a grant effect. When grants are inadequate, loans are offered to students as a substitute, sometimes even as an equivalent. Grant effects may also differ by race, if much of the current research is to be believed.

Fortunately for purposes of studying the grant effect, the älte Länder (old states) of the Bundesrepublik Deutschland offer a simpler experimental opportunity. In the former West Germany, from 1984 through 1989, grants were eliminated totally from students' financial aid in favor of loans. In both preceding and following periods, aid was given to financially needy students as part grant, part loan. The German government (Bundesministerium 1995) has collected information on the rate of first-time college enrollment, by parental occupation, from 1982 through 1993, presenting the opportunity to look at twelve years (six years each) of a grant/no-grant dichotomy, by social group.

Germany is suitable for comparison with the United States in that both are western industrial democracies with federal forms of government that value access to higher education. Similarities abound, even down to the year--1972--in which the respective national governments passed their major programs designed to make higher education accessible regardless of financial need. Moreover, what makes Germany especially suitable as a natural experiment is the absence of variables that make study of the grant effect difficult in the United States:

--In Germany, higher education is virtually all public sector;
--German higher education institutions do not charge tuition, so there is no need to calculate tuition, net of aid;
--With no tuition, there is no occasion for institutions to raise tuition to provide institutional student aid;
--Government student aid is unified in one program, Bafög, a federal/state, 65%/35% matching effort, limiting fungibility among grants and other items in a budget;
--The German population is relatively homogeneous racially.

The cost of university attendance in Germany during the 1984-89 period in question was not markedly different from that in the United States. In Germany, the costs were living expenses, books, supplies, and fees other than tuition. In the U.S., tuition at public institutions added, on average, approximately one-fifth more to these costs. Johnstone (1986) puts the respective figures for 1985-86 at $4398 versus $5314. Because German students are assigned to universities where there are openings, however, about one-quarter fewer German students than U.S. students were able to cut costs by living with parents. The German housing market is difficult; many German first-year students fail to find housing and return home. If forgone earnings are also compared as a cost of attendance, here the cost to German students would also likely be more, since German students entering universities have completed their Abitur, an educational accomplishment beyond a U.S. high school diploma.

The four social groups on which access data are collected in Germany are Selbständigenkinder, Beamtenkinder, Angestelltenkinder, and Arbeiterkinder, or what this paper will call, respectively, the children of self-employed, civil servant, employee/staff, and working class parents. How these classifications are determined is of no particular consequence to this study, but suffice it to note that they are used in Germany to judge progress over time in making higher education accessible to children of different social backgrounds. These Sozialgruppen, however, are not strictly hierarchical among themselves, and are not synonymous with social origins (Herkunftsfamilien) or income classifications because within the groups there can be considerable variation in income as well as status. For example, civil servants can be at low, middle, and high levels. An exception may be the working class, which the federal ministry suggests is more homogeneous than the others and to which it pays the most attention for access. Despite the attention to the working class, because financially needy students come from each of the four groups, grant effects in this study should be apparent, if they exist, in all four groups.

The specific hypothesis of this study is that higher education access is lower in each of the four groups, individually and in the aggregate, during the six years that grants were eliminated from the student aid program. The statistical significance level is set at .05 for all tests of the hypothesis.

The methods used in this paper to test for the grant effect are difference of means tests, analyses of variances, and interrupted time-series multiple regressions applied to each of four groups, and to the groups in aggregate, on which the federal ministry of education collects enrollment information. Higher education access, as measured by the German federal education ministry, is the rate of first time attendance in universities and in Fachhochschulen (technical institutes) by students in each group between the ages of 18 and 22. Financial need in Germany is measured much as it is in the United States, by a test of student and parental income and assets. Grant amounts are comparable to amounts in the U.S.

This study design may risk a so-called ecological fallacy, inasmuch as the four groups include non-needy as well as needy students, in a ratio of about three to one. It is possible that the presence or absence of grants has no effect on the financially needy, and that observed differences are in the non-needy populations. This seems highly unlikely, however, and in any case the hypothesis does not specify that it is the financially needy who suffer lower access, although such an assumption is implicit in the study design.

Data and Statistical Analysis

Table 1 lists the higher education access data and identifies the years in which the Bafög program included grants. These data were not collected prior to 1982.


Figure 1 compares average enrollment rates under grant and no-grant conditions for first year students by social group.


Table 2 shows the results of difference of means tests by financial aid policy for each social group. The p-value represents the statistical probability that the groups were not affected by the change in policy. The probability is less than .05 for each group, rejecting the null hypothesis (that there is no statistical relationship between access and grants).


Figure 2 takes the analysis one step further in displaying not the mean differences, but the variance of the enrollment rates in the form of box plots. In the box plots, the end points represent the two extremes, the ends of the boxes represent the interquartile range between the twenty-fifth and seventy-fifth percentiles, and the middle line represents the fiftieth percentile, or the median. Enrollment rates appear here on a logarithmic scale to highlight the working class.


Table 3 shows analysis of variance tests, which compare dispersions of the observations within each group to determine the statistical probability that the policy differences had no effect. While the difference in means tests use only the information displayed in the figure 1 bar graph, the analysis of variance tests make use of the information displayed in the figure 2 box plot. Only in the case of self employed is there insufficient variance to pass this test of statistical significance.


The two statistical tests above have not, however, ruled out other explanations for the observed differences in access. They permit only the acceptance or rejection of the null hypothesis on the basis of statistical probabilities. They do not confirm that the Bafög policy change is the cause of the differences where statistical significance is present. The decline in access during years when grants were not available could have been caused by other factors, such as changes in economic conditions, admissions policies, or job market perceptions, for example. The decline could also have been caused by other changes in the Bafög program itself that were implemented around the same time: although 1984 marked the first year of full conversion from grants to loans, there were grant cuts in the two previous years.

An interrupted time-series, multiple regression analysis is one way to control for such factors in order to be more certain that it was the full elimination of grants that caused the enrollment rate differences.

In the multiple regression equation Y = a + bXc + bXd + bXp + e, the dependent variable, Y, is the enrollment rate, a is the Y-axis intercept where the values of the independent X variables are zero, Xc is a counter or linear control variable (1982=1, 1983=2...), Xd is a dummy variable (0 for grants, 1 for no-grants), Xp is a post-policy variable (1982=0... 1984=1... 1989=6, 1990=0...), the b's are coefficients of the independent X variables, and e is the error term or variance in the actual observation of the enrollment rate that remains unexplained.

Figure 3 shows a chart of two different regression slopes for the children of working class parents: one for the years when the no-grant policy was in effect (1984 through 1989) and one for the years before and after. The table that accompanies the chart shows, among other calculations, the coefficients of the three independent variables and their p-values.

The coefficient of the counter variable indicates that the underlying enrollment rate of working class children increased approximately half a percent per year. This is the baseline rate given the social, educational, and economic conditions at the time, from which to measure the no-grant policy intervention. The coefficient of the policy or dummy variable indicates that the immediate effect of the no-grant policy intervention was a drop of nearly 2 percent. The coefficient of the post-policy variable shows that subsequent enrollment rate growth for the next five years was .288 below the baseline rate. These values, when graphically displayed, give a good picture of the effect of the no-grant policy on the children of the working class.


The graphic, however, cannot be fully interpreted without the help of the p-values. The p-value in figure 3 for the policy (dummy) variable indicates that other factors should not be ruled out at the .05 level to account for the observation of the immediate drop in the enrollment rate. Furthermore, the p-value of the post-policy variable is too high to conclude that the slope of the no-grant line is significantly different from the slope of the grant line.

Figures 4, 5, and 6 display the results for the other social groups. In each of these groups, the dummy policy variable is statistically significant at the .05 level, but the post-policy variable is not. In other words, the enrollment rate drop can be attributed to the no-grant policy intervention, but the policy did not continue to discourage enrollment even further in subsequent years.







Comparing the multiple regression analyses of all four groups, two questions come immediately to mind: why no statistical significance for the policy (dummy) variable in the working class group, and what can be concluded from the slopes of the post-policy lines?

Not too much should be made of the lack of statistical significance for the policy variable in the working class group. The small number of years studied makes significance harder to reach. Anecdotal evidence for the years 1994 and 1995 suggests that a longer view with two more observations will push the p-value below .05. Nevertheless, the reason the policy variable is not significant in this test, as opposed to the others performed on the same working class data, is that the policy intervention did not affect the enrollment rate as much when compared to the underlying change in the rate, which the other tests did not include. Interestingly, the multiple regression technique found statistical significance for the policy variable in the self employed group, suggesting that controlling for other trends in the time series for this group shows that the no-grant effect is stronger than suggested by the simpler analysis of variance test.

The slopes of the post-policy regression lines, despite their lack of statistical significance, should not be ignored. Children of the self employed seem to have recovered from the grant cut-off more rapidly than those of the other groups, and children of the working class had the most difficulty recovering. Children of the working class had the only negative coefficient for the post-policy variable, suggesting that the no-grant policy over the long term hurt higher education access for the working class more than it did others.

Finally, Figure 7 shows a multiple regression analysis for all four groups in the aggregate.


In the aggregate, the grant/no-grant dummy policy variable has a very strong p-value of .0038, meaning that it is statistically significant at the .01 level as well as the .05 level. The difference in the slope of the regression lines, as identified by the post-policy variable, is not statistically significant.

Conclusions and Implications

This natural experiment offers an exceptionally good opportunity to study the effects of grants both in the aggregate and by the social grouping of students' parents. The discrete on-off-on changes in grant policy make it unnecessary to design models to investigate the effects of incremental changes in grant policy. The fact that the natural experiment took place in the absence of both tuition and other grant programs eliminates the need to control for such factors. Recent research on grant effects has been inconclusive because the models tested have not had adequate controls (Oberg 1996a), making this study the more valuable.

This is likewise a good opportunity to look at the effectiveness of grants in a climate of favorable loan repayment conditions. German law on loan repayment is exceptionally generous, requiring repayment only of principal (not interest) as well as providing a five year grace period after leaving college and multiple opportunities for deferrals and partial cancellations (Oberg 1996b).

The primary conclusion of this study is that there is a strong grant effect on higher education access that cuts across all social groups. The "notches" in the graphical analyses above represent the loss to the country from excluding from higher education those who were academically qualified but financially unable to take advantage of it.

This conclusion, based solely on statistical analysis, is given added validity by 1985 survey research on enrolled students who previously had received Bafög support but turned down subsequent loans despite continued eligibility. The most frequently mentioned reason, representing 28% of all such reasons, was the reluctance to incur debt (Bundesminister 1986, 283). Nearly half of those who gave this reason gave it as the only reason for turning down the loan, substantiating the notion that a sizeable proportion of the population will not borrow to finance higher education. This Abschrekungseffect , as it applies to those who choose not to enroll at all, became a basis for a reversal of the no-grant policy four years later.

A second conclusion is less justified because the results are not statistically significant, but it seems doubtful that under the no-grant policy students would ever have caught back up to the enrollment rates associated with grants. Had grants not been restored after 1989, Germany ran the risk of both a widening gap of access by financial need and a widening gap between social groups at the same level of financial need. This is evidenced by the fact that in comparison to the other groups, the notch for the working class was growing larger with time, not smaller.

This study should be externally valid and applicable to other countries, including the United States, to the extent a country deals with the same questions of providing higher education access. According to these findings, tuition cannot be lowered, even to the point of elimination, to maximize higher education access. This study should also help answer a question posed by the Office of Management and Budget to the U.S. Department of Education in 1994: Are grants necessary for access if loans can be repaid on an income-contingent basis? Germany's repayment terms of the 1980s were at least as favorable as current U.S. income-contingent terms, yet access declined in Germany for lack of grants. And although this study has not dealt with the grant effect as it influences choice of institution (particularly important in the United States) and degree persistence (important everywhere), these findings should also be applicable to choice and persistence questions since such matters are aspects of access rather than separate issues.

It must be noted that Germany is in the process of changing access policies again, starting in the fall of 1996 (BMBF 1995, 1996). The Christian Democrat/Free Democrat coalition, in power since 1982, which advocated the 1984 elimination of grants only to reverse itself for 1990, is advancing a new access model. Some student loans will have to be repaid with market-level, variable interest, but grants will be increased, the cost of which will be shared by the national and state governments. States, old and new, are reconsidering tuition charges. Funds generated by these measures, above the cost of increasing grants, will in theory be spent on providing more spaces for students and upgrading academic quality. Protests against these policies, especially against tuition charges, have centered not so much on the equitability of a higher price, higher grant access model, but on fear that the funds will not be spent on higher education at all, or will be spent on sectors of higher education that will not benefit those who will pay more.

Finally, after reviewing all the procedures and calculations involved in this natural experiment, one may be left with the feeling that the findings make intuitive sense, that such findings are about what one would expect, so why go to the bother? The German government itself came to the conclusion that the restoration of grants would improve access, giving additional face-validity to this study. The point, however, is that recent empirical research on the grant effect in the U.S. is inconclusive if not counterintuitive. This study suggests that others' failures to find the grant effect may be rooted in faulty models. New research in the U.S. is needed, with new models to recognize the fungibility of grants in a multi-grant system and to recognize that the major U.S. student grant effort has at times operated with a significant institutional, not student, benefit. Such research is necessary to determine if the convincing anecdotal evidence of grants making a difference in U.S. students' lives actually has a stronger empirical basis that has been found to date. New research is likewise needed to avoid repeating the mistake of the German government, which had to reverse itself after making a poorly informed decision on the relationship of grants to higher education access.

References

BMBF (1996). "Stärkung der Hochschulausbildung und Struckturreform der Ausbildungsförderung." WWW: Diskussionsforen, Homepage des BMBF.

BMBF (1995). "Die Hochschulausbildung stärken - neue Spielraume für Zukunftsinvestionen schaffen." WWW: Diskussionsforen, Homepage des BMBF.

Bundesminister für Bildung und Wissenschaft (1986). Das soziale Bild der Studentenschaft in der Bundesrepublik Deutschland. 11.Sozialerhebung des Deutschen Studentenwerks. Bonn.

Bundesminister für Bildung und Wissenschaft (1988). Vorschläge zur Reform des Bundesausbildungsförderungsgesetzes (BAföG). Schriftenreihe 21. Bonn.

Bundesministerium für Bildung, Wissenschaft, Forschung und Technology (1995). Das soziale Bild der Studentenschaft in der Bundesrepublik Deutschland. 14. Sozialerhebung des Deutschen Studentenwerks. Bonn.

Johnstone, D.B. (1986). Sharing the Costs of Higher Education: Student Financial Assistance in the United Kingdom, the Federal Republic of Germany, France, Sweden, and the United States. New York: College Entrance Examination Board.

Leslie, L.L., and Brinkman, P.T. (1987). "Student Price Response in Higher Education: The Student Demand Studies." Journal of Higher Education, 58:2.

Leslie, L.L., and Brinkman, P.T. (1988). The Economic Value of Higher Education. New York: Macmillan Publishing Company.

Manski, C.F., and Wise, D.A. (1983). College Choice in America. Cambridge, Massachusetts: Harvard University Press.

Masten, C.L. (1993). "Empirical Evidence of the Effect of Tuition and Financial Aid on Postsecondary Educational Choices." Unpublished dissertation, The University of California at Davis.

McPherson, M.S., and Schapiro, M.O. (1991). Keeping College Affordable. Washington, D.C.: The Brookings Institution.

Oberg, J. H. (1996a). "Testing Federal Aid Fungibility in Higher Education." WWW: Homepage, Practicing Political Science.

Oberg, J.H. (1996b). "Student Loan Repayment Conditions in the Federal Republic of Germany." WWW: Homepage, Practicing Political Science.

U.S. Department of Education (1993). Financing Undergraduate Education: 1990. Draft Review Version. Washington: U.S. Department of Education.

U.S. Department of Education (1995). Biennial Evaluation Report. Washington: U.S. Department of Education.

U.S. Department of Education (1995). Digest of Education Statistics 1995. Washington: U.S. Department of Education.