Endogenous, ideas-led, growth theory and agent based modeling with neighborhood effects literature are crossed. In an economic overlapping generations framework, it is shown how social interactions and neighborhood effects are of vital importance in the endogenous determination of the long run number of skilled workers and therefore of the growth prospects of an economy. Neighborhood effects interact with the initial distribution of skilled agents across space and play a key role in the long run stabilization of the number of skilled individuals. Our model implies a tendency towards segregation, with a possibly positive influence on growth, if team effects operate. The long run growth rate is also shown to depend on the rate of time preference. Initial circumstances are of vital importance for long run outcomes. A poor initial education endowment will imply a long run reduced number of skilled workers and a mediocre growth rate, so there is no economic convergence tendency. On the contrary, poor societies will grow less, or will even fall into a poverty trap, and will diverge continuously from richer ones