Although the positive developmental effects of infrastructure provisioning are well documented, research on the potential role of governance in the improvement of infrastructure performance and individual-level service utilization is lacking. I explore the effect of infrastructure provisioning on individual-level health service utilization, paying close attention to whether governance at different levels shapes people's access to health care. The different geographical levels of infrastructure provisioning, governance, and health service utilization require a multilevel analysis, which I perform using Afrobarometer Round 5 survey data on 34 African countries in a three-stage mixed-effects modeling. Results show that the presence of health infrastructure is crucial for enhancing people's health service utilization. However, people encounter certain problems when receiving services at their local health clinics or hospitals, and these problems are directly linked with governance in the health sector as well as overall governance at the country level. Improvements in people's health service utilization therefore require both better infrastructure provisioning and better governance at different levels, as the former does not guarantee the latter. Development scholars need to widen their focus beyond national-level governance and help policy makers identify at which level state interventions are most needed for removing barriers to development.