What determines the similarity of musical rhythms? According to the “family” theory, which this paper presents, one factor is the temporal sequence of the onsets of notes: rhythms with the same pattern of interonset intervals tend to sound similar. Another factor is meter. It determines whether or not rhythms are members of the same family, where families depend only on three types of possibility for each metrical unit. If the beat is the relevant metrical unit, these three possibilities are: 1) a note starts on a beat and therefore reinforces the meter, 2) a syncopation anticipates the beat and lasts through its onset and therefore disturbs the meter, and 3) all other events such as rests or ties that start on the beat provided no syncopation anticipates them. Two experiments showed that similarity between rhythms depends on both their temporal patterns of onsets and their families, which combined give a better account than edit distance – a metric of the distance apart of two strings of symbols. Two further experiments examined the errors that participants made in reproducing rhythms by tapping them. Errors more often yielded rhythms in the same family as the originals than rhythms in a different family.