Although several studies showed the positive effects of exercise and physical activity on health and well-being for individuals with ID, there is a lack of information about the influence of sport specialization on fitness and health components. Therefore, the aims of this study were to assess: (a) physical fitness of athletes with intellectual disability (ID) compared with individuals included in recreational and leisure activity programs (non-athletic people); (b) contribution of sport specialization on athletes’ fitness; and c) correlation of each fitness variable with subjects’ ID levels. Twenty-two track and field, 19 basketball, and 23 non-athletic adults were recruited. Before and after a 9-month period, all participants performed fitness tests assessing body composition, flexibility (SR), arm muscular strength (HG), lower and upper-body muscular strength and endurance (SUP and PUP), explosive leg power (SLJ), cardiovascular endurance (ST), balance ability (FT), motor coordination (TUGT). The results showed that participants’ weight, BMI and FT were significantly affected by time; SLJ by activity; ST, HG, PUP, SUP, and TUGT by both time and activity. Only track and field athletes increased significantly ST. All athletes improved significantly HG, PUP and SUP, instead non-athletic people decreased significantly SUP (p <0.01). TUGT improved significantly in track and field athletes (p <0.05), and decreased significantly in non-athletic people. ID level was positively correlated to TUGT. Findings of this study showed that physical activity improved fitness in adult athletes with ID, decreasing health risks. Athletes with lower ID obtained higher performance scores in motor coordination test.