Cerebral palsy (CP) is a perinatally acquired nonprogressive brain damage resulting in motor impairment affecting mobility and posture. Early identification of infants with CP is desired to target early interventions and follow-up. During early infancy, distinct motion patterns occur which are highly predictive for later disability. These motor patterns can be observed and recorded. In this paper, a method to predict later CP based on early video recordings of the infants' spontaneous movements, applying optical flow and statistical pattern recognition, is presented. We extract motion information from video recordings of young infants using a total variation related optical flow method. By using wavelet analysis features from motion trajectories of points initiated on a regular grid were extracted and classified using a support vector machine.