Rayleigh scattering off a Bose-Einstein condensate was studied. Exposing an elongated condensate to a single off-resonant laser beam resulted in the observation of highly directional scattering of light and atoms. This collective light scattering is caused by the coherent center-of-mass motion of the atoms in the condensate. A directional beam of recoiling atoms was built up by matter wave amplification.