Synthetic antibody libraries with restricted chemical diversity were used to explore the intrinsic contributions of four amino acids (Tyr, Ser, Gly and Arg) to the affinity and specificity of antigen recognition. There was no correlation between nonspecific binding and the content of Tyr, Ser or Gly in the antigen-binding site, and in fact, the most specific antibodies were those with the highest Tyr content. In contrast, Arg content was clearly correlated with increased nonspecific binding. We combined Tyr, Ser and Gly to generate highly specific synthetic antibodies with affinities in the subnanomolar range, showing that the high abundance of Tyr, Ser and Gly in natural antibody germ line sequences reflects the intrinsic capacity of these residues to work together to mediate antigen recognition. Despite being a major functional contributor to co-evolved protein-protein interfaces, we find that Arg does not contribute generally to the affinity of naïve antigen-binding sites and is detrimental to specificity. Again, this is consistent with studies of natural antibodies, which have shown that nonspecific, self-reactive antibodies are rich in Arg and other positively charged residues. Our findings suggest that the principles governing naïve molecular recognition differ from those governing co-evolved interactions. Analogous studies can be designed to explore the roles of the other amino acids in molecular recognition. Results of such studies should illuminate the basic principles underlying natural protein-protein interactions and should aid the design of synthetic binding proteins with functions beyond the scope of natural proteins.