Here, we report computational studies of the SH3 protein domain interacting with various single-walled carbon nanotubes (SWCNT) either bare or functionalized by mimicking the proline-rich motif (PRM) ligand (PPPVPPRR) and compare it to the SH3-PRM complex binding. With prolines or a single arginine attached, the SWCNT gained slightly on specificity when compared with the bare control, whereas with multi-arginine systems the specificity dropped dramatically to our surprise. Although the electrostatic interaction provided by arginines is crucial in the recognition between PRM and SH3 domain, our results suggest that attaching multiple arginines to the SWCNT has a detrimental effect on the binding affinity. Detailed analysis of the MD trajectories found two main factors that modulate the specificity of the binding: the existence of competing acidic patches at the surface of SH3 that leads to "trapping and clamping" by the arginines, and the rigidity of the SWCNT introducing entropic penalties in the proper binding. Further investigation revealed that the same "clamping" phenomenon exits in the PRM-SH3 system, which has not been reported in previous literature. The competing effects between nanoparticle and its functionalization components revealed by our model system should be of value to current and future nanomedicine designs.