Polymer nanocomposites (PNCs) possess highly versatile characteristics, depending on the nanofiller properties such as its chemical composition, particle size, dimension, polydispersity, concentration, or surface functional groups. In comparison with micron-sized materials, the nanofiller having a large surface area facilitates stronger interaction with the matrix. In this work, various surface-functionalized nanodiamonds (sf-NDs) having hydroxyl, carboxyl, amino, and amide group were prepared, and dispersed into polycarbonate (PC) and poly(methyl methacrylate) (PMMA) polymers. The polymer nanocomposites (PNCs) which contain the ND content of 5 wt% were subjected to the measurements of mechanical properties such as hardness and Young's modulus by atomic force microscopy (AFM) nanoindentation. It was observed that the hardness and Young's modulus of the polymer nanocomposites depend on strongly the nature of functional groups. The amine or amide functionalization gives the high mechanical properties for both polymers. The interfacial interaction between sf-NDs and polymer matrices is an important factor determining the mechanical properties of the PNCs.