In Hedgehog (Hh) signaling, the seven-transmembrane protein Smoothened (Smo) acts as a signal transducer that is regulated by phosphorylation, ubiquitination, and cell surface accumulation. However, it is not clear how Smo cell surface accumulation and intracellular trafficking are regulated. Here, we demonstrate that inactivation of Hrs by deletion or RNAi accumulates Smo in the late endosome that is marked by late endosome markers. Inactivation of Hrs enhances the wing defects caused by dominant-negative Smo. We show that Hrs promotes Smo ubiquitination, deleting the ubiquitin-interacting-motif (UIM) in Hrs abolishes the ability of Hrs to regulate Smo ubiquitination. However, the UIM domain neither recognizes the ubiquitinated Smo nor directly interacts with Smo. Hrs lacking UIM domain still downregulates Smo activity even though to a less extent. We have characterized that the N-terminus of Hrs directly interacts with the PKA/CK1 phosphorylation clusters to prevent Smo phosphorylation and activation, indicating an ubiquitin-independent regulation of Smo by Hrs. Finally, we found that knockdown of Tsg101 accumulates Smo that is co-localized with Hrs and other late endosome markers. Taken together, our data indicate that Hrs mediates Smo trafficking in the late endosome by not only promoting Smo ubiquitination but also blocking Smo phosphorylation.